Require and validate role version

[chouseknecht]

From @gregdek on July 11, 2016 19:14

Time to start figuring this out. Unless there's significant backlash, because we live in the Python world here at Ansible, we're likely to implement PEP-440 for versioning:

https://www.python.org/dev/peps/pep-0440/

Copied from original issue: ansible/galaxy-issues#168

[chouseknecht]

From @geerlingguy on July 11, 2016 20:22

Yeesh, that reads like an IETF RFC... is there a good comparison/contrast between PEP-440 and Semantic Versioning? It seems like at least the core components of semver would be acceptable (e.g. [major].[minor].[patch]).

Edit: I did see a Semantic Versioning section. Looks like the core part is fine, but tags like 1.0.0-beta1 would not be (that would presumably be 1.0.0.b1?).

[chouseknecht]

From @gregdek on July 11, 2016 20:42

Notable: this is the mechanism for PyPi versioning, and this spec was written by the author of PyPi.

[chouseknecht]

From @geerlingguy on July 13, 2016 20:54

@gregdek - I know you closed ansible/galaxy-issues#165 in favor of this issue, but would this issue technically cover some sort of locking mechanism as well? E.g. If I wanted the latest of a "3.1.x" branch (and could choose 'stable', 'dev', 'alpha' etc.), is that the desired outcome of this issue? Or is it more just agreeing to or documenting the use of PEP 440?

If the latter, could we leave the Lockfile/version pinning issue open?

[chouseknecht]

From @gregdek on July 13, 2016 21:49

We can reopen if you like, sure. I suppose they are separate issues.

[chouseknecht]

From @goozbach on July 27, 2016 21:32

I'd vote for limited semvers as defined in pep-440 but I have simple needs and don't know most edge cases.

[chouseknecht]

From @bcoca on July 27, 2016 21:57

We already allow 'commit sha' as a 'role version'

[grahamlyons]

I'd really like to see immutability of versions published on Ansible Galaxy. The lockfile, mentioned here and on a separate issue, means nothing if the locked version can be replaced.

[geerlingguy]

@grahamlyons - doing that would require Galaxy establishing it's own permanent cache of all role versions, since right now Galaxy passes through the download to GitHub... and GitHub allows tags to be removed/re-pointed. I'm not sure if that's something that would be worth the cost of the extra infrastructure/bandwidth demands :/

Having a cache would be nice (especially if you wanted to build a local Galaxy cache for isolated networks, that sort of thing), but having a permanent cache that is used for all tag version downloads (effectively taking GitHub out of the loop as the source of truth) sounds to me like it might not happen.

[grahamlyons]

@geerlingguy, thanks for the explanation - I hadn't realised that assets were actually hosted on Github.

Perhaps encouraging the use of tags for released versions by default could be a good compromise? At least that way it's less likely that you'll pull in changes from master that you weren't expecting.

[joshmoore]

NB: we've just been surprised by this issue with https://galaxy.ansible.com/openmicroscopy/omero-server/

2.0.0-m4 is downloading rather than than the newer 2.0.0

cc: @sbesson

[chouseknecht]

Adding this to the 3.1 release. Here's what I think we want to do:

As we move toward integration with Pulp, Pulp expects content to be versioned. Once a version of content is loaded into a Pulp repository, that version is immutable. Pulp will always serve the exact same result for a given content version. If you want to make a change, then you have to increment the content version, and import the new version into Pulp.

Galaxy should follow this same pattern. It should require content repositories to be versioned, and enforce version immutability. This will make the integration with Pulp easier, and it will make Galaxy a much more reliable package manager.

Here are the specific tasks to be done:

• Make the import process version aware. Imports should be associated with versions, and once you import a version, that version can no longer be imported. Import once, and done. If you want to make changes, then follow the semantic versioning spec, increment the version number, and import the new version.
• Modify the import process to require content repositories to have versions
• Versions will continue to come from GitHub tags
• Drop leading v or V on tags
• Validate that GitHub tags, minus any leading 'v' or 'V', follow the semantic version format
• Insure that Galaxy only imports commits associated with a version. Commits may occur after the latest release, in which case Galaxy should ignore them.
• Insure that when versions are returned by the API, or presented in the UI, they are sorted correctly, in descending order.
• Insure that ansible-galaxy install defaults to the latest release or tag . If tags are sorted in correct descending order, the default tag will be at the top of the list.

Legacy content

Also, the idea is to apply this to new imports. Legacy content that isn't being actively maintained won't be affected. When we get to integrating with Pulp, we'll figure out a way to associate old content with a git commit hash, and leave it alone. If a change does happen to appear for such content, then the import process will reject it. The author will have to add a version to get the new release imported.

[bcoca]

related ansible/proposals#23