How to handle moving of content from collections inside the Ansible package to collections outside the Ansible package?

[felixfontein]

Summary

Reference: ansible-collections/community.general#4149

In the past, we only allowed migration of content to other collections out of community.general (and community.network) if the destination collection was part of the Ansible package. While this is nice for users of the Ansible package (nothing will break), it is less nice for users of the individual collections - they now need to install another collection as well for the redirects to work. (Note that removal can only target major releases, so you can say "no surprise, since it's a major release".) Basically this is about that we promise to deprecate things before removing them. We usually do so, but we didn't properly do that in case of migrations.

Now that we have a request to move some content from community.general (part of Ansible) to community.sap (not part of Ansible): how should we deal with this?

[felixfontein]

I'm kind of torn. I don't like the idea of deprecating the modules first for a major version before replacing them with redirects (that would be one possibility) or even tombstoning them (another possibility). I don't like - though I like this more than the other possibilities! - the idea of simply adding redirects for the next major release, with now only an announcement in the changelog.

[tadeboro]

I think we as a community need to make sure users of the Ansible Python package see as little disturbance as possible since I feel that those users are the ones that want to see things work for as long as possible. Putting those users through painful migrations, in my opinion, dramatically increases the risk of those users moving to some other tool because they will see Ansible as unreliable or hard to use.

Consumers of individual collections already took part in the responsibility for maintaining their environment up-to-date and are thus better equipped to handle minor issues with simple solutions (for example, installing an extra collection or two).

So my proposal would be:

1. If the moved content will stay part of the Ansible package, we should install the redirects and remove content from the donor collection in the next major release. Having an announcement in the changelog should be sufficient in this case.
2. If the move takes the content outside the Ansible package, we should deprecate the modules as soon as possible and tombstone them just like any other deprecated module or plugin.

If we follow my proposal, "normal" Ansible users will only see warnings if they need to change their playbooks, while "advanced" Ansible users will need to install a new collection to keep things working when updating the major version of collections.

It would be nice to give Ansible users some heads-up signal when such changes happen, but we would probably need some help from the core to make this possible. For example, if we could have a redirect in the collection that would print a warning if the target collection is not installed and fall back to using the old plugin version from the donor collection. We could add a redirect in version X.0.0 and remove the content in version (X+1).0.0.

[felixfontein]

In other words, you propose we stick to what we did in the past :) Which is fine for me, your arguments are convincing.

It would be nice to give Ansible users some heads-up signal when such changes happen, but we would probably need some help from the core to make this possible. For example, if we could have a redirect in the collection that would print a warning if the target collection is not installed and fall back to using the old plugin version from the donor collection. We could add a redirect in version X.0.0 and remove the content in version (X+1).0.0.

I'm a bit vary of this, since observed behavior then can (even more than now) depend on whether another collection is installed or not.

I think it's best to stick to the following procedure (what exactly the third step should be needs some more discussion), as it's the most predictable and doesn't depend on potential new features that will only be supported by certain versions (if ever implemented):

1. Deprecate content without redirects for at least one complete major release cycle;
2. Remove content, have deprecated redirect for at least one complete major release cycle;
3. One of:
   a) Remove deprecation, but keep redirect;
   b) Remove both deprecation and redirect, but do not add tombstone;
   c) Remove both deprecation and redirect, and add tombstone.

[tadeboro]

It would be nice to give Ansible users some heads-up signal when such changes happen, but we would probably need some help from the core to make this possible. For example, if we could have a redirect in the collection that would print a warning if the target collection is not installed and fall back to using the old plugin version from the donor collection. We could add a redirect in version X.0.0 and remove the content in version (X+1).0.0.

I'm a bit vary of this, since observed behavior then can (even more than now) depend on whether another collection is installed or not.

I totally agree, but this was the only thing that I could come up with that was not total garbage ;) Also, I was having a specific developer in mind when I proposed this: the kind that does not tolerate warnings and fixes then ASAP. In this case, things should spend very little time in a state that could cause trouble.

I think it's best to stick to the following procedure (what exactly the third step should be needs some more discussion), as it's the most predictable and doesn't depend on potential new features that will only be supported by certain versions (if ever implemented):

1. Deprecate content without redirects for at least one complete major release cycle;
2. Remove content, have deprecated redirect for at least one complete major release cycle;
3. One of:
   a) Remove deprecation, but keep redirect;
   b) Remove both deprecation and redirect, but do not add tombstone;
   c) Remove both deprecation and redirect, and add tombstone.

For me, the only reasonable action after we do step 1 is to delete content and go directly to 3c. Anything else seems like a lot of work for marginal benefits in the best case (we inform users that something will be gone) or unintuitive behavior in the worst case (going with 3a).

So I would "vote" for keeping things as simple as possible and start complicating them once we know the simple approach is problematic.

[felixfontein]

Also, I was having a specific developer in mind when I proposed this: the kind that does not tolerate warnings and fixes then ASAP. In this case, things should spend very little time in a state that could cause trouble.

The best way to fix this is install the new collection and update your FQCNs. Obviously that will require some more work if you don't have a workflow for installing required collections, but if we plan to remove content that has been moved to a collection not included in Ansible this is needed sooner or later anyway for users of that content.

1. Deprecate content without redirects for at least one complete major release cycle;
2. Remove content, have deprecated redirect for at least one complete major release cycle;
3. One of:
   a) Remove deprecation, but keep redirect;
   b) Remove both deprecation and redirect, but do not add tombstone;
   c) Remove both deprecation and redirect, and add tombstone.

For me, the only reasonable action after we do step 1 is to delete content and go directly to 3c. Anything else seems like a lot of work for marginal benefits in the best case (we inform users that something will be gone) or unintuitive behavior in the worst case (going with 3a).

I think right now, we mostly used 3a for content moved to another collection inside the Ansible package (looking at meta/runtime.yml in community.general).

On the other hand, this has mostly been for content that has been migrated from Ansible 2.9 and thus has a shortname. Without undeprecated redirects, these shortnames stop working. For content that has not been moved from 2.9 (and never had a shortname), 3c sounds more reasonable to me as well.

[felixfontein]

Let's continue this discussion in #167. If we have a solution for community.general and community.network, we can try to generalize it to all collections. This might be more successful than trying to solve the general case first.