Transition of ansible modules for podman to a new place

[sshnaidm]

Hi, all libpod devs and contributors

I'd like to raise a topic about podman/buildah Ansible modules. If this is not the best place to do it - please point me to the right one.

Preface:
Ansible project is going to remove their community modules from the main repo to collections in Ansible Galaxy and other repositories. Community modules should find them a new home. We're going through the same process in Openstack and other communities. I don't have a link for reference yet, but @gundalow from Ansible core team can confirm.

Required actions:
We have a few podman modules in Ansible now and a few of them on review, and some in Openstack repo of TripeO-Ansible.
I think the best place for Ansible podman/buildah modules will be the repository under containers/ namespace here.
Also, we need to choose a namespace to hold them in Ansible Galaxy, I think "podman." could be a logical choice. If you think different, please comment.
I see there is a repo ansible-podman here which contains one podman image module, that duplicates the existing podman image module in Ansible. I'm not sure what is the purpose of this repo and this duplicate, so I'd propose to go to a new repository name like "ansible-podman-collections". It will have a different structure and will contain current Ansible modules in the form of collection.

What is an advantage to have these modules here?
Ansible modules of podman can be kept up with the latest podman/buildah changes and ensure they are not broken when the new version of podman is released. They can be tested in real environments with real-life user scenarios and on various platforms. Like we have jobs for TripleO here, we can organize the same for Ansible podman collections. Also, it's very logical to have modules here, near the podman repos. And no much choice, to be honest.

License question.
Ansible and its modules are developed under GPLv3 license. Podman is developed under Apache2. To prevent a mix of licenses in this namespace, I'd like to relicense all my modules under Apache2. Podman image module was made by @samdoran and it's up to him to relicense it. All the rest is written by me, so no problem :)

Next steps as it seems to me.

1. Create a repo ansible-podman-collections under containers/ space.
2. Move podman modules from Ansible to a new repo with @gundalow help. Ansible modules stay still in Ansible upstream repo to prevent breakages in people playbooks and will be linked to new collections.
3. Set up Ansilbe Galaxy "podman." namespace (or whatever will be chosen)
4. Possibly to set up syncing jobs to backport changes to 2.9 when fixing critical issues in Ansible old module. Remember: new modules will be in usage from Ansible 2.10 only, 2.9 will still use old ones from Ansible repository. "Possibly" - because no much activity is there, so we can leave it as an exceptional manual task.
5. Set up a CI with good testing and publishing modules to Ansilbe Galaxy under "podman." namespace (or whatever will be chosen).
6. To move here Podman modules from TripleO-Ansible repo.

Please comment here what do you think.
Thanks.

/kind feature

[rhatdan]

This is fine with me.
How about we create ansible-container-creations? Rather then make it be specific to Podman. Would people be building buildah and potential skopeo content here?

[gundalow]

Hi,
Thanks for starting the discussion

1. To clarify github.com/ansible/ansible will contain only a very small number of modules and plugins. That list (and use-cases) are defined in ansible-minimal. Or put another way it's not just Community Content that will be moved from ansible/ansible into Collections.
2. As you mentioned, though just to clarify, the GitHub Org and repo names don't have to match the Collection Namespace or Collection Name.
3. Ansible Currently does Major (feature) releases twice a year, one of the major advantages of Collections is you can release them independently of Ansible so as podman has new features you can get them into the hands of your users quicker.
4. w.r.t. relicensing please ensure you check for anybody and everybody that's ever committed to these modules, plugins, docs_fragments, module_utils, tests, etc, etc.

Your next steps look spot on to me.

Naming things is always fun, so thinking about what goes where and what it might look like n-months in the future can help.

Please do shout out if there is anything I can assist with. The Core & Community team are all available in #ansible-devel on Freenode.

Docs

• Using Collections
• Developing Collections

Both of these docs are fairly new, and we know that things are still changing. If you spot anything that's missing/confusing/wrong please do let us know by raising a Docs issue

[sshnaidm]

This is fine with me.
How about we create ansible-container-creations? Rather then make it be specific to Podman. Would people be building buildah and potential skopeo content here?

Yeah, I think it's great to have other modules there too. At least we will be prepared for that.
On second thought nothing will block us to create repos ansible-skopeo-collection or whatever. I think it's possible to have different namespaces in one repo, but maybe easier and simpler to have repo-namespace one to one match, like ansible-podman-collections will go to "podman." namespace in Galaxy, ansible-skopeo-collections to "skopeo." namespace.
BTW, need to reserve these namespaces too.

[geerlingguy]

If using a container-y namespace, it might be good to manage all the container-related modules in there—docker, buildah, podman...

The only concern I have is what if one of those communities decides they want to split off and manage the release cadance/etc. differently?

[sshnaidm]

I think Docker devs might be want to develop this in https://github.com/docker when Docker is actually being developed. I think it's better to keep modules close to tools they wrap to reflect better changes in these tools.

If using a container-y namespace, it might be good to manage all the container-related modules in there—docker, buildah, podman...

The only concern I have is what if one of those communities decides they want to split off and manage the release cadance/etc. differently?

[rhatdan]

@sshnaidm I created the repository and invited you to administrate it.

[gundalow]

Where is the repo?

[sshnaidm]

@gundalow the repo is https://github.com/containers/ansible-podman-collections
I requested the namespace "containers": ansible/galaxy#2148
@rhatdan send me please your galaxy username, I'll add it as admin.

[gundalow]

Excellent, thank you.

You can use https://github.com/ansible-community/collection_migration to help populate the repo by creating a scenario file.

[rhatdan]

@sshnaidm I just grabbed rhatdan from galaxy.

[sshnaidm]

@gundalow @rhatdan great, we have a namespace on Galaxy, and I hope to start moving next week. If not, then after holidays.

[sshnaidm]

@rhatdan can you please make me a part of github.org "containers" organization https://github.com/orgs/containers/people ?
Seems like without it I can't do anything, galaxy requires you to be a part of organization to assign it to namespace. @gundalow can confirm

[rhatdan]

Sent you an invite.

[gundalow]

@rhatdan you should have powers now

[rhatdan]

Frankly I do not know what you want me to do?

[sshnaidm]

@rhatdan just enjoy it 😄

[sshnaidm]

Just an update, after working with @gundalow on Openstack modules moving, we'll start with podman modules.

[felixfontein]

I think Docker devs might be want to develop this in https://github.com/docker when Docker is actually being developed. I think it's better to keep modules close to tools they wrap to reflect better changes in these tools.

The Ansible docker_* modules are not related to Docker Inc., and as far as I'm aware the set of current docker_* maintainers/developers is disjoint from the set of people related to Docker Inc. (like employees). So I don't see why the docker_* modules should/could end up in https://github.com/docker. I personally would prefer if they end up in a "generic" container / cloud collection, instead of a standalone docker collection.

[rhatdan]

I am all for that. We should use the generic name container rather then the company name Docker, whenever possible.

[sshnaidm]

Well, in this case it makes sense to host docker_* modules under "containers" Galaxy namespace as @geerlingguy suggested. I have no objections to be it like: containers.podman.* and containers.docker.* or whatever.
@felixfontein The question is where to host the repositories with docker_* modules? Under this "containers" github organization? Or somewhere else? And who will handle docker_* modules migration there.
@rhatdan I'm not sure we can get rid off "docker" name in ansible modules, since we still need to differentiate them from podman ones.

[rhatdan]

Are these modules exec'ing docker/podman or talking directly to the docker.sock?

[sshnaidm]

@rhatdan ansible docker_* modules use docker sock and docker-py python libraries. They are very different from podman ones.

[rhatdan]

Well if you are following libpod, there is an effort to support docker.sock and some of docker-py with Podman. APIV2.
https://podman.io/releases/2020/01/17/podman-new-api.html
Podman might be able to work with docker ansible modules in the future.

[sshnaidm]

@rhatdan thanks, that's very interesting and promising!
Actually it may change we develop podman modules (currently as wrapper about podman commands). If new API will support most of the important docker-py functionality, Ansible users could just use docker_* modules as is with underlying podman.
Although podman modules structured differently and (I hope!) much better than current docker_* modules. So maybe it's worth to develop them with future podman python bindings, by keeping current docker_* modules for compatibility with old playbooks and actual current docker users. Or trying to combine them together while supporting "best effort" of functionality from both projects.
Well, this is whole and interesting topic to discuss. Maybe it deserves its own issue or maillist thread.
I'll be happy to start it after we move modules finally.

[sshnaidm]

@gundalow can we move here also ansible podman connection plugin? Issues like #4813 affects plugin funcitonality and blocks us to use it, for example for molecule testing. We'll need to wait until next ansible release to use it, after it's packaged, distributed, etc etc. Till then we're blocked.
Having connection plugin here would help to prevent this sort of issues or solve them quickly and release.

[gundalow]

Yes, all plugins should be moved into the new collection.

[rhatdan]

Can I close this issue now? Not sure this is actually a podman problem? Reopen if I am mistaken

[sshnaidm]

Not a problem, just issue for tracking. I think I'll send updates via ML about that later.

[sshnaidm]

FYI modules have been moved to new place: containers/ansible-podman-collections#3
@gundalow ^^