podman-docker incompatibilities in common_cli.py: KeyError: 'ApiVersion'

[pavelzw]

SUMMARY

The plugin fails when using podman-docker because it expects a specific format output format in docker version --format '{{ json . }}' which is not supported by `podman-docker:

community.docker/plugins/module_utils/common_cli.py

Line 83 in eddeb91

self.docker_api_version_str = self._version['Server']['ApiVersion']

$ docker version --format '{{ json . }}' | jq
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
{
  "Client": {
    "APIVersion": "5.1.1",
    "Version": "5.1.1",
    "GoVersion": "go1.22.3",
    "GitCommit": "",
    "BuiltTime": "Tue Jun  4 02:00:00 2024",
    "Built": 1717459200,
    "OsArch": "linux/amd64",
    "Os": "linux"
  }
}

ISSUE TYPE

• Bug Report

COMPONENT NAME

community.docker/plugins/module_utils/common_cli.py

ANSIBLE VERSION

ansible [core 2.16.6]
  config file = None
  configured module search path = ['/Users/pavel/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = .../.pixi/envs/default/lib/python3.12/site-packages/ansible
  ansible collection location = /Users/pavel/.ansible/collections:/usr/share/ansible/collections
  executable location = .../.pixi/envs/default/bin/ansible
  python version = 3.12.3 | packaged by conda-forge | (main, Apr 15 2024, 18:35:20) [Clang 16.0.6 ] (.../.pixi/envs/default/bin/python3.12)
  jinja version = 3.1.4
  libyaml = True

COLLECTION VERSION

❯ ansible-galaxy collection list community.docker

# /Users/pavel/.ansible/collections/ansible_collections
Collection       Version
---------------- -------
community.docker 3.10.4 

# .../.pixi/envs/default/lib/python3.12/site-packages/ansible_collections
Collection       Version
---------------- -------
community.docker 3.9.0  

CONFIGURATION

CONFIG_FILE() = None

OS / ENVIRONMENT

macOS M1

target OS: fedora 40 with podman and podman-docker installed

$ dnf list installed | grep podman
podman.x86_64                                 5:5.1.1-1.fc40             @updates
podman-compose.noarch                         1.1.0-1.fc40               @updates
podman-docker.noarch                          5:5.1.1-1.fc40             @updates

STEPS TO REPRODUCE

• Install podman, podman-docker
• create a docker-compose on the target machine

    - name: Install packages
      ansible.builtin.dnf:
        name:
          - podman
          - podman-docker
        state: present
    - name: Install docker-compose
      ansible.builtin.get_url:
        url: https://github.com/docker/compose/releases/download/v2.28.0/docker-compose-linux-x86_64
        dest: /usr/local/bin/docker-compose
        mode: '0755'
        checksum: sha256:359043c2336e243662d7038c3edfeadcd5b9fc28dabe6973dbaecf48c0c1f967
    - name: Start docker container
      community.docker.docker_compose_v2:
        project_src: path/to/docker-dir
        state: present

EXPECTED RESULTS

This should go through

ACTUAL RESULTS

module_stdout: |-
    Traceback (most recent call last):
      File "/home/pavel/.ansible/tmp/ansible-tmp-1719100884.715269-51508-141209575582183/AnsiballZ_docker_compose_v2.py", line 107, in <module>
        _ansiballz_main()
      File "/home/pavel/.ansible/tmp/ansible-tmp-1719100884.715269-51508-141209575582183/AnsiballZ_docker_compose_v2.py", line 99, in _ansiballz_main
        invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)
      File "/home/pavel/.ansible/tmp/ansible-tmp-1719100884.715269-51508-141209575582183/AnsiballZ_docker_compose_v2.py", line 47, in invoke_module
        runpy.run_module(mod_name='ansible_collections.community.docker.plugins.modules.docker_compose_v2', init_globals=dict(_module_fqn='ansible_collections.community.docker.plugins.modules.docker_compose_v2', _modlib_path=modlib_path),
      File "<frozen runpy>", line 226, in run_module
      File "<frozen runpy>", line 98, in _run_module_code
      File "<frozen runpy>", line 88, in _run_code
      File "/tmp/ansible_community.docker.docker_compose_v2_payload_s26629r8/ansible_community.docker.docker_compose_v2_payload.zip/ansible_collections/community/docker/plugins/modules/docker_compose_v2.py", line 642, in <module>
      File "/tmp/ansible_community.docker.docker_compose_v2_payload_s26629r8/ansible_community.docker.docker_compose_v2_payload.zip/ansible_collections/community/docker/plugins/modules/docker_compose_v2.py", line 626, in main
      File "/tmp/ansible_community.docker.docker_compose_v2_payload_s26629r8/ansible_community.docker.docker_compose_v2_payload.zip/ansible_collections/community/docker/plugins/module_utils/common_cli.py", line 303, in __init__
      File "/tmp/ansible_community.docker.docker_compose_v2_payload_s26629r8/ansible_community.docker.docker_compose_v2_payload.zip/ansible_collections/community/docker/plugins/module_utils/common_cli.py", line 83, in __init__
    KeyError: 'ApiVersion'

[felixfontein]

This is not a bug in this collection since we do not support podman-docker. If something works fine with podman-docker you're lucky, but if it does not it's not a bug for this collection.

Your playbook doesn't look like it could ever work anyway:

1. You don't seem to be starting podman's docker socket. With that docker-compose won't be able to access podman as Docker.
2. You are installing the docker-compose binary, but the docker_compose_v2 module needs the Compose plugin. It uses docker compose instead of docker-compose to access Compose.

[pavelzw]

You don't seem to be starting podman's docker socket. With that docker-compose won't be able to access podman as Docker.

I am, only forgot to include this in the reproducer

It uses docker compose instead of docker-compose to access Compose.

actually, docker compose also invokes the docker-compose binary in some way on my system 🤔

[felixfontein]

It uses docker compose instead of docker-compose to access Compose.

actually, docker compose also invokes the docker-compose binary in some way on my system 🤔

Then that aspect should be fine, but as long as podman version does not provide the Docker API version, the module will still not work. (The module also expects docker info to provide the Compose plugin's version; podman info doesn't seem to do that, so this would probably be the next thing that fails.)

[ch0fe]

I believe it would be valuable if podman could also be supported by this module.
What would be required for the module to test for capabilities rather than a version string?

[felixfontein]

Skipping the API version check is pretty easy to implement, but as I wrote above, the big problem will be the capabilities based Compose check, which doesn't work since podman info isn't providing that information at all.

[apollo13]

@felixfontein Would executing docker compose version instead be an option? This would also work nicely if one uses the real compose plugin with docker.

[felixfontein]

@apollo13 docker compose version does not provide a good way to determine whether the plugin is installed if it fails.

[apollo13]

We could fall back to the existing code if `docker compose version` doesn't return the expected string. In my limited testing this allows to use the module via podman.

…

On Sat, Dec 28, 2024, at 17:00, Felix Fontein wrote: @apollo13 <https://github.com/apollo13> `docker compose version` does not provide a good way to determine whether the plugin is installed if it fails. — Reply to this email directly, view it on GitHub <#891 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAT5C5SAT7ZGD3PLH3YYIL2H3DIBAVCNFSM6AAAAABJX2XFN6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRUGM3DQNBTHE>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[felixfontein]

The next release will contain a PR by @apollo13 which should make this work.