Running docker-compose with podman as a backend on MacOS is failing #13069

mikegit21 · 2022-01-28T16:26:56Z

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Running docker-compose with podman as a backend on MacOS is failing

Steps to reproduce the issue:

Install podman using brew: brew install podman
Initialize the podman machine: podman machine init
Started the machine: podman machine start
Installed docker-compose:

sudo curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

sudo chmod +x /usr/local/bin/docker-compose

Try to run docker-compose up with some docker-compose.yaml file

Describe the results you received:

docker-compose cannot fetch the API version of the docker backend (in this case podman) or is somehow blocked:

Traceback (most recent call last):
  File "urllib3/connectionpool.py", line 670, in urlopen
  File "urllib3/connectionpool.py", line 392, in _make_request
  File "http/client.py", line 1255, in request
  File "http/client.py", line 1301, in _send_request
  File "http/client.py", line 1250, in endheaders
  File "http/client.py", line 1010, in _send_output
  File "http/client.py", line 950, in send
  File "docker/transport/unixconn.py", line 43, in connect
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "requests/adapters.py", line 439, in send
  File "urllib3/connectionpool.py", line 726, in urlopen
  File "urllib3/util/retry.py", line 410, in increment
  File "urllib3/packages/six.py", line 734, in reraise
  File "urllib3/connectionpool.py", line 670, in urlopen
  File "urllib3/connectionpool.py", line 392, in _make_request
  File "http/client.py", line 1255, in request
  File "http/client.py", line 1301, in _send_request
  File "http/client.py", line 1250, in endheaders
  File "http/client.py", line 1010, in _send_output
  File "http/client.py", line 950, in send
  File "docker/transport/unixconn.py", line 43, in connect
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "docker/api/client.py", line 214, in _retrieve_server_version
  File "docker/api/daemon.py", line 181, in version
  File "docker/utils/decorators.py", line 46, in inner
  File "docker/api/client.py", line 237, in _get
  File "requests/sessions.py", line 543, in get
  File "requests/sessions.py", line 530, in request
  File "requests/sessions.py", line 643, in send
  File "requests/adapters.py", line 498, in send
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 81, in main
  File "compose/cli/main.py", line 200, in perform_command
  File "compose/cli/command.py", line 60, in project_from_options
  File "compose/cli/command.py", line 152, in get_project
  File "compose/cli/docker_client.py", line 41, in get_client
  File "compose/cli/docker_client.py", line 170, in docker_client
  File "docker/api/client.py", line 197, in __init__
  File "docker/api/client.py", line 221, in _retrieve_server_version
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))
[63757] Failed to execute script docker-compose

I tried running the command with sudo in case there are any entitlement problems, but this didn't fix the problem.

Describe the results you expected:

I would expect docker-compose to perform the up command as it would be when running docker as a backend.

Additional information you deem important (e.g. issue happens only occasionally):

I created a symlink for podman (ln -s podman docker), so running docker version gives me the same and I can actually run containers using docker run even though docker is not installed.

docker compose version:

docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1h  22 Sep 2020

Output of podman version:

Client:
Version:      3.4.4
API Version:  3.4.4
Go Version:   go1.17.6
Built:        Wed Dec  8 19:41:11 2021
OS/Arch:      darwin/amd64

Server:
Version:      3.4.4
API Version:  3.4.4
Go Version:   go1.16.8
Built:        Wed Dec  8 22:45:07 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.30-2.fc35.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.30, commit: '
  cpus: 1
  distribution:
    distribution: fedora
    variant: coreos
    version: "35"
  eventLogger: journald
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.15.17-200.fc35.x86_64
  linkmode: dynamic
  logDriver: journald
  memFree: 3539537920
  memTotal: 4104261632
  ociRuntime:
    name: crun
    package: crun-1.4-1.fc35.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.4
      commit: 3daded072ef008ef0840e8eccb0b52a7efbd165d
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc35.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 1h 40m 1.17s (Approximately 0.04 days)
plugins:
  log:
  - k8s-file
  - none
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 0
    stopped: 1
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 8
  runRoot: /run/user/1000/containers
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 3.4.4
  Built: 1638999907
  BuiltTime: Wed Dec  8 21:45:07 2021
  GitCommit: ""
  GoVersion: go1.16.8
  OsArch: linux/amd64
  Version: 3.4.4

Package info (e.g. output of rpm -q podman or apt list podman):

brew info podman

podman: stable 3.4.4 (bottled), HEAD
Tool for managing OCI containers and pods
https://podman.io/
/usr/local/Cellar/podman/3.4.4 (170 files, 40.1MB) *
  Poured from bottle on 2022-01-28 at 10:38:30
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/podman.rb
License: Apache-2.0
==> Dependencies
Build: go ✘, go-md2man ✘
Required: qemu ✔
==> Options
--HEAD
	Install HEAD version
==> Caveats
zsh completions have been installed to:
  /usr/local/share/zsh/site-functions
==> Analytics
install: 12,877 (30 days), 36,389 (90 days), 88,923 (365 days)
install-on-request: 12,876 (30 days), 36,394 (90 days), 88,915 (365 days)
build-error: 1 (30 days)

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

**Additional environment details (AWS, VirtualBox, physical, etc.):

MacOS Montery 12.1

The text was updated successfully, but these errors were encountered:

matejvasek · 2022-01-28T17:37:56Z

EDIT -- sorry this probably won't work 😞

@mikegit21 you need to set DOCKER_HOST envvar.
Something like:

export DOCKER_HOST="$(podman system connection ls --format="{{.URI}}" | grep root)"

Also you need to add podman machine key to ssh-agent:

ssh-add -k "$HOME/.ssh/podman-machine-default"

I recommend using rootful socket:

podman system connection default podman-machine-default-root

and put the server key to known hosts file.

podman machine ssh # this will prompt for trust to the server

Note this complicated set up might not be needed in future.

@n1hility is working on auto exposure of socket so it would work automatically.

matejvasek · 2022-01-28T17:59:09Z

hmm this ^^^ might not work actually:
podman dial-stdio has been added just in v4.0.0-RC2.

matejvasek · 2022-01-28T18:10:25Z

more info:
#11397

matejvasek · 2022-01-28T18:12:45Z

#11397 (comment)

mikegit21 · 2022-01-31T08:10:24Z

hmm this ^^^ might not work actually: podman dial-stdio has been added just in v4.0.0-RC2.

Does this mean I could upgrade to this version and it should be working? Is it a stable one?

When running it with my current podman version I get:

Traceback (most recent call last):
  File "docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 81, in main
  File "compose/cli/main.py", line 200, in perform_command
  File "compose/cli/command.py", line 60, in project_from_options
  File "compose/cli/command.py", line 152, in get_project
  File "compose/cli/docker_client.py", line 41, in get_client
  File "compose/cli/docker_client.py", line 137, in docker_client
  File "docker/context/context.py", line 35, in __init__
  File "docker/context/config.py", line 77, in get_context_host
  File "docker/utils/utils.py", line 255, in parse_host
docker.errors.DockerException: Invalid bind address format: no path allowed for this protocol: ssh://root@localhost:63111/run/podman/podman.sock
[72891] Failed to execute script docker-compose

When removing the ssh part I again get:

Traceback (most recent call last):
  File "docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 81, in main
  File "compose/cli/main.py", line 200, in perform_command
  File "compose/cli/command.py", line 60, in project_from_options
  File "compose/cli/command.py", line 152, in get_project
  File "compose/cli/docker_client.py", line 41, in get_client
  File "compose/cli/docker_client.py", line 170, in docker_client
  File "docker/api/client.py", line 197, in __init__
  File "docker/api/client.py", line 221, in _retrieve_server_version
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', BadStatusLine('SSH-2.0-OpenSSH_8.7\r\n'))
[73017] Failed to execute script docker-compose

mikegit21 · 2022-01-31T08:11:37Z

#11397 (comment)

When trying to do this I am getting: ssh: connect to host localhost port 64724: Connection refused

matejvasek · 2022-01-31T10:13:58Z

@mikegit21 you must adjust the port number, it's auto-generated and it will be different on your machine.

mikegit21 · 2022-01-31T10:17:23Z

How do I see which port my podman machine is running on? Is it podman system connection ls?

matejvasek · 2022-01-31T10:21:23Z

podman system connection ls

mikegit21 · 2022-01-31T10:23:28Z

This did actually work, thank you a lot!

tuxoff · 2022-02-01T11:00:06Z

I have the same issue.

I did using the next steps

brew install podman
podman machine init
podman machine start
export DOCKER_HOST="$(podman system connection ls --format="{{.URI}}" | grep root)"
ssh-add -k "$HOME/.ssh/podman-machine-default"
podman system connection default podman-machine-default-root
podman machine ssh (successfully connected to the SSH)
> docker-compose up -d
Traceback (most recent call last):
  File "docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 81, in main
  File "compose/cli/main.py", line 200, in perform_command
  File "compose/cli/command.py", line 60, in project_from_options
  File "compose/cli/command.py", line 152, in get_project
  File "compose/cli/docker_client.py", line 41, in get_client
  File "compose/cli/docker_client.py", line 137, in docker_client
  File "docker/context/context.py", line 35, in __init__
  File "docker/context/config.py", line 77, in get_context_host
  File "docker/utils/utils.py", line 255, in parse_host
docker.errors.DockerException: Invalid bind address format: no path allowed for this protocol: ssh://root@localhost:51098/run/podman/podman.sock
[42754] Failed to execute script docker-compose

podman system connection ls

Name                          Identity                                   URI
podman-machine-default        /Users/myusername/.ssh/podman-machine-default  ssh://core@localhost:51098/run/user/1000/podman/podman.sock
podman-machine-default-root*  /Users/myusername/.ssh/podman-machine-default  ssh://root@localhost:51098/run/podman/podman.sock

podman version
Client:
Version:      3.4.4
API Version:  3.4.4
Go Version:   go1.17.6
Built:        Wed Dec  8 20:41:11 2021
OS/Arch:      darwin/amd64

Server:
Version:      3.4.4
API Version:  3.4.4
Go Version:   go1.16.8
Built:        Wed Dec  8 23:45:07 2021
OS/Arch:      linux/amd64

ENV: MacOS Monterey 12.1

matejvasek · 2022-02-01T11:14:23Z

@tuxoff try this #11397 (comment) (adjust the port there)

tuxoff · 2022-02-01T11:50:34Z

@matejvasek It works, thanks, will continue my testing.

AntonTutuka · 2022-02-22T01:42:53Z

I tried the approach above including this:

@tuxoff try this #11397 (comment) (adjust the port there)

and I'm getting the error:
`→ docker-compose up -d
Traceback (most recent call last):
File "urllib3/connectionpool.py", line 670, in urlopen
File "urllib3/connectionpool.py", line 392, in _make_request
File "http/client.py", line 1255, in request
File "http/client.py", line 1301, in _send_request
File "http/client.py", line 1250, in endheaders
File "http/client.py", line 1010, in _send_output
File "http/client.py", line 950, in send
File "docker/transport/unixconn.py", line 43, in connect
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "requests/adapters.py", line 439, in send
File "urllib3/connectionpool.py", line 726, in urlopen
File "urllib3/util/retry.py", line 410, in increment
File "urllib3/packages/six.py", line 734, in reraise
File "urllib3/connectionpool.py", line 670, in urlopen
File "urllib3/connectionpool.py", line 392, in _make_request
File "http/client.py", line 1255, in request
File "http/client.py", line 1301, in _send_request
File "http/client.py", line 1250, in endheaders
File "http/client.py", line 1010, in _send_output
File "http/client.py", line 950, in send
File "docker/transport/unixconn.py", line 43, in connect
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "docker/api/client.py", line 214, in _retrieve_server_version
File "docker/api/daemon.py", line 181, in version
File "docker/utils/decorators.py", line 46, in inner
File "docker/api/client.py", line 237, in _get
File "requests/sessions.py", line 543, in get
File "requests/sessions.py", line 530, in request
File "requests/sessions.py", line 643, in send
File "requests/adapters.py", line 498, in send
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "docker-compose", line 3, in
File "compose/cli/main.py", line 81, in main
File "compose/cli/main.py", line 200, in perform_command
File "compose/cli/command.py", line 60, in project_from_options
File "compose/cli/command.py", line 152, in get_project
File "compose/cli/docker_client.py", line 41, in get_client
File "compose/cli/docker_client.py", line 170, in docker_client
File "docker/api/client.py", line 197, in init
File "docker/api/client.py", line 221, in _retrieve_server_version
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))
[3891] Failed to execute script docker-compose`

openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Jan 28, 2022

github-actions bot added macos MacOS (OSX) related remote Problem is in podman-remote labels Jan 28, 2022

rhatdan closed this as completed Feb 1, 2022

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 20, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 20, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Running docker-compose with podman as a backend on MacOS is failing #13069

Running docker-compose with podman as a backend on MacOS is failing #13069

mikegit21 commented Jan 28, 2022

matejvasek commented Jan 28, 2022 •

edited

Loading

matejvasek commented Jan 28, 2022

matejvasek commented Jan 28, 2022

matejvasek commented Jan 28, 2022

mikegit21 commented Jan 31, 2022 •

edited

Loading

mikegit21 commented Jan 31, 2022

matejvasek commented Jan 31, 2022

mikegit21 commented Jan 31, 2022 •

edited

Loading

matejvasek commented Jan 31, 2022

mikegit21 commented Jan 31, 2022

tuxoff commented Feb 1, 2022 •

edited

Loading

matejvasek commented Feb 1, 2022

tuxoff commented Feb 1, 2022

AntonTutuka commented Feb 22, 2022

Running docker-compose with podman as a backend on MacOS is failing #13069

Running docker-compose with podman as a backend on MacOS is failing #13069

Comments

mikegit21 commented Jan 28, 2022

matejvasek commented Jan 28, 2022 • edited Loading

matejvasek commented Jan 28, 2022

matejvasek commented Jan 28, 2022

matejvasek commented Jan 28, 2022

mikegit21 commented Jan 31, 2022 • edited Loading

mikegit21 commented Jan 31, 2022

matejvasek commented Jan 31, 2022

mikegit21 commented Jan 31, 2022 • edited Loading

matejvasek commented Jan 31, 2022

mikegit21 commented Jan 31, 2022

tuxoff commented Feb 1, 2022 • edited Loading

matejvasek commented Feb 1, 2022

tuxoff commented Feb 1, 2022

AntonTutuka commented Feb 22, 2022

matejvasek commented Jan 28, 2022 •

edited

Loading

mikegit21 commented Jan 31, 2022 •

edited

Loading

mikegit21 commented Jan 31, 2022 •

edited

Loading

tuxoff commented Feb 1, 2022 •

edited

Loading