Permission denied when trying to use the /var/run/docker.sock file generated by the mac-helper

[rafaelfranca]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

When trying to use the VSCode Remote container extension with podman, one of the commands that is executed tries to use the /var/run/docker.sock file to mount a volume. That file has different file permission than the original podman socket it is being liked to:

% ls -la /var/run/docker.sock
lrwxr-xr-x  1 root  daemon  70 30 Mar 15:33 /var/run/docker.sock -> /Users/rafaelfranca/.local/share/containers/podman/machine/podman.sock

 % ls -la /Users/rafaelfranca/.local/share/containers/podman/machine/podman.sock
lrwxr-xr-x  1 rafaelfranca  staff  93 30 Mar 15:04 /Users/rafaelfranca/.local/share/containers/podman/machine/podman.sock -> /Users/rafaelfranca/.local/share/containers/podman/machine/podman-machine-default/podman.sock

This is causing the command to fail with permission denied. I believe that symlink should have the same permissions as the original file.

Steps to reproduce the issue:

podman run -v /var/run/docker.sock:/var/run/docker.sock vsc-volume-bootstrap with any valid container.

Describe the results you received:

Command failed: podman run -d --mount type=volume,src=rails-main-ecb592c6d095efde43d1d0e49d27304d,dst=/workspaces -v /var/run/docker.sock:/var/run/docker.sock vsc-volume-bootstrap sleep infinity
[953 ms] Error: statfs /var/run/docker.sock: permission denied

Describe the results you expected:

Command executed with success

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Client:       Podman Engine
Version:      4.0.2
API Version:  4.0.2
Go Version:   go1.17.8

Built:      Wed Mar  2 09:04:36 2022
OS/Arch:    darwin/arm64

Server:       Podman Engine
Version:      4.0.2
API Version:  4.0.2
Go Version:   go1.16.14

Built:      Thu Mar  3 09:58:50 2022
OS/Arch:    linux/arm64

Output of podman info --debug:

host:
  arch: arm64
  buildahVersion: 1.24.1
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.0-2.fc35.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.0, commit: '
  cpus: 4
  distribution:
    distribution: fedora
    variant: coreos
    version: "35"
  eventLogger: journald
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
    uidmap:
    - container_id: 0
      host_id: 501
      size: 1
    - container_id: 1
      host_id: 100000
      size: 1000000
  kernel: 5.15.18-200.fc35.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 3075612672
  memTotal: 4044693504
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.4.2-1.fc35.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.4.2
      commit: f6fbc8f840df1a414f31a60953ae514fa497c748
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/501/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc35.aarch64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 26m 39.43s
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - docker.io
store:
  configFile: /var/home/core/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/home/core/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 4
  runRoot: /run/user/501/containers
  volumePath: /var/home/core/.local/share/containers/storage/volumes
version:
  APIVersion: 4.0.2
  Built: 1646319530
  BuiltTime: Thu Mar  3 09:58:50 2022
  GitCommit: ""
  GoVersion: go1.16.14
  OsArch: linux/arm64
  Version: 4.0.2

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

[mheon]

@baude PTAL

[aaronjwhiteside]

I have a similar issue on podman 4.0.3 on Mac (m1).

This is simulating how testcontainers starts the ryuk container.

# podman run -it --rm -v /var/run/docker.sock:/var/run/docker.sock testcontainers/ryuk:0.3.3
2022/04/30 01:56:16 Pinging Docker...
panic: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/_ping: dial unix /var/run/docker.sock: connect: permission denied

goroutine 1 [running]:
main.main()
	/go/src/github.com/testcontainers/moby-ryuk/main.go:36 +0x457

Even with :Z it still cannot read the docker.sock file.

$ podman run -it --rm -v /var/run/docker.sock:/var/run/docker.sock:Z testcontainers/ryuk:0.3.3
2022/04/30 01:59:36 Pinging Docker...
panic: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/_ping: dial unix /var/run/docker.sock: connect: permission denied

goroutine 1 [running]:
main.main()
	/go/src/github.com/testcontainers/moby-ryuk/main.go:36 +0x457

Another more direct way to test it.

# podman run -it --rm --entrypoint=/bin/sh -v /var/run/docker.sock:/var/run/docker.sock:Z fedora:36
# dnf install socat
# socat - UNIX-CONNECT:/var/run/docker.sock
2022/04/30 02:19:41 socat[31] E connect(5, AF=1 "/var/run/docker.sock", 22): Permission denied

# podman version
Client:       Podman Engine
Version:      4.0.3
API Version:  4.0.3
Go Version:   go1.18
Built:        Fri Apr  1 08:28:59 2022
OS/Arch:      darwin/arm64

Server:       Podman Engine
Version:      4.0.3
API Version:  4.0.3
Go Version:   go1.18
Built:        Fri Apr  1 11:22:39 2022
OS/Arch:      linux/arm64

# podman info --debug
host:
  arch: arm64
  buildahVersion: 1.24.3
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - pids
  - misc
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.1.0-2.fc36.aarch64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.0, commit: '
  cpus: 4
  distribution:
    distribution: fedora
    variant: coreos
    version: "36"
  eventLogger: journald
  hostname: localhost.localdomain
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.17.3-300.fc36.aarch64
  linkmode: dynamic
  logDriver: journald
  memFree: 9910640640
  memTotal: 10401304576
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.4.4-1.fc36.aarch64
    path: /usr/bin/crun
    version: |-
      crun version 1.4.4
      commit: 6521fcc5806f20f6187eb933f9f45130c86da230
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: true
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.2.0-0.2.beta.0.fc36.aarch64
    version: |-
      slirp4netns version 1.2.0-beta.0
      commit: 477db14a24ff1a3de3a705e51ca2c4c1fe3dda64
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 32m 22.92s
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  docker.io:
    Blocked: false
    Insecure: true
    Location: docker.io
    MirrorByDigestOnly: false
    Mirrors:
    - Insecure: true
      Location: maven.it.yapstone.com:5000
    Prefix: docker.io
  search:
  - docker.io
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 8
    paused: 0
    running: 0
    stopped: 8
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: xfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 3
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.0.3
  Built: 1648837359
  BuiltTime: Fri Apr  1 11:22:39 2022
  GitCommit: ""
  GoVersion: go1.18
  OsArch: linux/arm64
  Version: 4.0.3

podman was setup using:

# brew install podman

# podman machine init --cpus 4 --disk-size 100 -m 10240 -v $HOME:$HOME
# podman machine start
# podman machine ssh
# sudo -i
# rpm-ostree install qemu-user-static
# exit
# exit
# podman machine stop; podman machine start
# sudo podman-mac-helper install

[vkochnev]

Someone may correct me, but in my recent experience:
First of all you need to take into account that volume mounts reference filesystem inside of podman machine and not on you mac filesystem.

Error: statfs /var/run/docker.sock: permission denied

Means that you don't have podman-docker package installed in podman machine and as a result podman doesn't listen on the socket.
podman machine ssh sudo rpm-ostree install podman-docker might help to solve it.

panic: Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/_ping: dial unix /var/run/docker.sock: connect: permission denied

Is caused by SELinux, see coreos/fedora-coreos-tracker#585 for details.
May be fixed with setting SELINUX=disabled in /etc/selinux/config in podman machine.

As a proper solution maybe podman-docker package should be preinstalled along with podman and SELinux should be preconfigured via default ignition to allow /var/run/docker.sock to be mounted into containers.
Also it helps in rootful mode, but I'm not sure if it helps in rootless mode.

[shanesmith]

According to this accessing /var/run/docker.sock requires to run the container with --privileged.

@aaronjwhiteside I've had success with both your testcontainers/ryuk:0.3.3 and fedora:36 examples with --privileged.

@rafaelfranca I'm not familiar with VSCode Remote container extension and the vsc-volume-bootstrap doesn't seem to exist, I suppose it's something that VSCode builds? In any case, could you try with --privileged?

Also, I don't believe the podman-mac-helper is needed for either of you. At the very least, I was able to run @aaronjwhiteside's examples without it.

[rhatdan]

You do not need to run --privileged you could disable SELinux for the container.

podman run --security-opt label:disabled

[rhatdan]

I am moving this to discussion, since I do not believe Podman is doing anything incorrect here.