Failure to connect to container's attach socket

[mfleader]

I am an intern working on a dashboard application for the OpenShift Performance and Scale team. I use podman for containerized development and deployment. This issue has come up for me before now, but appeared to be resolved by upgrading podman from 2.x to 3.0.0.

Steps to reproduce the issue:

1. podman -it run node

Describe the results you received:

Error: failed to connect to container's attach socket: /run/user/1000/snap.code/libpod/tmp/socket/0bfea2ffb6974989ab5d39809cb5188ee524de858cd9c6c3714ef5fc113fe5a5/attach: no such file or directory

Describe the results you expected:

A container running some version of NodeJS.

Additional information you deem important (e.g. issue happens only occasionally):

I have installed snap, and vscode through snap. The snap.code directory seems to come up in situations where those two co-occur.

Output of podman version:

Version:      3.1.0
API Version:  3.1.0
Go Version:   go1.16
Built:        Mon Apr  5 10:37:55 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.20.0
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon-2.0.27-1.fc34.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.0.27, commit: '
  cpus: 8
  distribution:
    distribution: fedora
    version: "34"
  eventLogger: journald
  hostname: localhost.localdomain
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.11.13-300.fc34.x86_64
  linkmode: dynamic
  memFree: 22658531328
  memTotal: 33436647424
  ociRuntime:
    name: crun
    package: crun-0.19-1.fc34.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 0.19
      commit: e67a75672412975916dac6b87f8346f870e4b99a
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    selinuxEnabled: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.9-1.fc34.x86_64
    version: |-
      slirp4netns version 1.1.8+dev
      commit: 6dc0186e020232ae1a6fcc1f7afbc3ea02fd3876
      libslirp: 4.4.0
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.0
  swapFree: 25379725312
  swapTotal: 25379725312
  uptime: 7m 6.34s
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /home/mleader/.config/containers/storage.conf
  containerStore:
    number: 11
    paused: 0
    running: 0
    stopped: 11
  graphDriverName: overlay
  graphOptions:
    overlay.mount_program:
      Executable: /usr/bin/fuse-overlayfs
      Package: fuse-overlayfs-1.5.0-1.fc34.x86_64
      Version: |-
        fusermount3 version: 3.10.2
        fuse-overlayfs: version 1.5
        FUSE library version 3.10.2
        using FUSE kernel interface version 7.31
  graphRoot: /home/mleader/.local/share/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageStore:
    number: 192
  runRoot: /run/user/1000/snap.code/containers
  volumePath: /home/mleader/.local/share/containers/storage/volumes
version:
  APIVersion: 3.1.0
  Built: 1617633475
  BuiltTime: Mon Apr  5 10:37:55 2021
  GitCommit: ""
  GoVersion: go1.16
  OsArch: linux/amd64
  Version: 3.1.0

Package info (e.g. output of rpm -q podman or apt list podman):

podman-3.1.0-2.fc34.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

physical, Lenovo Thinkpad T490s, i7

[mheon]

conmon-2.0.27-1.fc34.x86_64 and Podman 3.1.0 - this should have the fixes in both Podman and Conmon.

@haircommander @giuseppe Any ideas here? Per the report, we regressed on this since 3.0.0, but I don't see any changes that would have done that.

[haircommander]

hm this is unexpected...

[haircommander]

wait is the XDG_RUNTIME_DIR being changed by snap? I would not expect snap.code to be in that path

[mfleader]

since @haircommander mentioned XDG_RUNTIME_DIR, I found this issue made on vscode. Maybe it's a snap packaging thing? Is this helpful?

My system shows this:

> echo $XDG_RUNTIME_DIR
/run/user/1000

[haircommander]

did this work around help you microsoft/vscode#95384 (comment)

[mfleader]

@haircommander Do you mean trying this?

> export XDG_RUNTIME_DIR=/run/user/1000/snap.code
> podman run -it --rm node
Error: sd-bus call: Permission denied: OCI permission denied

[mfleader]

> export XDG_RUNTIME_DIR=/run/user/1000
> podman run -it --rm node
Error: failed to connect to container's attach socket: /run/user/1000/snap.code/libpod/tmp/socket/a752790ad4b1ca700969f830d15d264f21d7d2422cf1a8115d24042439e4789e/attach: no such file or directory

socket filename: a752790ad4b1ca700969f830d15d264f21d7d2422cf1a8115d24042439e4789e

> ls /run/user/1000/snap.code/libpod/tmp/socket
a752790ad4b1ca700969f830d15d264f21d7d2422cf1a8115d24042439e4789

Does the socket filename being short by the last letter affect anything?

[haircommander]

ah! that should be fixed by #8979 (which I have not had the cycles to push through)

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

Since that PR has merged, I am going to assume this is fixed.