Silverblue toolbox script broke because rootless 'podman start' fails

[debarshiray]

/kind bug

Description

Here is the toolbox script that we have been working on for Fedora Silverblue.

If you follow the README.md until the fedora-toolbox enter step, then you'll see that the podman start ... command fails:

$ ./fedora-toolbox --verbose enter
unable to start container "fedora-toolbox-rishi:28": error reading container (probably exited) json message: EOF
./fedora-toolbox: failed to start container fedora-toolbox-rishi:28

This works fine if I use podman-0.9.1.1 with the fix for #1452 cherry-picked on top, but breaks if I use the 0.9.3 release.

Note that you'll need the patch from runc PR #1862 or you have to rollback to fedora-toolbox commit a878a1fe40e4c24a.

Output of podman version:

Version:       0.9.3
Go Version:    go1.10.4
OS/Arch:       linux/amd64

Output of podman info:

host:
  Conmon:
    package: podman-0.9.3-4.gita723353.fc28.x86_64
    path: /usr/libexec/podman/conmon
    version: 'conmon version 1.12.0-dev, commit: 0fc9fe1e482dc5cb8dfca6d485cb1e27ec738f9d-dirty'
  MemFree: 11055898624
  MemTotal: 16696311808
  OCIRuntime:
    package: runc-1.0.0-53.1.dev.git70ca035.fc28.x86_64
    path: /usr/bin/runc
    version: 'runc version spec: 1.0.0'
  SwapFree: 4208979968
  SwapTotal: 4208979968
  arch: amd64
  cpus: 4
  hostname: bollard
  kernel: 4.18.7-200.fc28.x86_64
  os: linux
  uptime: 12m 24.61s
insecure registries:
  registries: []
registries:
  registries:
  - docker.io
  - registry.fedoraproject.org
  - quay.io
  - registry.access.redhat.com
  - registry.centos.org
store:
  ContainerStore:
    number: 0
  GraphDriverName: vfs
  GraphOptions: []
  GraphRoot: /var/home/rishi/.local/share/containers/storage
  GraphStatus: {}
  ImageStore:
    number: 2
  RunRoot: /run/user/1000/run

Additional environment details (AWS, VirtualBox, physical, etc.):

This is a physical laptop running Fedora 28 Silverblue 28.20180918.0, with podman and runc overlaid on top (see above).

[giuseppe]

I think this is fixed with c4b15ce, could you please check it?

[giuseppe]

as a workaround for this issue, you could try to pull the image separately (podman pull image) before creating the container

[debarshiray]

I think this is fixed with c4b15ce, could you please check it?

Yes, #1522 is fixed. If I cherry-pick c4b15ce on top of the podman-0.9.2 release, then things work. However, if I use the podman-0.9.3 release, then I hit this bug.

[mheon]

Known issue with 0.9.3 - SELinux labelling broke runc in this case (specifically the tmpfs mount contexts added in the mount flag pr). We're looking into getting a patch out to resolve this and cutting a 0.9.3.1 today.

…

On Tue, Sep 25, 2018, 07:43 Debarshi Ray ***@***.***> wrote: I think this is fixed with c4b15ce <c4b15ce>, could you please check it? Yes, #1522 <#1522> is fixed. If I cherry-pick c4b15ce <c4b15ce> on top of the podman-0.9.2 release, then things work. However, if I use the podman-0.9.3 release, then I hit this bug. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1535 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHYHCBwjSdMI_hNrMl-xBQxiUCKN1-jCks5uehbHgaJpZM4W4Lku> .

[debarshiray]

Known issue with 0.9.3 - SELinux labelling broke runc in this case
(specifically the tmpfs mount contexts added in the mount flag pr). We're
looking into getting a patch out to resolve this and cutting a 0.9.3.1
today.

I see, ok.

Note that I have SELinux set to permissive in the host, and the toolbox container is created with --security-opt label=disable.

[mheon]

We know SELinux permissive doesn't fix the issue, but disabling labelling hasn't been tested before. Thinking about the code in question, I believe it might not respect that flag, which we should also fix when we resolve this.

…

On Tue, Sep 25, 2018, 08:09 Debarshi Ray ***@***.***> wrote: Known issue with 0.9.3 - SELinux labelling broke runc in this case (specifically the tmpfs mount contexts added in the mount flag pr). We're looking into getting a patch out to resolve this and cutting a 0.9.3.1 today. I see, ok. Note that I have SELinux set to permissive in the host, and the toolbox container is created with --security-opt label=disable. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1535 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHYHCNqyDAL56Cqoem7jEAlij-AdFo8Wks5ueh0TgaJpZM4W4Lku> .

[rhatdan]

Could someone give me a command that causes this?

[rhatdan]

Fixed in #1541

[debarshiray]

Yes, indeed, this works for me with podman-0.9.3.1. Thanks a lot for the quick fix!