Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] podman Windows WSL support #3288

Closed
kkimdev opened this issue Jun 10, 2019 · 40 comments
Closed

[Feature Request] podman Windows WSL support #3288

kkimdev opened this issue Jun 10, 2019 · 40 comments
Assignees
Labels
kind/feature Categorizes issue or PR as related to a new feature. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@kkimdev
Copy link

kkimdev commented Jun 10, 2019

/kind feature

Description
Windows WSL support. Note that WSL 2 is expected to be released in June 2019

So probably worthwhile to wait first.

Steps to reproduce the issue:

  1. Install podman on Windows WSL Ubuntu https://github.com/containers/libpod/blob/master/install.md#ubuntu
  2. podman run --rm -it ubuntu

Describe the results you received:

$ podman run --rm -it ubuntu
journal error: could not connect to journald socket
ERRO[0000] unable to write pod event: "journal error: could not connect to journald socket"
journal error: could not connect to journald socket
ERRO[0000] unable to write pod event: "journal error: could not connect to journald socket"
Error: error creating network namespace for container dbc11a4b136f186e62ba24a670e3ab291cd60e881c5e738c70d4b255dfc2bb63: mount --make-rshared /var/run/netns failed: "invalid argument"

Describe the results you expected:
Successfully running Ubuntu container.

Output of podman version:

$ podman version
Version:            1.3.2-dev
RemoteAPI Version:  1
Go Version:         go1.10.4
OS/Arch:            linux/amd64

Output of podman info --debug:

$ podman info --debug
debug:
  compiler: gc
  git commit: ""
  go version: go1.10.4
  podman version: 1.3.2-dev
host:
  BuildahVersion: 1.9.0-dev
  Conmon:
    package: 'conmon: /usr/libexec/crio/conmon'
    path: /usr/libexec/crio/conmon
    version: 'conmon version 0.2.0, commit: '
  Distribution:
    distribution: ubuntu
    version: "18.04"
  MemFree: 8412651520
  MemTotal: 17057861632
  OCIRuntime:
    package: 'cri-o-runc: /usr/bin/runc'
    path: /usr/bin/runc
    version: 'runc version spec: 1.0.1-dev'
  SwapFree: 51499941888
  SwapTotal: 51539607552
  arch: amd64
  cpus: 8
  hostname: DESKTOP-BG5LGLR
  kernel: 4.4.0-17134-Microsoft
  os: linux
  rootless: false
  uptime: 1h 47m 5.64s (Approximately 0.04 days)
registries:
  blocked: null
  insecure: null
  search:
  - docker.io
store:
  ConfigFile: /etc/containers/storage.conf
  ContainerStore:
    number: 0
  GraphDriverName: overlay
  GraphOptions: null
  GraphRoot: /var/lib/containers/storage
  GraphStatus:
    Backing Filesystem: <unknown>
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  ImageStore:
    number: 4
  RunRoot: /var/run/containers/storage
  VolumePath: /var/lib/containers/storage/volumes
@openshift-ci-robot openshift-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Jun 10, 2019
@rhatdan
Copy link
Member

rhatdan commented Jun 10, 2019

@baude is this podman attempting to write events to the journal?

@mheon
Copy link
Member

mheon commented Jun 10, 2019 via email

@baude
Copy link
Member

baude commented Jun 12, 2019

yes, it is simply saying it cannot find journald. I don't believe the error is fatal. This can be remedied in a couple of different ways but the easiest would be to alter the event logger in libpod.conf.

@baude
Copy link
Member

baude commented Jun 12, 2019

In fact, i wonder if we should include that in the error message?

@kkimdev
Copy link
Author

kkimdev commented Jun 12, 2019

@baude I think that's a good suggestion. When I encountered, I couldn't figure out if it's fatal or not just from the error messages.

@baude
Copy link
Member

baude commented Jun 12, 2019

BTW, here you can just change to file and it should help.

https://github.com/containers/libpod/blob/master/libpod.conf#L104

@baude
Copy link
Member

baude commented Jun 12, 2019

is this still an issue?

@kkimdev
Copy link
Author

kkimdev commented Jun 12, 2019

@baude yes

$ podman run --rm -it ubuntu
Error: error creating network namespace for container 3d988a6d443f6851a2ac5ec4f474830e297c163ec4a9787cb381bb39cce95bf0: mount --make-rshared /var/run/netns failed: "invalid argument"

@mheon
Copy link
Member

mheon commented Jun 12, 2019

I think that's probably WSL v1 (syscall emulation) not being able to do everything we need.

@rhatdan
Copy link
Member

rhatdan commented Jun 12, 2019

Does --net=host work?

@kkimdev
Copy link
Author

kkimdev commented Jun 12, 2019

Still the same

$ podman run --rm -it ubuntu --net=host
Error: error creating network namespace for container 2a111e0f68aad45981bd73b8bc63917acd355a9b70aab71ab3068b8060ee9bed: mount --make-rshared /var/run/netns failed: "invalid argument"

@kkimdev
Copy link
Author

kkimdev commented Jun 12, 2019

Actually, I just noticed that the flags are supposed to be before "ubuntu". Now I'm getting a different error

$ podman run --net=host --rm -it ubuntu
Error: container create failed: time="2019-06-12T13:45:44-07:00" level=error msg="systemd cgroup flag passed, but systemd support for managing cgroups is not available\n"
systemd cgroup flag passed, but systemd support for managing cgroups is not available
: internal libpod error

@rhatdan
Copy link
Member

rhatdan commented Jun 12, 2019

Need to default libpod.conf to use cgroupfs rather then systemd.
podman --cgroup-manager=cgroupfs ...

Should fix this.

@kkimdev
Copy link
Author

kkimdev commented Jun 12, 2019

I guess it's getting closer, one step at a time! =D

$ podman run --cgroup-manager=cgroupfs --net=host --rm -it ubuntu
Error: container create failed: time="2019-06-12T13:50:17-07:00" level=warning msg="signal: killed"
time="2019-06-12T13:50:17-07:00" level=error msg="container_linux.go:346: starting container process caused \"process_linux.go:297: applying cgroup configuration for process caused \\\"mountpoint for cgroup not found\\\"\"\n"
container_linux.go:346: starting container process caused "process_linux.go:297: applying cgroup configuration for process caused \"mountpoint for cgroup not found\""
: internal libpod error

@mheon
Copy link
Member

mheon commented Jun 13, 2019

With the release of WSL 2 (though only through the Insiders program thus far), I think we should be ready; I'd be interested to see someone try Podman on the new WSL and report back.

@mheon
Copy link
Member

mheon commented Jun 14, 2019 via email

@baude
Copy link
Member

baude commented Jun 23, 2019

Couple of notable items here, i tried wsl2 with ubuntu:

  • it looks like the cni network cannot be configured due to an invalid iptables command. iptables on bionic is quite old. i tried manually compiled iptables 1.8.2 but that didn't work either. trying a newer distribution is probably worth our while, will do next week.
  • rootless also fails with an error about not being able to re-exec. im not sure if that is the core issue or symptomatic.

@rhatdan
Copy link
Member

rhatdan commented Jun 24, 2019

Does WSL Support User Namespace?
As root, attempt to run with a user namespace

--uidmap 0:100000:5000

@mheon
Copy link
Member

mheon commented Jun 24, 2019

Might want to see if we can pull the flags they compiled their kernel with - they might not be shipping userns-enabled kernels?

@baude
Copy link
Member

baude commented Jun 24, 2019

results for uidmap

root@DESKTOP-2O0KKB1:~# sudo podman --log-level=debug run -it --uidmap 0:100000:5000 alpine ls
DEBU[0000] Initializing boltdb state at /var/lib/containers/storage/libpod/bolt_state.db
DEBU[0000] Using graph driver overlay
DEBU[0000] Using graph root /var/lib/containers/storage
DEBU[0000] Using run root /var/run/containers/storage
DEBU[0000] Using static dir /var/lib/containers/storage/libpod
DEBU[0000] Using tmp dir /var/run/libpod
DEBU[0000] Using volume path /var/lib/containers/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] overlay test mount with multiple lowers succeeded
DEBU[0000] overlay test mount indicated that metacopy is not being used
DEBU[0000] backingFs=<unknown>, projectQuotaSupported=false, useNativeDiff=true, usingMetacopy=false
DEBU[0000] Initializing event backend file
INFO[0000] Found CNI network podman (type=bridge) at /etc/cni/net.d/87-podman-bridge.conflist
DEBU[0001] parsed reference into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]docker.io/library/alpine:latest"
DEBU[0001] parsed reference into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]@4d90542f0623c71f1f9c11be3da23167174ac9d93731cf91912922e916bab02c"
DEBU[0001] exporting opaque data as blob "sha256:4d90542f0623c71f1f9c11be3da23167174ac9d93731cf91912922e916bab02c"
DEBU[0001] parsed reference into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]@4d90542f0623c71f1f9c11be3da23167174ac9d93731cf91912922e916bab02c"
DEBU[0001] exporting opaque data as blob "sha256:4d90542f0623c71f1f9c11be3da23167174ac9d93731cf91912922e916bab02c"
DEBU[0001] parsed reference into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]@4d90542f0623c71f1f9c11be3da23167174ac9d93731cf91912922e916bab02c"
DEBU[0001] Got mounts: []
DEBU[0001] Got volumes: []
DEBU[0001] Using bridge netmode
DEBU[0001] created OCI spec and options for new container
DEBU[0001] Allocated lock 9 for container 417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225
DEBU[0001] parsed reference into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]@4d90542f0623c71f1f9c11be3da23167174ac9d93731cf91912922e916bab02c"
DEBU[0001] exporting opaque data as blob "sha256:4d90542f0623c71f1f9c11be3da23167174ac9d93731cf91912922e916bab02c"
DEBU[0001] created container "417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225"
DEBU[0002] container "417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225" has work directory "/var/lib/containers/storage/overlay-containers/417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225/userdata"
DEBU[0002] container "417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225" has run directory "/var/run/containers/storage/overlay-containers/417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225/userdata"
DEBU[0002] New container created "417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225"
DEBU[0002] container "417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225" has CgroupParent "/libpod_parent/libpod-417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225"
DEBU[0002] Handling terminal attach
DEBU[0002] overlay: mount_data=lowerdir=/var/lib/containers/storage/overlay/l/V6UI4YISJWEFRZIPRAWK3TRHEE:/var/lib/containers/storage/overlay/l/V6UI4YISJWEFRZIPRAWK3TRHEE/../diff1:/var/lib/containers/storage/overlay/l/KGEJ2TZZMASJ5NTB3SUQ2SP6KH,upperdir=/var/lib/containers/storage/overlay/2401519d5cb1eb0ee3a11e7a077fbf96fa40186c8fc0ead977f3e745f8fabc81/diff,workdir=/var/lib/containers/storage/overlay/2401519d5cb1eb0ee3a11e7a077fbf96fa40186c8fc0ead977f3e745f8fabc81/work
DEBU[0002] mounted container "417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225" at "/var/lib/containers/storage/overlay/2401519d5cb1eb0ee3a11e7a077fbf96fa40186c8fc0ead977f3e745f8fabc81/merged"
DEBU[0002] Created root filesystem for container 417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225 at /var/lib/containers/storage/overlay/2401519d5cb1eb0ee3a11e7a077fbf96fa40186c8fc0ead977f3e745f8fabc81/merged
DEBU[0002] /etc/system-fips does not exist on host, not mounting FIPS mode secret
DEBU[0002] Setting CGroup path for container 417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225 to /libpod_parent/libpod-417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225
DEBU[0002] reading hooks from /usr/share/containers/oci/hooks.d
DEBU[0002] reading hooks from /etc/containers/oci/hooks.d
DEBU[0002] Created OCI spec for container 417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225 at /var/lib/containers/storage/overlay-containers/417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225/userdata/config.json
DEBU[0002] /usr/libexec/crio/conmon messages will be logged to syslog
DEBU[0002] running conmon: /usr/libexec/crio/conmon      args=[-c 417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225 -u 417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225 -n sleepy_davinci -r /usr/bin/runc -b /var/lib/containers/storage/overlay-containers/417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225/userdata -p /var/run/containers/storage/overlay-containers/417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225/userdata/pidfile --exit-dir /var/run/libpod/exits --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /var/run/containers/storage --exit-command-arg --log-level --exit-command-arg debug --exit-command-arg --cgroup-manager --exit-command-arg cgroupfs --exit-command-arg --tmpdir --exit-command-arg /var/run/libpod --exit-command-arg --runtime --exit-command-arg runc --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg container --exit-command-arg cleanup --exit-command-arg 417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225 --socket-dir-path /var/run/libpod/socket -t -l k8s-file:/var/lib/containers/storage/overlay-containers/417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225/userdata/ctr.log --log-level debug --syslog]
DEBU[0003] Received container pid: -1
DEBU[0003] Cleaning up container 417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225
DEBU[0003] Network is already cleaned up, skipping...
DEBU[0003] unmounted container "417df521d863ef34ad091638e698ec1a69a03556cf1c18ceb89972465c79a225"
ERRO[0004] container create failed: time="2019-06-24T08:58:12-05:00" level=fatal msg="nsexec:602 nsenter: could not ensure we are a cloned binary: Permission denied"
time="2019-06-24T08:58:12-05:00" level=error msg="container_linux.go:346: starting container process caused \"process_linux.go:319: getting the final child's pid from pipe caused \\\"EOF\\\"\"\n"
container_linux.go:346: starting container process caused "process_linux.go:319: getting the final child's pid from pipe caused \"EOF\""
: internal libpod error

@baude
Copy link
Member

baude commented Jun 24, 2019

It looks like the kernel supplied for WSL does not support user namespaces.

@kkimdev
Copy link
Author

kkimdev commented Jul 2, 2019

Just tried on WSL2 Ubuntu

$ podman --log-level=debug run --cgroup-manager=cgroupfs --net=host --rm -it ubuntu
INFO[0000] running as rootless
DEBU[0000] Initializing boltdb state at /home/kkimdev/.local/share/containers/storage/libpod/bolt_state.db
DEBU[0000] Using graph driver vfs
DEBU[0000] Using graph root /home/kkimdev/.local/share/containers/storage
DEBU[0000] Using run root /tmp/run-1000
DEBU[0000] Using static dir /home/kkimdev/.local/share/containers/storage/libpod
DEBU[0000] Using tmp dir /tmp/run-1000/libpod/tmp
DEBU[0000] Using volume path /home/kkimdev/.local/share/containers/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] [graphdriver] trying provided driver "vfs"
DEBU[0000] Initializing event backend journald
DEBU[0000] parsed reference into "[vfs@/home/kkimdev/.local/share/containers/storage+/tmp/run-1000]docker.io/library/ubuntu:latest"
DEBU[0000] parsed reference into "[vfs@/home/kkimdev/.local/share/containers/storage+/tmp/run-1000]@4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] exporting opaque data as blob "sha256:4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] parsed reference into "[vfs@/home/kkimdev/.local/share/containers/storage+/tmp/run-1000]@4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] exporting opaque data as blob "sha256:4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] parsed reference into "[vfs@/home/kkimdev/.local/share/containers/storage+/tmp/run-1000]@4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] Got mounts: []
DEBU[0000] Got volumes: []
DEBU[0000] Using host netmode
DEBU[0000] created OCI spec and options for new container
DEBU[0000] Allocated lock 0 for container 57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4
DEBU[0000] parsed reference into "[vfs@/home/kkimdev/.local/share/containers/storage+/tmp/run-1000]@4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] exporting opaque data as blob "sha256:4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] created container "57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4"
DEBU[0000] container "57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4" has work directory "/home/kkimdev/.local/share/containers/storage/vfs-containers/57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4/userdata"
DEBU[0000] container "57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4" has run directory "/tmp/run-1000/vfs-containers/57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4/userdata"
ERRO[0000] unable to write pod event: "write unixgram @00020->/run/systemd/journal/socket: sendmsg: no such file or directory"
DEBU[0000] New container created "57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4"
DEBU[0000] container "57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4" has CgroupParent "/libpod_parent/libpod-57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4"
DEBU[0000] Handling terminal attach
DEBU[0000] mounted container "57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4" at "/home/kkimdev/.local/share/containers/storage/vfs/dir/7369c65bc66f2a26d9e55f13462f6b528726242549a162ea37527f82b7f8d3b8"
DEBU[0000] Created root filesystem for container 57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4 at /home/kkimdev/.local/share/containers/storage/vfs/dir/7369c65bc66f2a26d9e55f13462f6b528726242549a162ea37527f82b7f8d3b8
DEBU[0000] /etc/system-fips does not exist on host, not mounting FIPS mode secret
DEBU[0000] Created OCI spec for container 57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4 at /home/kkimdev/.local/share/containers/storage/vfs-containers/57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4/userdata/config.json
DEBU[0000] /usr/bin/conmon messages will be logged to syslog
DEBU[0000] running conmon: /usr/bin/conmon               args="[-c 57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4 -u 57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4 -n hopeful_minsky -r /usr/bin/runc -b /home/kkimdev/.local/share/containers/storage/vfs-containers/57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4/userdata -p /tmp/run-1000/vfs-containers/57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4/userdata/pidfile --exit-dir /tmp/run-1000/libpod/tmp/exits --conmon-pidfile /tmp/run-1000/vfs-containers/57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4/userdata/conmon.pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /home/kkimdev/.local/share/containers/storage --exit-command-arg --runroot --exit-command-arg /tmp/run-1000 --exit-command-arg --log-level --exit-command-arg debug --exit-command-arg --cgroup-manager --exit-command-arg cgroupfs --exit-command-arg --tmpdir --exit-command-arg /tmp/run-1000/libpod/tmp --exit-command-arg --runtime --exit-command-arg runc --exit-command-arg --storage-driver --exit-command-arg vfs --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4 --socket-dir-path /tmp/run-1000/libpod/tmp/socket -t -l k8s-file:/home/kkimdev/.local/share/containers/storage/vfs-containers/57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4/userdata/ctr.log --log-level debug --syslog]"
WARN[0000] Failed to add conmon to cgroupfs sandbox cgroup: mkdir /sys/fs/cgroup/freezer/libpod_parent: permission denied
DEBU[0000] Received container pid: -1
DEBU[0000] Cleaning up container 57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4
DEBU[0000] Network is already cleaned up, skipping...
DEBU[0000] unmounted container "57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4"
DEBU[0000] Cleaning up container 57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4
DEBU[0000] Network is already cleaned up, skipping...
DEBU[0000] Container 57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4 storage is already unmounted, skipping...
DEBU[0000] Container 57e5368529a88c5f02d065284709bdc6385d9f4a347660b0c5e8b5f1acce56b4 storage is already unmounted, skipping...
ERRO[0000] unable to write pod event: "write unixgram @00020->/run/systemd/journal/socket: sendmsg: no such file or directory"
ERRO[0000] time="2019-07-01T18:33:38-07:00" level=warning msg="exit status 1"
time="2019-07-01T18:33:38-07:00" level=error msg="container_linux.go:346: starting container process caused \"process_linux.go:449: container init caused \\\"rootfs_linux.go:58: mounting \\\\\\\"proc\\\\\\\" to rootfs \\\\\\\"/home/kkimdev/.local/share/containers/storage/vfs/dir/7369c65bc66f2a26d9e55f13462f6b528726242549a162ea37527f82b7f8d3b8\\\\\\\" at \\\\\\\"/proc\\\\\\\" caused \\\\\\\"operation not permitted\\\\\\\"\\\"\"\n"
container_linux.go:346: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting \\\"proc\\\" to rootfs \\\"/home/kkimdev/.local/share/containers/storage/vfs/dir/7369c65bc66f2a26d9e55f13462f6b528726242549a162ea37527f82b7f8d3b8\\\" at \\\"/proc\\\" caused \\\"operation not permitted\\\"\""
: OCI runtime error

@kkimdev
Copy link
Author

kkimdev commented Jul 2, 2019

Actually, "sudo" in front of the command worked

$ sudo podman --log-level=debug run --cgroup-manager=cgroupfs --net=host --rm -it ubuntu
DEBU[0000] Initializing boltdb state at /var/lib/containers/storage/libpod/bolt_state.db
DEBU[0000] Using graph driver overlay
DEBU[0000] Using graph root /var/lib/containers/storage
DEBU[0000] Using run root /var/run/containers/storage
DEBU[0000] Using static dir /var/lib/containers/storage/libpod
DEBU[0000] Using tmp dir /var/run/libpod
DEBU[0000] Using volume path /var/lib/containers/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] cached value indicated that overlay is supported
DEBU[0000] cached value indicated that metacopy is not being used
DEBU[0000] cached value indicated that native-diff is usable
DEBU[0000] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=true, usingMetacopy=false
DEBU[0000] Initializing event backend journald
INFO[0000] Found CNI network podman (type=bridge) at /etc/cni/net.d/87-podman-bridge.conflist
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]docker.io/library/ubuntu:latest"
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]@4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] exporting opaque data as blob "sha256:4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]@4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] exporting opaque data as blob "sha256:4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]@4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] Got mounts: []
DEBU[0000] Got volumes: []
DEBU[0000] Using host netmode
DEBU[0000] created OCI spec and options for new container
DEBU[0000] Allocated lock 0 for container 41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082
DEBU[0000] parsed reference into "[overlay@/var/lib/containers/storage+/var/run/containers/storage]@4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] exporting opaque data as blob "sha256:4c108a37151f54439950335c409802e948883e00c93fdb751d206c9a9674c1f6"
DEBU[0000] created container "41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082"
DEBU[0000] container "41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082" has work directory "/var/lib/containers/storage/overlay-containers/41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082/userdata"
DEBU[0000] container "41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082" has run directory "/var/run/containers/storage/overlay-containers/41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082/userdata"
ERRO[0000] unable to write pod event: "write unixgram @00028->/run/systemd/journal/socket: sendmsg: no such file or directory"
DEBU[0000] New container created "41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082"
DEBU[0000] container "41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082" has CgroupParent "/libpod_parent/libpod-41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082"
DEBU[0000] Handling terminal attach
DEBU[0000] overlay: mount_data=lowerdir=/var/lib/containers/storage/overlay/l/L6SZR5Z47WSUQTRJD2LKWORLBZ:/var/lib/containers/storage/overlay/l/F77KMERWGJCDGDA4EDF7K7GQJX:/var/lib/containers/storage/overlay/l/4BBGKYFB3UM6WYGDT7DNLOFSZ3:/var/lib/containers/storage/overlay/l/FFMQH7ACNCLAH6MKWLEPACYLQC,upperdir=/var/lib/containers/storage/overlay/9a3950a8da20ab7121f1a89a36e387ab120d6a1d659240d427a0f702179ebc3d/diff,workdir=/var/lib/containers/storage/overlay/9a3950a8da20ab7121f1a89a36e387ab120d6a1d659240d427a0f702179ebc3d/work
DEBU[0000] mounted container "41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082" at "/var/lib/containers/storage/overlay/9a3950a8da20ab7121f1a89a36e387ab120d6a1d659240d427a0f702179ebc3d/merged"
DEBU[0000] Created root filesystem for container 41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082 at /var/lib/containers/storage/overlay/9a3950a8da20ab7121f1a89a36e387ab120d6a1d659240d427a0f702179ebc3d/merged
DEBU[0000] /etc/system-fips does not exist on host, not mounting FIPS mode secret
DEBU[0000] Setting CGroup path for container 41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082 to /libpod_parent/libpod-41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082
DEBU[0000] reading hooks from /usr/share/containers/oci/hooks.d
DEBU[0000] reading hooks from /etc/containers/oci/hooks.d
DEBU[0000] Created OCI spec for container 41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082 at /var/lib/containers/storage/overlay-containers/41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082/userdata/config.json
DEBU[0000] /usr/bin/conmon messages will be logged to syslog
DEBU[0000] running conmon: /usr/bin/conmon               args="[-c 41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082 -u 41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082 -n cool_wu -r /usr/bin/runc -b /var/lib/containers/storage/overlay-containers/41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082/userdata -p /var/run/containers/storage/overlay-containers/41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082/userdata/pidfile --exit-dir /var/run/libpod/exits --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/storage --exit-command-arg --runroot --exit-command-arg /var/run/containers/storage --exit-command-arg --log-level --exit-command-arg debug --exit-command-arg --cgroup-manager --exit-command-arg cgroupfs --exit-command-arg --tmpdir --exit-command-arg /var/run/libpod --exit-command-arg --runtime --exit-command-arg runc --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg container --exit-command-arg cleanup --exit-command-arg --rm --exit-command-arg 41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082 --socket-dir-path /var/run/libpod/socket -t -l k8s-file:/var/lib/containers/storage/overlay-containers/41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082/userdata/ctr.log --log-level debug --syslog]"
DEBU[0000] Received container pid: 5375
DEBU[0000] Created container 41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082 in OCI runtime
ERRO[0000] unable to write pod event: "write unixgram @00028->/run/systemd/journal/socket: sendmsg: no such file or directory"
DEBU[0000] Attaching to container 41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082
DEBU[0000] connecting to socket /var/run/libpod/socket/41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082/attach
DEBU[0000] Starting container 41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082 with command [/bin/bash]
DEBU[0000] Received a resize event: {Width:189 Height:96}
DEBU[0000] Started container 41928e9dac7e36d9a6c1d42e7e697bf82c8e0790d1068c2f8e8a3f987b071082
ERRO[0000] unable to write pod event: "write unixgram @00028->/run/systemd/journal/socket: sendmsg: no such file or directory"
root@DESKTOP-BG5LGLR:/# ERRO[0000] unable to write pod event: "write unixgram @00028->/run/systemd/journal/socket: sendmsg: no such file or directory"
DEBU[0000] Enabling signal proxying

root@DESKTOP-BG5LGLR:/#

@rhatdan
Copy link
Member

rhatdan commented Jul 2, 2019

I thought someone had confirmed that the Kernel used did not have User Namespace enabled, meaning rootless podman will not work. Someone should point out to Microsoft or Ubuntu or whoever that this should be enabled in the kernel.

@baude
Copy link
Member

baude commented Jul 2, 2019

the kernel always comes from microsoft. and indeed, it does not support user namespace. so both rootless and rooted without --net=host fail.

@kkimdev
Copy link
Author

kkimdev commented Jul 2, 2019

I guess we should file an issue to https://github.com/microsoft/WSL2-Linux-Kernel then?

@rhatdan
Copy link
Member

rhatdan commented Jul 2, 2019

@kkimdev Could you take care of that?

@kkimdev
Copy link
Author

kkimdev commented Jul 2, 2019

@Biswa96
Copy link

Biswa96 commented Jul 2, 2019

This may help microsoft/WSL#4165

@kkimdev
Copy link
Author

kkimdev commented Jul 2, 2019

@Biswa96 Thanks for the information!

I just tried curl -L https://raw.githubusercontent.com/moby/moby/master/contrib/check-config.sh | bash on WSL 2 and it reports CONFIG_USER_NS is already enabled.

I'm not really familiar with the kernel/namespace stuff so I will back off from further investigation though.

@nalind
Copy link
Member

nalind commented Jul 2, 2019

Does sysctl user.max_user_namespaces print 0? Older versions of RHEL had the support for user namespaces compiled in but disabled by default, so that could be why something like unshare -Ur cat /proc/self/uid_map would fail to create a new user namespace.

@kkimdev
Copy link
Author

kkimdev commented Jul 2, 2019

Does sysctl user.max_user_namespaces print 0? Older versions of RHEL had the support for user namespaces compiled in but disabled by default, so that could be why something like unshare -Ur cat /proc/self/uid_map would fail to create a new user namespace.

No I got the following on WSL 2.

$ sysctl user.max_user_namespaces
user.max_user_namespaces = 59538

Also,

$ unshare -Ur cat /proc/self/uid_map
         0       1000          1
$ echo $?
0
$

@rhatdan
Copy link
Member

rhatdan commented Aug 5, 2019

Since we have another issue on this #3000 lets continue the discussion there.
@kkimdev @baude ^^

@rhatdan rhatdan closed this as completed Aug 5, 2019
@prototact
Copy link

prototact commented Dec 10, 2019

Hello! The issue with podman is still ongoing, it throws an error about insufficient ids in namespace.
Note I am using WSL2, Windows Build 19037.1

$ podman pull registry.access.redhat.com/ubi8/ubi
Trying to pull registry.access.redhat.com/ubi8/ubi...
Getting image source signatures
Copying blob 340ff6d7f58c done
Copying blob 0e8ea260d026 done
Copying config cb642e6a99 done
Writing manifest to image destination
Storing signatures
  Error processing tar file(exit status 1): there might not be enough IDs available in the namespace (requested 0:12 for /var/spool/mail): lchown /var/spool/mail: invalid argument
Error: error pulling image "registry.access.redhat.com/ubi8/ubi": unable to pull registry.access.redhat.com/ubi8/ubi: unable to pull image: Error committing the finished image: error adding layer with blob "sha256:340ff6d7f58c908c438ce89bb845caee3649de828a81569317ed0fe169a97de2": Error processing tar file(exit status 1): there might not be enough IDs available in the namespace (requested 0:12 for /var/spool/mail): lchown /var/spool/mail: invalid argument

Apparently there is no userids allocated for other than root. So for root it works.

podman unshare cat /proc/self/uid_map
         0       1000          1

Possibly this mockbuild user has something to do with it?

cat /etc/subuid
mockbuild:100000:65536
orfeaskar:165536:65536

For root, I get an error when running a container about a journal socket

$ sudo podman run registry.access.redhat.com/ubi8/ubi ls
ERRO[0000] unable to write pod event: "write unixgram @00046->/run/systemd/journal/socket: sendmsg: no such file or directory"
Error: cannot open sd-bus: No such file or directory: OCI runtime command not found error

@rhatdan
Copy link
Member

rhatdan commented Dec 11, 2019

Could you set the --event-logger=file. does that fix the root running problem?

@zhangdaopin
Copy link

According to the above problem description and treatment methods, I still haven't solved the problem that podman started alpine container. Is there any new progress in this problem? ?

root@N-20HEPF15JN1Q:/etc/apt/sources.list.d# podman run --cgroup-manager=cgroupfs --net=host -it --rm docker.io/library/alpine /bin/sh Your kernel does not support pids limit capabilities or the cgroup is not mounted. PIDs limit discarded. Error: container_linux.go:370: starting container process caused: process_linux.go:338: getting the final child's pid from pipe caused: EOF: OCI runtime error

@rhatdan
Copy link
Member

rhatdan commented Sep 8, 2020

@baude @ashley-cui PTAL

@mheon mheon assigned baude and ashley-cui and unassigned baude Sep 8, 2020
@imperialguy
Copy link

imperialguy commented Dec 23, 2020

According to the above problem description and treatment methods, I still haven't solved the problem that podman started alpine container. Is there any new progress in this problem? ?

root@N-20HEPF15JN1Q:/etc/apt/sources.list.d# podman run --cgroup-manager=cgroupfs --net=host -it --rm docker.io/library/alpine /bin/sh Your kernel does not support pids limit capabilities or the cgroup is not mounted. PIDs limit discarded. Error: container_linux.go:370: starting container process caused: process_linux.go:338: getting the final child's pid from pipe caused: EOF: OCI runtime error

I am getting the exact same error on WSL. It doesn't matter what image you are trying to work with. Here's my example. Tried this as root on WSL Ubuntu 20.04

root@DESKTOP-FWIH23B:/# podman run -it registry.access.redhat.com/ubi8/ubi-minimal bash
Your kernel does not support pids limit capabilities or the cgroup is not mounted. PIDs limit discarded.
Error: error creating network namespace for container 1c450a359a05cf89b4b8580f314bb3d2a1aa4ca828c5f71f6be297307d7b9e20: failed to create namespace: permission denied```

@baude
Copy link
Member

baude commented Dec 24, 2020

can you use wsl2?

@gbraad
Copy link
Member

gbraad commented Jan 27, 2021

Currently with minor modifications, like dnf reinstall shadow-utils or setting the rights on /usr/bin/newuidmap, /usr/bin/newgidmap and editing containers.conf

cgroup_manager = "cgroupfs"
events_logger = "file"

this should work. However, there is no easy way to use podman system connection from the host as the WSL2 IP changes and it does not run ssh out of the box as there is no systemd.

@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 22, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 22, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/feature Categorizes issue or PR as related to a new feature. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Development

No branches or pull requests

13 participants