Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error: mkdir /sys/fs/cgroup/init: permission denied #15498

Closed
MarkusSchoelzel opened this issue Aug 26, 2022 · 2 comments · Fixed by #15503
Closed

Error: mkdir /sys/fs/cgroup/init: permission denied #15498

MarkusSchoelzel opened this issue Aug 26, 2022 · 2 comments · Fixed by #15503
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@MarkusSchoelzel
Copy link

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

I have a podman in rootless docker setup, which looks like this:
OS -> rootless docker as user daemon -> run privileged container -> podman runs as user

With 4.2.0 I can't run run podman system service as user in this setup, maybe because of #14787

Steps to reproduce the issue:

podman --log-level debug system service --time 0 unix:///tmp/podman.sock

Describe the results you received:

INFO[0000] podman filtering at log level debug
DEBU[0000] Called service.PersistentPreRunE(podman --log-level debug system service --time 0 unix:///tmp/podman.sock)
DEBU[0000] Merged system config "/etc/containers/containers.conf"
DEBU[0000] Using conmon: "/usr/local/bin/conmon"
DEBU[0000] Initializing boltdb state at /home/jenkins/.local/share/containers/storage/libpod/bolt_state.db
DEBU[0000] systemd-logind: dial unix /var/run/dbus/system_bus_socket: connect: no such file or directory
DEBU[0000] Using graph driver overlay
DEBU[0000] Using graph root /home/jenkins/.local/share/containers/storage
DEBU[0000] Using run root /tmp/podman-run-1000/containers
DEBU[0000] Using static dir /home/jenkins/.local/share/containers/storage/libpod
DEBU[0000] Using tmp dir /tmp/podman-run-1000/libpod/tmp
DEBU[0000] Using volume path /home/jenkins/.local/share/containers/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] Not configuring container store
DEBU[0000] Initializing event backend file
DEBU[0000] Configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argument
DEBU[0000] Configured OCI runtime runj initialization failed: no valid executable found for OCI runtime runj: invalid argument
DEBU[0000] Configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument
DEBU[0000] Configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument
DEBU[0000] Configured OCI runtime krun initialization failed: no valid executable found for OCI runtime krun: invalid argument
DEBU[0000] Using OCI runtime "/usr/local/bin/crun"
INFO[0000] podman filtering at log level debug
DEBU[0000] Called service.PersistentPreRunE(podman --log-level debug system service --time 0 unix:///tmp/podman.sock)
DEBU[0000] Merged system config "/etc/containers/containers.conf"
DEBU[0000] Using conmon: "/usr/local/bin/conmon"
DEBU[0000] Initializing boltdb state at /home/jenkins/.local/share/containers/storage/libpod/bolt_state.db
DEBU[0000] systemd-logind: dial unix /var/run/dbus/system_bus_socket: connect: no such file or directory
DEBU[0000] Using graph driver overlay
DEBU[0000] Using graph root /home/jenkins/.local/share/containers/storage
DEBU[0000] Using run root /tmp/podman-run-1000/containers
DEBU[0000] Using static dir /home/jenkins/.local/share/containers/storage/libpod
DEBU[0000] Using tmp dir /tmp/podman-run-1000/libpod/tmp
DEBU[0000] Using volume path /home/jenkins/.local/share/containers/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] [graphdriver] trying provided driver "overlay"
DEBU[0000] overlay: test mount with multiple lowers succeeded
DEBU[0000] Cached value indicated that overlay is supported
DEBU[0000] overlay: test mount indicated that metacopy is not being used
DEBU[0000] backingFs=extfs, projectQuotaSupported=false, useNativeDiff=true, usingMetacopy=false
DEBU[0000] Initializing event backend file
DEBU[0000] Configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argument
DEBU[0000] Configured OCI runtime runj initialization failed: no valid executable found for OCI runtime runj: invalid argument
DEBU[0000] Configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument
DEBU[0000] Configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument
DEBU[0000] Configured OCI runtime krun initialization failed: no valid executable found for OCI runtime krun: invalid argument
DEBU[0000] Using OCI runtime "/usr/local/bin/crun"
DEBU[0000] Successfully loaded 1 networks
DEBU[0000] Initialized SHM lock manager at path /libpod_rootless_lock_1000
DEBU[0000] Podman detected system restart - performing state refresh
INFO[0000] Setting parallel job count to 73
**Error: mkdir /sys/fs/cgroup/init: permission denied**

Describe the results you expected:

INFO[0000] podman filtering at log level debug
DEBU[0000] Called service.PersistentPreRunE(podman --log-level debug system service --time 0 unix:///tmp/podman.sock)
DEBU[0000] Merged system config "/etc/containers/containers.conf"
DEBU[0000] Using conmon: "/usr/local/bin/conmon"
DEBU[0000] Initializing boltdb state at /home/jenkins/.local/share/containers/storage/libpod/bolt_state.db
DEBU[0000] systemd-logind: dial unix /var/run/dbus/system_bus_socket: connect: no such file or directory
DEBU[0000] Using graph driver vfs
DEBU[0000] Using graph root /home/jenkins/.local/share/containers/storage
DEBU[0000] Using run root /tmp/podman-run-1000/containers
DEBU[0000] Using static dir /home/jenkins/.local/share/containers/storage/libpod
DEBU[0000] Using tmp dir /tmp/podman-run-1000/libpod/tmp
DEBU[0000] Using volume path /home/jenkins/.local/share/containers/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] Not configuring container store
DEBU[0000] Initializing event backend file
DEBU[0000] Configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argument
DEBU[0000] Configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument
DEBU[0000] Configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument
DEBU[0000] Configured OCI runtime krun initialization failed: no valid executable found for OCI runtime krun: invalid argument
DEBU[0000] Using OCI runtime "/usr/local/bin/crun"
INFO[0000] podman filtering at log level debug
DEBU[0000] Called service.PersistentPreRunE(podman --log-level debug system service --time 0 unix:///tmp/podman.sock)
DEBU[0000] Merged system config "/etc/containers/containers.conf"
DEBU[0000] Using conmon: "/usr/local/bin/conmon"
DEBU[0000] Initializing boltdb state at /home/jenkins/.local/share/containers/storage/libpod/bolt_state.db
DEBU[0000] systemd-logind: dial unix /var/run/dbus/system_bus_socket: connect: no such file or directory
DEBU[0000] Using graph driver vfs
DEBU[0000] Using graph root /home/jenkins/.local/share/containers/storage
DEBU[0000] Using run root /tmp/podman-run-1000/containers
DEBU[0000] Using static dir /home/jenkins/.local/share/containers/storage/libpod
DEBU[0000] Using tmp dir /tmp/podman-run-1000/libpod/tmp
DEBU[0000] Using volume path /home/jenkins/.local/share/containers/storage/volumes
DEBU[0000] Set libpod namespace to ""
DEBU[0000] [graphdriver] trying provided driver "vfs"
DEBU[0000] Initializing event backend file
DEBU[0000] Configured OCI runtime runc initialization failed: no valid executable found for OCI runtime runc: invalid argument
DEBU[0000] Configured OCI runtime kata initialization failed: no valid executable found for OCI runtime kata: invalid argument
DEBU[0000] Configured OCI runtime runsc initialization failed: no valid executable found for OCI runtime runsc: invalid argument
DEBU[0000] Configured OCI runtime krun initialization failed: no valid executable found for OCI runtime krun: invalid argument
DEBU[0000] Using OCI runtime "/usr/local/bin/crun"
DEBU[0000] Successfully loaded 1 networks
DEBU[0000] Initialized SHM lock manager at path /libpod_rootless_lock_1000
DEBU[0000] Podman detected system restart - performing state refresh
INFO[0000] Setting parallel job count to 73
DEBU[0000] registered SIGHUP watcher for config
INFO[0000] API service listening on "/tmp/podman.sock". URI: "unix:///tmp/podman.sock"
DEBU[0000] CORS Headers were not set
INFO[0000] API service listening on "/tmp/podman.sock"
DEBU[0000] waiting for SIGHUP to reload configuration
DEBU[0000] API service(s) shutting down, idle for 0s
DEBU[0000] API service shutdown request ignored as timeout Duration is UnlimitedService

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Client:       Podman Engine
Version:      4.2.0
API Version:  4.2.0
Go Version:   go1.19
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      linux/amd64

Output of podman info:

host:
  arch: amd64
  buildahVersion: 1.27.0
  cgroupControllers:
  - memory
  - pids
  cgroupManager: cgroupfs
  cgroupVersion: v2
  conmon:
    package: Unknown
    path: /usr/local/bin/conmon
    version: 'conmon version 2.1.3, commit: ab52a597278b20173440140cd810dc9fa8785c93-dirty'
  cpuUtilization:
    idlePercent: 99.88
    systemPercent: 0.06
    userPercent: 0.06
  cpus: 24
  distribution:
    codename: focal
    distribution: ubuntu
    version: "20.04"
  eventLogger: file
  hostname: 15150d64b287
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 10000
      size: 65535
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 10000
      size: 65535
  kernel: 5.15.0-46-generic
  linkmode: static
  logDriver: k8s-file
  memFree: 399939493888
  memTotal: 405685620736
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: Unknown
    path: /usr/local/bin/crun
    version: |-
      crun version 1.5
      commit: 54ebb8ca8bf7e6ddae2eb919f5b82d1d96863dea
      spec: 1.0.0
      +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /tmp/podman-run-1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_AUDIT_WRITE,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_MKNOD,CAP_NET_BIND_SERVICE,CAP_NET_RAW,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: ""
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/local/bin/slirp4netns
    package: Unknown
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.3
  swapFree: 0
  swapTotal: 0
  uptime: 48h 38m 43.00s (Approximately 2.00 days)
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  docker.io:
    Blocked: false
    Insecure: false
    Location: <private:5000
    MirrorByDigestOnly: false
    Mirrors: null
    Prefix: docker.io
    PullFromMirror: ""
  search:
  - docker.io
store:
  configFile: /home/jenkins/.config/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/jenkins/.local/share/containers/storage
  graphRootAllocated: 105089261568
  graphRootUsed: 6835048448
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 0
  runRoot: /tmp/podman-run-1000/containers
  volumePath: /home/jenkins/.local/share/containers/storage/volumes
version:
  APIVersion: 4.2.0
  Built: 0
  BuiltTime: Thu Jan  1 01:00:00 1970
  GitCommit: ""
  GoVersion: go1.19
  Os: linux
  OsArch: linux/amd64
  Version: 4.2.0

Package info (e.g. output of rpm -q podman or apt list podman):

not packaged

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

containers.conf

[containers]
cgroupns = "host"
cgroups = "disabled"

[engine]
cgroup_manager = "cgroupfs"
events_logger="file"

Ubuntu 22.04.1 LTS
cgroup v2 (by default)
@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Aug 26, 2022
@mheon
Copy link
Member

mheon commented Aug 26, 2022

@giuseppe PTAL

giuseppe added a commit to giuseppe/libpod that referenced this issue Aug 26, 2022
@giuseppe
service: make move to sub-cgroup non fatal
e448122 
if we are running in a container in the root cgroup, Podman tries to
move itself to a sub-cgroup.  This could be a problem in a setup where
the cgroups are not writeable, so just log a debug message and
continue, since anyway it is a best-effort operation.

Closes: containers#15498

[NO NEW TESTS NEEDED]

Signed-off-by: Giuseppe Scrivano <[email protected]>
@giuseppe giuseppe mentioned this issue Aug 26, 2022
Merged
@giuseppe
Copy link
Member

opened a PR: #15503

mheon pushed a commit to mheon/libpod that referenced this issue Sep 6, 2022
@giuseppe @mheon
service: make move to sub-cgroup non fatal
fc5cd0e 
if we are running in a container in the root cgroup, Podman tries to
move itself to a sub-cgroup.  This could be a problem in a setup where
the cgroups are not writeable, so just log a debug message and
continue, since anyway it is a best-effort operation.

Closes: containers#15498

[NO NEW TESTS NEEDED]

Signed-off-by: Giuseppe Scrivano <[email protected]>
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 18, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 18, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

service: make move to sub-cgroup non fatal giuseppe/libpod
3 participants
@giuseppe @mheon @MarkusSchoelzel