Bug: K3s OCI runtime create failed: runc create failed: unable to start container process: unable to apply cgroup configuration:

[tharwan]

Describe the bug

When using the K3s container on my ARM Mac with Colima I am unable to run deployments in the K3s cluster. The K3s cluster from Colima itself works fine.

To Reproduce

k3s = K3SContainer()
k3s.start()

env = os.environ.copy()
with NamedTemporaryFile(mode="w", delete=False) as tmp_file:
    tmp_file.write(k3s.config_yaml())
    tmp_file.close()
    env["KUBECONFIG"] = tmp_file.name
    subprocess.check_call(
        ["kubectl", "run", "busybox", "--image=busybox"],
        env=env,
    )

this will start the cluster and insert a pod. But the pod will not start and instead see a logs like

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: unable to apply cgroup configuration: failed to write 4188: write /sys/fs/cgroup/kubepods/besteffort/pod86fac936-f8f2-4cfa-8a62-77c892d5ef3d/ba75980054e795ee5b552902df784e5be731cee71fa20aacb51d4459d5f49173/cgroup.procs: no such file or directory: unknown

the logs from the K3s container look similarly for services like coreDNS.

Runtime environment

Darwin MABP-JRP4R0W3XJ 23.3.0 Darwin Kernel Version 23.3.0: Wed Dec 20 21:31:00 PST 2023; root:xnu-10002.81.5~7/RELEASE_ARM64_T6020 arm64

Python 3.11.6

docker info:

Server:
 Containers: 26
  Running: 2
  Paused: 0
  Stopped: 24
 Images: 21
 Server Version: 24.0.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc version: v1.1.10-0-g18a0cb0
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.0-35-generic
 Operating System: Ubuntu 23.10
 OSType: linux
 Architecture: aarch64
 CPUs: 4
 Total Memory: 3.816GiB
 Name: colima
 ID: 264a16ae-9445-4626-99ba-d62c497cf7e5
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

[tharwan]

Quick update. Same results on the linux machine (without Colima) from a colleague.

[tharwan]

Second update, if I remove this line everything works fine.

I could not quite figure out what this line is supposed to do.

[alexanderankin]

if you want, i can add a constructor boolean parameter for that, i have no idea what it is intended to do.

I'd also add a warning if that parameter is passed in as experimental as again, no idea what its for, and also boolean parameters are not great for maintainability, so we'd probably want to figure out what its for and avoid the extra param or generalize a bit.

[tharwan]

Maybe @ash1425 still knows? I assume it has something to do with k3s-io/k3s#7265 but I am not sure. From the k3s docs I don’t get why the volume mount might be needed.

But since we can’t get it to work on two different systems, I thing an option would be good. Otherwise we will just use our own implementation, which is also fine.

[alexanderankin]

might as well accept the contribution might help someone else, might also learn the answer if someone comes back to this issue with it. opened #592

[alexanderankin]

if it works for you ill release it @tharwan

[alexanderankin]

I did also verify that mounting it does work in the java implementation - alexanderankin/how-to-itest@766d170

…

On Thu, May 30, 2024 at 1:10 AM Florian Kühnlenz ***@***.***> wrote: Maybe @ash1425 <https://github.com/ash1425> still knows? I assume it has something to do with k3s-io/k3s#7265 <k3s-io/k3s#7265> but I am not sure. From the k3s docs I don’t get why the volume mount might be needed. But since we can’t get it to work on two different systems, I thing an option would be good. Otherwise we will just use our own implementation, which is also fine. — Reply to this email directly, view it on GitHub <#591 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACECGJGE6V6BPYNT54KNY43ZE2YEBAVCNFSM6AAAAABIO5URT6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZYGY4TENZYHE> . You are receiving this because you commented.Message ID: ***@***.***>

[alexanderankin]

@tharwan this is out now