Running with rootless podman seems not to respect property Delegate=yes #2916

plluksie · 2022-09-08T08:25:35Z

What happened: I'm not able to create cluster.

What you expected to happen: I'm able to create cluster.

How to reproduce it (as minimally and precisely as possible):

❯ sudo cat /etc/systemd/system/[email protected]/delegate.conf
[Service]
Delegate=yes

❯ KIND_EXPERIMENTAL_PROVIDER=podman kind -v 5 create cluster
using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
ERROR: failed to create cluster: running kind with rootless provider requires setting systemd property "Delegate=yes", see https://kind.sigs.k8s.io/docs/user/rootless/
Stack Trace: 
sigs.k8s.io/kind/pkg/errors.New
	sigs.k8s.io/kind/pkg/errors/errors.go:28
sigs.k8s.io/kind/pkg/cluster/internal/create.validateProvider
	sigs.k8s.io/kind/pkg/cluster/internal/create/create.go:253
sigs.k8s.io/kind/pkg/cluster/internal/create.Cluster
	sigs.k8s.io/kind/pkg/cluster/internal/create/create.go:70
sigs.k8s.io/kind/pkg/cluster.(*Provider).Create
	sigs.k8s.io/kind/pkg/cluster/provider.go:182
sigs.k8s.io/kind/pkg/cmd/kind/create/cluster.runE
	sigs.k8s.io/kind/pkg/cmd/kind/create/cluster/createcluster.go:80
sigs.k8s.io/kind/pkg/cmd/kind/create/cluster.NewCommand.func1
	sigs.k8s.io/kind/pkg/cmd/kind/create/cluster/createcluster.go:55
github.com/spf13/cobra.(*Command).execute
	github.com/spf13/[email protected]/command.go:856
github.com/spf13/cobra.(*Command).ExecuteC
	github.com/spf13/[email protected]/command.go:974
github.com/spf13/cobra.(*Command).Execute
	github.com/spf13/[email protected]/command.go:902
sigs.k8s.io/kind/cmd/kind/app.Run
	sigs.k8s.io/kind/cmd/kind/app/main.go:53
sigs.k8s.io/kind/cmd/kind/app.Main
	sigs.k8s.io/kind/cmd/kind/app/main.go:35
main.main
	sigs.k8s.io/kind/main.go:25
runtime.main
	runtime/proc.go:250
runtime.goexit
	runtime/asm_amd64.s:1571

Following @cwrau recommendation expressed in #2872 (comment):

❯ KIND_EXPERIMENTAL_PROVIDER=podman systemd-run --user --property=Delegate=yes kind -v 5 create cluster
Running as unit: run-r28fa7c60a5144fdea8b349e1033c4370.service

❯ systemctl --user | grep run-r28fa7c60a5144fdea8b349e1033c4370.service
  ● run-r28fa7c60a5144fdea8b349e1033c4370.service     loaded failed     failed    /home/lsiemiradzki/.local/bin/kind -v 5 create cluster                            

❯ journalctl -u run-r28fa7c60a5144fdea8b349e1033c4370.service
-- Logs begin at Tue 2022-05-24 15:06:54 CEST, end at Thu 2022-09-08 10:03:16 CEST. --
-- No entries --

Anything else we need to know?:
It was working fine 1 month ago. In the meantime there were some standard package upgrades and security updates. I cannot precisely state what exactly has changed. It is Ubuntu:
❯ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 20.04.5 LTS
Release: 20.04
Codename: focal

Environment:

kind version: (use kind version): kind v0.14.0 go1.18.2 linux/amd64
Kubernetes version: (use kubectl version): Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.13", GitCommit:"a43c0904d0de10f92aa3956c74489c45e6453d6e", GitTreeState:"clean", BuildDate:"2022-08-18T02:28:16Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}
Docker version: (use docker info): I don't have docker, I'm using podman:
❯ podman info

host:
  arch: amd64
  buildahVersion: 1.27.0-dev
  cgroupControllers:
  - memory
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon_100:2.1.2~0_amd64
    path: /usr/libexec/podman/conmon
    version: 'conmon version 2.1.2, commit: '
  cpuUtilization:
    idlePercent: 94.79
    systemPercent: 1.57
    userPercent: 3.64
  cpus: 8
  distribution:
    codename: focal
    distribution: ubuntu
    version: "20.04"
  eventLogger: journald
  hostname: pollrnd1007
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.14.0-1051-oem
  linkmode: dynamic
  logDriver: journald
  memFree: 5156880384
  memTotal: 16142012416
  networkBackend: cni
  ociRuntime:
    name: crun
    package: crun_100:1.2-2_amd64
    path: /usr/bin/crun
    version: |-
      crun version UNKNOWN
      commit: ea1fe3938eefa14eb707f1d22adff4db670645d6
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns_100:1.1.8-4_amd64
    version: |-
      slirp4netns version 1.1.8
      commit: unknown
      libslirp: 4.3.1-git
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.4.3
  swapFree: 24574423040
  swapTotal: 24574423040
  uptime: 0h 27m 25.00s
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries:
  search:
  - docker.io
  - quay.io
store:
  configFile: /home/lsiemiradzki/.config/containers/storage.conf
  containerStore:
    number: 5
    paused: 0
    running: 1
    stopped: 4
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/lsiemiradzki/.local/share/containers/storage
  graphRootAllocated: 475998666752
  graphRootUsed: 73949675520
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 61
  runRoot: /run/user/1000/containers
  volumePath: /home/lsiemiradzki/.local/share/containers/storage/volumes
version:
  APIVersion: 4.2.0-rc2
  Built: 1659567994
  BuiltTime: Thu Aug  4 01:06:34 2022
  GitCommit: ""
  GoVersion: devel go1.20-f28fa952b5 Wed Aug 3 19:52:27 2022 +0000
  Os: linux
  OsArch: linux/amd64
  Version: 4.2.0-rc2

OS (e.g. from /etc/os-release): Ubuntu 20.04.5 LTS

The text was updated successfully, but these errors were encountered:

plluksie · 2022-09-08T10:45:52Z

Seems to be an issue with podman. It works with

❯ podman -v 
podman version 3.4.2

I'll check with podman project.

deftdawg · 2023-01-12T22:46:41Z

@plluksie I'm having the same issue with podman 4.3.1, did you get it resolved without downgrading or is there an issue in podman for it?

cloudguruab · 2023-05-04T18:41:02Z

@deftdawg you ever figure it out? I'm running podman 4.3.1 with the same issue

BenTheElder · 2023-05-04T18:43:34Z

Perhaps the systemd-run note https://kind.sigs.k8s.io/docs/user/rootless/#creating-a-kind-cluster-with-rootless-podman ?

deftdawg · 2023-05-04T18:44:01Z

@deftdawg you ever figure it out? I'm running podman 4.3.1 with the same issue

No, gave up and switched to running docker-rootless

cloudguruab · 2023-05-04T18:44:50Z

Ah, thanks for the update @BenTheElder & @deftdawg

sword-jin · 2023-12-22T03:45:42Z

I encounter the same problem:

❯ sudo cat /etc/systemd/system/[email protected]/delegate.conf
[Service]
Delegate=yes

> podman version

Client:       Podman Engine
Version:      5.0.0-dev
API Version:  5.0.0-dev
Go Version:   go1.21.1
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/amd64


❯ KIND_EXPERIMENTAL_PROVIDER=podman kind create cluster --name test --config kind-config.yaml
using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
ERROR: failed to create cluster: running kind with rootless provider requires setting systemd property "Delegate=yes", see https://kind.sigs.k8s.io/docs/user/rootless/

Overmorrow1501 · 2024-01-11T16:49:06Z

I encounter the same problem:

❯ sudo cat /etc/systemd/system/[email protected]/delegate.conf
[Service]
Delegate=yes

> podman version

Client:       Podman Engine
Version:      5.0.0-dev
API Version:  5.0.0-dev
Go Version:   go1.21.1
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/amd64


❯ KIND_EXPERIMENTAL_PROVIDER=podman kind create cluster --name test --config kind-config.yaml
using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
ERROR: failed to create cluster: running kind with rootless provider requires setting systemd property "Delegate=yes", see https://kind.sigs.k8s.io/docs/user/rootless/

I got the same problem

sword-jin · 2024-01-12T06:05:51Z

I encounter the same problem:

❯ sudo cat /etc/systemd/system/[email protected]/delegate.conf
[Service]
Delegate=yes

> podman version

Client:       Podman Engine
Version:      5.0.0-dev
API Version:  5.0.0-dev
Go Version:   go1.21.1
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/amd64


❯ KIND_EXPERIMENTAL_PROVIDER=podman kind create cluster --name test --config kind-config.yaml
using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
ERROR: failed to create cluster: running kind with rootless provider requires setting systemd property "Delegate=yes", see https://kind.sigs.k8s.io/docs/user/rootless/

I got the same problem

update your crun version

jocutajar · 2024-04-15T14:44:06Z

The unit file seems to be ignored on my systemd as well (debian bookworm).

This -p "Delegate=yes" seems to have done it:

systemd-run --scope --user -p "Delegate=yes" kind create cluster

Found the trick in regard to LXC.

podman version 4.3.1
crun version 1.8.1
systemd 252 (252.22-1~deb12u1)
kind version 0.22.0

jsmrcka · 2024-04-18T10:42:00Z

The unit file seems to be ignored on my systemd as well (debian bookworm).

This -p "Delegate=yes" seems to have done it:
systemd-run --scope --user -p "Delegate=yes" kind create cluster
Found the trick in regard to LXC.
* podman version 4.3.1

* crun version 1.8.1

* systemd 252 (252.22-1~deb12u1)

* kind version 0.22.0

That did the trick for me too, thanks.

podman 4.9.3
crun 1.14.4
systemd 255 (255.4-1-manjaro)
kind 0.22.0

domeales-paloit · 2024-06-21T09:47:04Z

The unit file seems to be ignored on my systemd as well (debian bookworm).

This -p "Delegate=yes" seems to have done it:
systemd-run --scope --user -p "Delegate=yes" kind create cluster
Found the trick in regard to LXC.

podman version 4.3.1

crun version 1.8.1

systemd 252 (252.22-1~deb12u1)

kind version 0.22.0

Worked for me too, cheers!

podman 5.1.1
crun 1.15
systemd 253 (253.5-1ubuntu6.1)
kind 0.23.0
Ubuntu 23.10

BenTheElder · 2024-06-21T17:31:24Z

mind adding an update to https://kind.sigs.k8s.io/docs/user/rootless/#creating-a-kind-cluster-with-rootless-podman ?

jstaf · 2024-07-10T02:16:51Z

None of these suggestions seem to work on OpenSUSE Tumbleweed. To reproduce:

Install Tumbleweed
Install either nerdctl or podman in rootless mode
Follow all the rootless setup steps here: https://kind.sigs.k8s.io/docs/user/rootless/
kind create cluster with either podman or nerdctl fails with:

ERROR: failed to create cluster: running kind with rootless provider requires setting systemd property "Delegate=yes", see https://kind.sigs.k8s.io/docs/user/rootless/

khanhtc1202 · 2024-08-20T15:48:30Z

@BenTheElder I submitted a patch for the docs 🙏

plluksie added the kind/bug Categorizes issue or PR as related to a bug. label Sep 8, 2022

plluksie closed this as completed Sep 8, 2022

BenTheElder added the area/provider/podman Issues or PRs related to podman label Sep 9, 2022

domeales-paloit mentioned this issue Jun 21, 2024

Additional information for troubleshooting southsidedean/deploy-kind-using-podman-ubuntu#1

Open

khanhtc1202 mentioned this issue Aug 20, 2024

Add install note to rootless docs #3721

Merged

cmoulliard mentioned this issue Nov 5, 2024

podman rootless support cnoe-io/idpbuilder#356

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Running with rootless podman seems not to respect property Delegate=yes #2916

Running with rootless podman seems not to respect property Delegate=yes #2916

plluksie commented Sep 8, 2022 •

edited

Loading

plluksie commented Sep 8, 2022

deftdawg commented Jan 12, 2023

cloudguruab commented May 4, 2023

BenTheElder commented May 4, 2023

deftdawg commented May 4, 2023

cloudguruab commented May 4, 2023

sword-jin commented Dec 22, 2023

Overmorrow1501 commented Jan 11, 2024

sword-jin commented Jan 12, 2024

jocutajar commented Apr 15, 2024

jsmrcka commented Apr 18, 2024 •

edited

Loading

domeales-paloit commented Jun 21, 2024 •

edited

Loading

BenTheElder commented Jun 21, 2024

jstaf commented Jul 10, 2024

khanhtc1202 commented Aug 20, 2024

Running with rootless podman seems not to respect property Delegate=yes #2916

Running with rootless podman seems not to respect property Delegate=yes #2916

Comments

plluksie commented Sep 8, 2022 • edited Loading

plluksie commented Sep 8, 2022

deftdawg commented Jan 12, 2023

cloudguruab commented May 4, 2023

BenTheElder commented May 4, 2023

deftdawg commented May 4, 2023

cloudguruab commented May 4, 2023

sword-jin commented Dec 22, 2023

Overmorrow1501 commented Jan 11, 2024

sword-jin commented Jan 12, 2024

jocutajar commented Apr 15, 2024

jsmrcka commented Apr 18, 2024 • edited Loading

domeales-paloit commented Jun 21, 2024 • edited Loading

BenTheElder commented Jun 21, 2024

jstaf commented Jul 10, 2024

khanhtc1202 commented Aug 20, 2024

plluksie commented Sep 8, 2022 •

edited

Loading

jsmrcka commented Apr 18, 2024 •

edited

Loading

domeales-paloit commented Jun 21, 2024 •

edited

Loading