[BUG] Podman rootless not working as described

[aksdb]

What did you do

• How was the cluster created?

  export DOCKER_HOST=unix:///run/user/1000/podman/podman.sock
  k3d cluster create test
  

What did you expect to happen

The cluster should be created.

Screenshots or terminal output

INFO[0000] Prep: Network                                
INFO[0000] Created network 'k3d-test'                   
INFO[0000] Created image volume k3d-test-images         
INFO[0000] Starting new tools node...                   
ERRO[0000] Failed to run tools container for cluster 'test' 
INFO[0001] Creating node 'k3d-test-server-0'            
INFO[0001] Creating LoadBalancer 'k3d-test-serverlb'    
INFO[0002] Using the k3d-tools node to gather environment information 
INFO[0002] Starting new tools node...                   
ERRO[0002] Failed to run tools container for cluster 'test' 
ERRO[0002] failed to gather environment information used for cluster creation: failed to run k3d-tools node for cluster 'test': failed to create node 'k3d-test-tools': runtime failed to create node 'k3d-test-tools': failed to create container for node 'k3d-test-tools': docker failed to create container 'k3d-test-tools': Error response from daemon: make cli opts(): error making volume mountpoint for volume /var/run/docker.sock: mkdir /var/run/docker.sock: permission denied 
ERRO[0002] Failed to create cluster >>> Rolling Back    
INFO[0002] Deleting cluster 'test'                      
INFO[0003] Deleting cluster network 'k3d-test'          
INFO[0003] Deleting 2 attached volumes...               
WARN[0003] Failed to delete volume 'k3d-test-images' of cluster 'test': failed to find volume 'k3d-test-images': Error: No such volume: k3d-test-images -> Try to delete it manually 
FATA[0003] Cluster creation FAILED, all changes have been rolled back! 

Which OS & Architecture

Linux, x64

Which version of k3d

k3d version v5.4.1
k3s version v1.22.7-k3s1 (default)

Which version of docker podman

❯ podman version
Client:       Podman Engine
Version:      4.0.3
API Version:  4.0.3
Go Version:   go1.18
Git Commit:   62534053086fdeba7b93117e7c4dc6e797835a3e
Built:        Mon Apr  4 14:54:02 2022
OS/Arch:      linux/amd64

[aksdb]

Judging by the debug output, in contrast to the docs, one has to set DOCKER_SOCK=/run/user/... instead of DOCKER_HOST (notice: different env var, and no protocol scheme).

Using that, the cluster gets created, but still can't be accessed. Might be a different problem then, though. I am still investigating.

[narthollis]

I can confirm that setting DOCKER_SOCK=$XDG_RUNTIME_DIR/podman/podman.sock has succeded further where DOCKER_HOST=unix:// did not.

[iwilltry42]

Hi @aksdb , thanks for opening this issue!
This should be fixed by #1045 which landed in v5.4.2
Feel free to reopen this issue if it's still a problem in v5.4.2 👍

[Nepoxx]

I am encountering this issue with k3d v5.4.9

> k3d --version
k3d version v5.4.9
k3s version v1.25.7-k3s1 (default)

> k3d cluster create test
INFO[0000] Prep: Network                                
INFO[0000] Created network 'k3d-test'                   
INFO[0000] Created image volume k3d-test-images         
INFO[0000] Starting new tools node...                   
ERRO[0000] Failed to run tools container for cluster 'test' 
INFO[0001] Creating node 'k3d-test-server-0'            
INFO[0001] Creating LoadBalancer 'k3d-test-serverlb'    
INFO[0001] Using the k3d-tools node to gather environment information 
INFO[0001] Starting new tools node...                   
ERRO[0001] Failed to run tools container for cluster 'test' 
ERRO[0001] failed to gather environment information used for cluster creation: failed to run k3d-tools node for cluster 'test': failed to create node 'k3d-test-tools': runtime failed to create node 'k3d-test-tools': failed to create container for node 'k3d-test-tools': docker failed to create container 'k3d-test-tools': Error response from daemon: make cli opts(): making volume mountpoint for volume /var/run/docker.sock: mkdir /var/run/docker.sock: permission denied 
ERRO[0001] Failed to create cluster >>> Rolling Back    
INFO[0001] Deleting cluster 'test'                      
INFO[0001] Deleting cluster network 'k3d-test'          
INFO[0001] Deleting 1 attached volumes...               
FATA[0001] Cluster creation FAILED, all changes have been rolled back! 

> podman version
Client:       Podman Engine
Version:      4.4.4
API Version:  4.4.4
Go Version:   go1.18.1
Built:        Wed Dec 31 19:00:00 1969
OS/Arch:      linux/amd64

> curl -v --unix-socket /run/user/1000/podman/podman.sock http://localhost/version
*   Trying /run/user/1000/podman/podman.sock:0...
* Connected to localhost (/run/user/1000/podman/podman.sock) port 80 (#0)
> GET /version HTTP/1.1
> Host: localhost
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Api-Version: 1.41
< Content-Type: application/json
< Libpod-Api-Version: 4.4.4
< Server: Libpod/4.4.4 (linux)
< X-Reference-Id: 0xc000469248
< Date: Wed, 05 Apr 2023 13:25:41 GMT
< Content-Length: 934
< 
{"Platform":{"Name":"linux/amd64/ubuntu-22.04"},"Components":[{"Name":"Podman Engine","Version":"4.4.4","Details":{"APIVersion":"4.4.4","Arch":"amd64","BuildTime":"1969-12-31T19:00:00-05:00","Experimental":"false","GitCommit":"","GoVersion":"go1.18.1","KernelVersion":"5.19.0-38-generic","MinAPIVersion":"4.0.0","Os":"linux"}},{"Name":"Conmon","Version":"conmon version 2.1.7, commit: ","Details":{"Package":"conmon_2:2.1.7-0ubuntu22.04+obs15.12_amd64"}},{"Name":"OCI Runtime (crun)","Version":"crun version 1.8.3\ncommit: 59f2beb7efb0d35611d5818fd0311883676f6f7e\nrundir: /run/user/1000/crun\nspec: 1.0.0\n+SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL","Details":{"Package":"crun_101:1.8.3-0ubuntu22.04+obs54.3_amd64"}}],"Version":"4.4.4","ApiVersion":"1.41","MinAPIVersion":"1.24","GitCommit":"","GoVersion":"go1.18.1","Os":"linux","Arch":"amd64","KernelVersion":"5.19.0-38-generic","BuildTime":"1969-12-31T19:00:00-05:00"}
* Connection #0 to host localhost left intact

> echo $DOCKER_HOST
unix:///run/user/1000/podman/podman.sock

As mentionned above, setting DOCKER_SOCK gets a bit further but ends up stuck at INFO[0002] Starting Node 'k3d-test-server-0'