Direct access to containerd socket?

[deitch]

Description

Is there a way to get direct access to a containerd socket running on a lima VM? The use case I am thinking of is, rather than having to execute a cli - limactl shell <name> nerdctl (or limactl shell <name> ctr - we could communicate directly via ctr or the library.

[jandubois]

There is nothing magical about it; you can just forward it from the VM to the host, the same way the docker example does it:

lima/examples/docker.yaml

Lines 77 to 79 in baca0e7

	portForwards:
	- guestSocket: "/run/user/{{.UID}}/docker.sock"
	hostSocket: "{{.Dir}}/sock/docker.sock"

Keep in mind though that not all operations can be performed over the socket, but might need direct access to the storage layer (e.g. nerdctl build).

[jandubois]

I could not test from host because I am on a Mac

That's the part I was wondering about, if forwarding to a symlink is working or not. So if you ever get around to testing, please let us know if it worked.

I don't understand why you need to create a symlink for the system socket; it should already be in a fixed location at /run/containerd/containerd.sock.

[deitch]

I don't understand why you need to create a symlink for the system socket; it should already be in a fixed location at /run/containerd/containerd.sock

I don't really. I was just experimenting with setting the symlink as system rather than user.

That's the part I was wondering about, if forwarding to a symlink is working or not. So if you ever get around to testing, please let us know if it worked.

Will do.

[deitch]

First, it actually does work. I built ctr on my macOS and ran it:

$ ctr --address $HOME/.containerd-user.sock image ls
REF TYPE DIGEST SIZE PLATFORMS LABELS
$ ctr --address $HOME/.containerd-user.sock image pull docker.io/library/alpine:3.17
docker.io/library/alpine:3.17:                                                 resolved       |++++++++++++++++++++++++++++++++++++++|
index-sha256:8914eb54f968791faf6a8638949e480fef81e697984fba772b3976835194c6d4: done           |++++++++++++++++++++++++++++++++++++++|
elapsed: 4.3 s                                                                 total:   0.0 B (0.0 B/s)
unpacking darwin/arm64/v8 sha256:8914eb54f968791faf6a8638949e480fef81e697984fba772b3976835194c6d4...
ctr: no match for platform in manifest sha256:8914eb54f968791faf6a8638949e480fef81e697984fba772b3976835194c6d4: not found
$ ctr --address $HOME/.containerd-user.sock image ls
ERRO[0000] failed calculating size for image docker.io/library/alpine:3.17  error="content digest sha256:af06af3514c44a964d3b905b498cf6493db8f1cde7c10e078213a89c87308ba0: not found"
REF                           TYPE                                                      DIGEST                                                                  SIZE    PLATFORMS                                                                                LABELS
docker.io/library/alpine:3.17 application/vnd.docker.distribution.manifest.list.v2+json sha256:8914eb54f968791faf6a8638949e480fef81e697984fba772b3976835194c6d4 2.1 KiB linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8,linux/ppc64le,linux/s390x -

There are issues, but I don't think they are relating to the forward-to-symlink.

Second, I found that the provision has an issue if you stop and start the VM. When you do, the child_pid changes, so the symlink is out of date, but continues to exist. Any method for ensuring provisioning runs with every start, not just initial boot?

[jandubois]

Second, I found that the provision has an issue if you stop and start the VM. When you do, the child_pid changes, so the symlink is out of date, but continues to exist. Any method for ensuring provisioning runs with every start, not just initial boot?

The script runs on every boot. I think the problem is just that the destination already exists, so maybe try to always delete it first (again, untested):

provision:
- mode: user
  script: |
    rm -f $HOME/containerd.sock
    ln -s /proc/$(cat $XDG_RUNTIME_DIR/containerd-rootless/child_pid)/root/run/containerd/containerd.sock $HOME/containerd.sock

[deitch]

Yup, that was it. Thank @jandubois !