[Carry 2535] rootless: support detach-netns mode #2723

AkihiroSuda · 2024-01-01T01:08:55Z

When RootlessKit v2.0 (rootless-containers/rootlesskit#379) is installed, containerd-rootless.sh launches it with --detach-netns so as to run the daemon in the host network namespace.

This will enable:

Accelerated (and deflaked) nerdctl pull, nerdctl push, nerdctl build, etc
Proper support for nerdctl pull 127.0.0.1:.../...
Proper support for nerdctl run --net=host

Replaces Fahed Dorgaa's PR #2535
Fixes #814
Fixes #86

Note

Rootless containerd recognizes the following environment variables to configure the behavior of RootlessKit:

...
CONTAINERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS=(auto|true|false): whether to launch rootlesskit with the "detach-netns" mode.
Defaults to "auto", which is resolved to "true" if RootlessKit >= 2.0 is installed.
The "detached-netns" mode accelerates nerdctl (pull|push|build) and enables nerdctl run --net=host,
however, there is a relatively minor drawback with BuildKit prior to v0.13:
the host loopback IP address (127.0.0.1) is exposed to Dockerfile's "RUN" instructions during nerdctl build (not nerdctl run).
The drawback is fixed in BuildKit v0.13. Upgrading from a prior version of BuildKit needs removing the old systemd unit:
containerd-rootless-setuptool.sh uninstall-buildkit && rm -f ~/.config/buildkit/buildkitd.toml

To set these variables, create ~/.config/systemd/user/containerd.service.d/override.conf as follows:

[Service]
Environment=CONTAINERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS="false"

And then run the following commands:

systemctl --user daemon-reload
systemctl --user restart containerd

https://github.com/containerd/nerdctl/blob/main/docs/rootless.md

AkihiroSuda · 2024-01-04T07:35:06Z

extras/rootless/containerd-rootless.sh

+#   The "detached-netns" mode accelerates `nerdctl (pull|push|build)` and enables `nerdctl run --net=host`,
+#   however, there is a relatively minor drawback with the current version of BuildKit:
+#   the host loopback IP address (127.0.0.1) is exposed to Dockerfile's "RUN" instructions during `nerdctl build` (not `nerdctl run`).
+#   If you want to hide 127.0.0.1 from "RUN" instructions, you should set CONTAINERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS to "false".


@containerd/nerdctl-maintainers

This is kinda breaking change.
Let me know we should release this as "nerdctl v2.0" (ahead of containerd v2.0), or we can just stick to "nerdctl v1.8".
I'm leaning toward to the former one.

nerdctl v2.0

SGTM

docs/rootless.md

extras/rootless/containerd-rootless-setuptool.sh

fahedouch · 2024-01-18T06:58:24Z

extras/rootless/containerd-rootless.sh

+#   The "detached-netns" mode accelerates `nerdctl (pull|push|build)` and enables `nerdctl run --net=host`,
+#   however, there is a relatively minor drawback with the current version of BuildKit:
+#   the host loopback IP address (127.0.0.1) is exposed to Dockerfile's "RUN" instructions during `nerdctl build` (not `nerdctl run`).
+#   If you want to hide 127.0.0.1 from "RUN" instructions, you should set CONTAINERD_ROOTLESS_ROOTLESSKIT_DETACH_NETNS to "false".


nerdctl v2.0

SGTM

pkg/cmd/container/create.go

fahedouch · 2024-01-18T07:44:05Z

pkg/netutil/subnet/subnet.go

+	var addrs []net.Addr
+	if err := rootlessutil.WithDetachedNetNSIfAny(func() error {
+		var err2 error
+		addrs, err2 = net.InterfaceAddrs()


Ah, I think this is what I was missing in my PR; I wasn't pointing to the right IP addresses, right? It's not an issue with runc as we discussed in Chicago.

fahedouch · 2024-01-18T07:49:08Z

thanks @AkihiroSuda

LGTM overall, just this small typo

Thanks again for taking the time to address this feature

AkihiroSuda · 2024-01-19T03:46:08Z

Will rebase when BuildKit v0.13 beta2 is released

Signed-off-by: Akihiro Suda <[email protected]>

When RootlessKit v2.0 (rootless-containers/rootlesskit PR 379) is installed, `containerd-rootless.sh` launches it with `--detach-netns` so as to run the daemon in the host network namespace. This will enable: - Accelerated (and deflaked) `nerdctl pull`, `nerdctl push`, `nerdctl build`, etc - Proper support for `nerdctl pull 127.0.0.1:.../...` - Proper support for `nerdctl run --net=host` Replaces Fahed Dorgaa's PR 2535 Co-authored-by: fahed dorgaa <[email protected]> Signed-off-by: Akihiro Suda <[email protected]>

extras/rootless/containerd-rootless.sh

AkihiroSuda added area/rootless Rootless mode impact/major labels Jan 1, 2024

AkihiroSuda force-pushed the detach-netns branch from fb193e0 to adb89ad Compare January 1, 2024 01:22

AkihiroSuda added this to the v1.8, maybe v2.0 milestone Jan 1, 2024

AkihiroSuda mentioned this pull request Jan 1, 2024

[rootless] Support detach netns #2535

Closed

AkihiroSuda force-pushed the detach-netns branch 3 times, most recently from 752703f to 5119337 Compare January 1, 2024 02:54

This comment was marked as resolved.

Sign in to view

AkihiroSuda force-pushed the detach-netns branch 2 times, most recently from b0df933 to 18acfa9 Compare January 2, 2024 00:35

This comment was marked as resolved.

Sign in to view

AkihiroSuda force-pushed the detach-netns branch 3 times, most recently from 79513ca to fd56c62 Compare January 4, 2024 06:15

This comment was marked as resolved.

Sign in to view

AkihiroSuda force-pushed the detach-netns branch 2 times, most recently from 3d57868 to fd2ed9c Compare January 4, 2024 07:30

AkihiroSuda commented Jan 4, 2024

View reviewed changes

AkihiroSuda force-pushed the detach-netns branch from fd2ed9c to 457ecc6 Compare January 4, 2024 07:58

AkihiroSuda modified the milestones: v2.0.1, v2.0.0 (tentative) Jan 4, 2024

AkihiroSuda force-pushed the detach-netns branch from 457ecc6 to 41135e2 Compare January 4, 2024 08:04

AkihiroSuda mentioned this pull request Jan 4, 2024

support workaround rootless loopback listening #2127

Closed

AkihiroSuda force-pushed the detach-netns branch from 41135e2 to f407714 Compare January 4, 2024 08:29

AkihiroSuda marked this pull request as ready for review January 4, 2024 08:31

AkihiroSuda mentioned this pull request Jan 5, 2024

v2.0 planning #2734

Closed

AkihiroSuda requested a review from fahedouch January 8, 2024 09:51

AkihiroSuda mentioned this pull request Jan 11, 2024

cniprovider: support rootlesskit --detach-netns (RootlessKit v2) moby/buildkit#4546

Merged

AkihiroSuda force-pushed the detach-netns branch from f407714 to 0746dcc Compare January 11, 2024 05:37

AkihiroSuda force-pushed the detach-netns branch 2 times, most recently from cebf66f to 5d31647 Compare January 15, 2024 01:38

AkihiroSuda mentioned this pull request Jan 15, 2024

Update RootlessKit to v2.0.0 (support Pasta) moby/moby#47076

Merged

AkihiroSuda force-pushed the detach-netns branch from 5d31647 to b7799e5 Compare January 15, 2024 02:58

AkihiroSuda requested review from a team January 17, 2024 23:52

ktock approved these changes Jan 18, 2024

View reviewed changes

fahedouch reviewed Jan 18, 2024

View reviewed changes

pkg/cmd/container/create.go Show resolved Hide resolved

AkihiroSuda mentioned this pull request Jan 18, 2024

--net=host: remove /var/run/docker/netns/default from OCI Runtime Config moby/moby#47100

Closed

fahedouch reviewed Jan 18, 2024

View reviewed changes

fahedouch approved these changes Jan 18, 2024

View reviewed changes

AkihiroSuda mentioned this pull request Jan 18, 2024

rootless: support detach-netns mode (Support real localhost registries and --net=host) moby/moby#47103

Draft

AkihiroSuda force-pushed the detach-netns branch 2 times, most recently from 6994e39 to d3d2da7 Compare January 31, 2024 03:25

AkihiroSuda requested a review from ktock January 31, 2024 03:27

AkihiroSuda force-pushed the detach-netns branch from d3d2da7 to c4a967d Compare January 31, 2024 03:29

containerd-rootless-setuptool.sh: avoid creating buildkitd.toml

5e38fa1

Signed-off-by: Akihiro Suda <[email protected]>

AkihiroSuda force-pushed the detach-netns branch from c4a967d to 0f458cf Compare January 31, 2024 03:59

AkihiroSuda and others added 3 commits January 31, 2024 14:43

update BuildKit (0.13.0-beta2)

58cc30c

Signed-off-by: Akihiro Suda <[email protected]>

update RootlessKit (2.0.0)

328497a

Signed-off-by: Akihiro Suda <[email protected]>

AkihiroSuda force-pushed the detach-netns branch from 0f458cf to 4892364 Compare January 31, 2024 05:49

ktock approved these changes Jan 31, 2024

View reviewed changes

AkihiroSuda merged commit 9102092 into containerd:main Jan 31, 2024
22 checks passed

fahedouch mentioned this pull request Feb 24, 2024

--net=host is not honored in rootless mode #2092

Closed

apostasie reviewed Jul 12, 2024

View reviewed changes

extras/rootless/containerd-rootless.sh Show resolved Hide resolved

github-actions bot mentioned this pull request Nov 10, 2024

Bump nerdctl from 1.7.6 to 2.0.0 kokyhm/kubespray#73

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Carry 2535] rootless: support detach-netns mode #2723

[Carry 2535] rootless: support detach-netns mode #2723

AkihiroSuda commented Jan 1, 2024 •

edited

Loading

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

AkihiroSuda Jan 4, 2024

ktock Jan 4, 2024

fahedouch Jan 18, 2024

fahedouch Jan 18, 2024

fahedouch Jan 18, 2024

fahedouch commented Jan 18, 2024

AkihiroSuda commented Jan 19, 2024

[Carry 2535] rootless: support detach-netns mode #2723

[Carry 2535] rootless: support detach-netns mode #2723

Conversation

AkihiroSuda commented Jan 1, 2024 • edited Loading

This comment was marked as resolved.

This comment was marked as resolved.

This comment was marked as resolved.

AkihiroSuda Jan 4, 2024

Choose a reason for hiding this comment

ktock Jan 4, 2024

Choose a reason for hiding this comment

fahedouch Jan 18, 2024

Choose a reason for hiding this comment

fahedouch Jan 18, 2024

Choose a reason for hiding this comment

fahedouch Jan 18, 2024

Choose a reason for hiding this comment

fahedouch commented Jan 18, 2024

AkihiroSuda commented Jan 19, 2024

AkihiroSuda commented Jan 1, 2024 •

edited

Loading