Disable rootless mode except RootlessCgMgr when executed as the root in userns (fix Docker-in-LXD regression) #1862

AkihiroSuda · 2018-08-10T07:12:37Z

This PR decomposes libcontainer/configs.Config.Rootless bool into RootlessEUID bool and
RootlessCgroups bool, so as to make "runc-in-userns" to be more compatible with "rootful" runc.

RootlessEUID denotes that runc is being executed as a non-root user (euid != 0) in
the current user namespace. RootlessEUID is almost identical to the former Rootless
except cgroups stuff.

RootlessCgroups denotes that runc is unlikely to have the full access to cgroups.
RootlessCgroups is set to false if runc is executed as the root (euid == 0) in the initial namespace.
Otherwise RootlessCgroups is set to true.
(Hint: if RootlessEUID is true, RootlessCgroups becomes true as well)

When runc is executed as the root (euid == 0) in an user namespace (e.g. by Docker-in-LXD, Podman, Usernetes),
RootlessEUID is set to false but RootlessCgroups is set to true.
So, "runc-in-userns" behaves almost same as "rootful" runc except that cgroups errors are ignored.

This PR does not have any impact on CLI flags and state.json.

Note about CLI:

Now runc --rootless=(auto|true|false) CLI flag is only used for setting RootlessCgroups.
Now runc spec --rootless is only required when RootlessEUID is set to true.
For runc-in-userns, runc spec without --rootless should work, when sufficient numbers of
UID/GID are mapped.

Note about $XDG_RUNTIME_DIR (e.g. /run/user/1000):

$XDG_RUNTIME_DIR is ignored if runc is being executed as the root (euid == 0) in the initial namespace, for backward compatibility.
(/run/runc is used)
If runc is executed as the root (euid == 0) in an user namespace, $XDG_RUNTIME_DIR is honored if $USER != "" && $USER != "root".
This allows unprivileged users to allow execute runc as the root in userns, without mounting writable /run/runc.

Note about state.json:

rootless is set to true when RootlessEUID == true && RootlessCgroups == true.

Signed-off-by: Akihiro Suda [email protected]

rhatdan · 2018-08-21T18:48:22Z

@giuseppe PTAL

giuseppe · 2018-08-22T08:00:23Z

libcontainer/configs/config.go

 	// IntelRdt specifies settings for Intel RDT/CAT group that the container is placed into
 	// to limit the resources (e.g., L3 cache) the container has available
 	IntelRdt *IntelRdt `json:"intel_rdt,omitempty"`
+
+	// RootlessEUID specifies is set when the runc was launched with non-zero EUID.


please drop "specifies" or "is set"

giuseppe · 2018-08-22T08:15:48Z

rootless_linux.go

+	}
+	// euid = 0, in a userns.
+	u := os.Getenv("USER")
+	return u != "" && u != "root"


could this be something like?

u, ok := os.LookupEnv("USER") return !ok || u != "root"

Otherwise it breaks current users like Podman (and probably Buildah as well) that don't pass $USER down to the runtime

kolyshkin · 2018-08-23T22:41:15Z

this needs a rebase

AkihiroSuda · 2018-08-24T06:01:53Z

rebased

AkihiroSuda · 2018-08-28T15:48:57Z

@giuseppe LGTY?

giuseppe · 2018-08-28T15:53:25Z

@AkihiroSuda yes, I've tested it with podman and it works well for me

thaJeztah · 2018-08-28T20:19:02Z

/cc @crosbymichael ptal

giuseppe · 2018-08-31T20:47:11Z

an error I've seen (not introduced by this patch but that could probably be fixed as part of it), is:

create a rootless user namespace with multiple IDs mapped in, then create a runc container that has:

    "uidMappings": [
      {
        "hostID": 1,
        "containerID": 0,
        "size": 1
      },
      {
        "hostID": 0,
        "containerID": 1,
        "size": 1
      }
    ],
    "gidMappings": [
      {
        "hostID": 1,
        "containerID": 0,
        "size": 1
      },
      {
        "hostID": 0,
        "containerID": 1,
        "size": 1
      }
    ],

if I try to exec (from the first user namespace) into the container, I get:

$ runc exec foo echo hi
exec failed: container_linux.go:337: starting container process caused "process_linux.go:90: adding pid 5652 to cgroups caused \"failed to write 5652 to cgroup.procs: write /sys/fs/cgroup/systemd/user.slice/user-1000.slice/[email protected]/gnome-terminal-server.service/foo/cgroup.procs: permission denied\""

…in userns This PR decomposes `libcontainer/configs.Config.Rootless bool` into `RootlessEUID bool` and `RootlessCgroups bool`, so as to make "runc-in-userns" to be more compatible with "rootful" runc. `RootlessEUID` denotes that runc is being executed as a non-root user (euid != 0) in the current user namespace. `RootlessEUID` is almost identical to the former `Rootless` except cgroups stuff. `RootlessCgroups` denotes that runc is unlikely to have the full access to cgroups. `RootlessCgroups` is set to false if runc is executed as the root (euid == 0) in the initial namespace. Otherwise `RootlessCgroups` is set to true. (Hint: if `RootlessEUID` is true, `RootlessCgroups` becomes true as well) When runc is executed as the root (euid == 0) in an user namespace (e.g. by Docker-in-LXD, Podman, Usernetes), `RootlessEUID` is set to false but `RootlessCgroups` is set to true. So, "runc-in-userns" behaves almost same as "rootful" runc except that cgroups errors are ignored. This PR does not have any impact on CLI flags and `state.json`. Note about CLI: * Now `runc --rootless=(auto|true|false)` CLI flag is only used for setting `RootlessCgroups`. * Now `runc spec --rootless` is only required when `RootlessEUID` is set to true. For runc-in-userns, `runc spec` without `--rootless` should work, when sufficient numbers of UID/GID are mapped. Note about `$XDG_RUNTIME_DIR` (e.g. `/run/user/1000`): * `$XDG_RUNTIME_DIR` is ignored if runc is being executed as the root (euid == 0) in the initial namespace, for backward compatibility. (`/run/runc` is used) * If runc is executed as the root (euid == 0) in an user namespace, `$XDG_RUNTIME_DIR` is honored if `$USER != "" && $USER != "root"`. This allows unprivileged users to allow execute runc as the root in userns, without mounting writable `/run/runc`. Note about `state.json`: * `rootless` is set to true when `RootlessEUID == true && RootlessCgroups == true`. Signed-off-by: Akihiro Suda <[email protected]>

AkihiroSuda · 2018-09-07T06:05:24Z

@giuseppe should be fixed in the latest revision

giuseppe · 2018-09-18T21:22:47Z

@giuseppe should be fixed in the latest revision

yes thanks, I've verified that it works now

Assuming a host UID of 1000, the UID mapping inside the user namespace created by rootless podman for the toolbox container was: 0 1000 1 1 100000 65536 ... which was the same as seen from the host: 0 1000 1 1 100000 65536 Therefore, when running with an UID of 1000 inside the container, it got mapped to UID 100999 on the host. That means, for example, files created by the user inside the container end up looking funny from the host. This is addressed by creating another user namespace that's a child of the initial user namespace created by rootless podman. Assuming a host UID of 1000, the UID mapping inside this child namespace is: 1000 0 1 0 1 1000 1001 1001 64536 ... which when seen from the host is: 1000 1000 1 0 100000 1000 1001 101000 64536 This means that UID 1000 inside the child namespace is mapped to the same UID 1000 on the host via the intermediate namespace created by rootless podman. UIDs 0 to 999 inside the child namespace are mapped to UIDs 100000 to 100999 in the host. This change requires this runc pull request to work: opencontainers/runc#1862 As suggested by Giuseppe Scrivano.

AkihiroSuda · 2018-09-26T13:18:52Z

ping @cyphar

AkihiroSuda · 2018-10-03T05:41:39Z

This PR seems also needed for Docker-on-Chromebook https://www.reddit.com/r/Crostini/comments/9jabhq/docker_now_working/

debarshiray · 2018-10-05T18:22:15Z

Without this PR one can't use rootless podman create to map the host UID into a container, which is a deal-breaker for fedora-toolbox.

caniszczyk · 2018-10-09T18:02:03Z

ping @opencontainers/runc-maintainers

AkihiroSuda · 2018-10-10T03:46:13Z

This is being discussed at the dev ML: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/jOaNYre5M40

crosbymichael · 2018-10-10T14:56:30Z

I'm reviewing and testing this today

crosbymichael · 2018-10-11T17:28:45Z

LGTM

mrunalp · 2018-10-15T14:18:32Z

I'll review this today.

mrunalp · 2018-10-16T00:30:31Z

LGTM

AkihiroSuda · 2018-10-16T04:22:07Z

Thanks Michael and Mrunal!

Could you also review #1880 (very easy) and #1869 (complicated) when you have time?

This was referenced Aug 10, 2018

rootless: fix Docker-in-LXD regression #1833

Closed

Don't always enable rootless mode in userns #1837

Closed

AkihiroSuda force-pushed the decompose-rootless-pr branch 4 times, most recently from 89d6837 to fceaba2 Compare August 10, 2018 09:31

AkihiroSuda changed the title ~~Disable rootless mode except RootlessCgMgr when executed as the root in userns~~ Disable rootless mode except RootlessCgMgr when executed as the root in userns (fix Docker-in-LXD regression) Aug 10, 2018

AkihiroSuda force-pushed the decompose-rootless-pr branch 3 times, most recently from 26e5251 to 3c63ff4 Compare August 10, 2018 12:24

AkihiroSuda mentioned this pull request Aug 10, 2018

experimental support for rootless mode containerd/containerd#2006

Closed

thaJeztah mentioned this pull request Aug 15, 2018

[DO NOT MERGE] Bump runc to beadf0e moby/moby#37568

Closed

AkihiroSuda mentioned this pull request Aug 20, 2018

Rootless build: cannot specify gid= mount options for unmapped gid in rootless containers containers/podman#1147

Closed

giuseppe reviewed Aug 22, 2018

View reviewed changes

AkihiroSuda force-pushed the decompose-rootless-pr branch 3 times, most recently from 95dbfbc to 0851385 Compare August 22, 2018 09:17

kolyshkin mentioned this pull request Aug 23, 2018

Update containerd client and dependencies to v1.2.0-rc.1 moby/moby#37710

Merged

AkihiroSuda force-pushed the decompose-rootless-pr branch from 0851385 to 3334b2e Compare August 24, 2018 05:19

cyphar self-requested a review August 27, 2018 22:52

cyphar self-assigned this Aug 27, 2018

cyphar added the rootless-containers label Aug 27, 2018

AkihiroSuda force-pushed the decompose-rootless-pr branch from 3334b2e to 06f789c Compare September 7, 2018 06:05

kolyshkin mentioned this pull request Sep 11, 2018

Can't restore containers moby/moby#35691

Closed

This was referenced Sep 21, 2018

Silverblue toolbox script broke because rootless 'podman create' can't chown $XDG_DATA_HOME/containers/storage/vfs containers/podman#1522

Closed

Silverblue toolbox script broke because rootless 'podman start' fails containers/podman#1535

Closed

This was referenced Sep 26, 2018

Rootless podman loses capabilities for binaries containers/podman#1550

Closed

Silverblue toolbox can't be used after rebooting the host containers/podman#1554

Closed

AkihiroSuda mentioned this pull request Oct 3, 2018

Can't pull docker images containing device nodes when running docker inside user namespace moby/moby#35245

Open

cyphar mentioned this pull request Oct 10, 2018

Question: runc 1.0.0 release plan #1905

Closed

AkihiroSuda mentioned this pull request Oct 15, 2018

[UMBRELLA] patch submission status rootless-containers/usernetes#42

Closed

3 tasks

mrunalp merged commit a00bf01 into opencontainers:master Oct 16, 2018

This was referenced Oct 16, 2018

simplify rootless moby/buildkit#689

Merged

bump up runc moby/moby#38048

Merged

xiaochenshen mentioned this pull request Oct 16, 2018

libcontainer: intelrdt: add support for Intel RDT/MBA in runc #1632

Merged

debarshiray mentioned this pull request Oct 23, 2018

setup user: cannot set any additional groups in a rootless container containers/toolbox#4

Closed

thaJeztah mentioned this pull request Mar 22, 2021

libcontainer/system: move userns utilities, remove GetParentNSeuid, UIDMapInUserNS #2850

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Disable rootless mode except RootlessCgMgr when executed as the root in userns (fix Docker-in-LXD regression) #1862

Disable rootless mode except RootlessCgMgr when executed as the root in userns (fix Docker-in-LXD regression) #1862

AkihiroSuda commented Aug 10, 2018

rhatdan commented Aug 21, 2018

giuseppe Aug 22, 2018

AkihiroSuda Aug 22, 2018

giuseppe Aug 22, 2018

AkihiroSuda Aug 22, 2018

kolyshkin commented Aug 23, 2018

AkihiroSuda commented Aug 24, 2018

AkihiroSuda commented Aug 28, 2018

giuseppe commented Aug 28, 2018

thaJeztah commented Aug 28, 2018

giuseppe commented Aug 31, 2018

AkihiroSuda commented Sep 7, 2018

giuseppe commented Sep 18, 2018

AkihiroSuda commented Sep 26, 2018

AkihiroSuda commented Oct 3, 2018

debarshiray commented Oct 5, 2018

caniszczyk commented Oct 9, 2018

AkihiroSuda commented Oct 10, 2018

crosbymichael commented Oct 10, 2018

crosbymichael commented Oct 11, 2018 •

edited by caniszczyk

Loading

mrunalp commented Oct 15, 2018

mrunalp commented Oct 16, 2018 •

edited by caniszczyk

Loading

AkihiroSuda commented Oct 16, 2018

Disable rootless mode except RootlessCgMgr when executed as the root in userns (fix Docker-in-LXD regression) #1862

Disable rootless mode except RootlessCgMgr when executed as the root in userns (fix Docker-in-LXD regression) #1862

Conversation

AkihiroSuda commented Aug 10, 2018

rhatdan commented Aug 21, 2018

giuseppe Aug 22, 2018

Choose a reason for hiding this comment

AkihiroSuda Aug 22, 2018

Choose a reason for hiding this comment

giuseppe Aug 22, 2018

Choose a reason for hiding this comment

AkihiroSuda Aug 22, 2018

Choose a reason for hiding this comment

kolyshkin commented Aug 23, 2018

AkihiroSuda commented Aug 24, 2018

AkihiroSuda commented Aug 28, 2018

giuseppe commented Aug 28, 2018

thaJeztah commented Aug 28, 2018

giuseppe commented Aug 31, 2018

AkihiroSuda commented Sep 7, 2018

giuseppe commented Sep 18, 2018

AkihiroSuda commented Sep 26, 2018

AkihiroSuda commented Oct 3, 2018

debarshiray commented Oct 5, 2018

caniszczyk commented Oct 9, 2018

AkihiroSuda commented Oct 10, 2018

crosbymichael commented Oct 10, 2018

crosbymichael commented Oct 11, 2018 • edited by caniszczyk Loading

mrunalp commented Oct 15, 2018

mrunalp commented Oct 16, 2018 • edited by caniszczyk Loading

AkihiroSuda commented Oct 16, 2018

crosbymichael commented Oct 11, 2018 •

edited by caniszczyk

Loading

mrunalp commented Oct 16, 2018 •

edited by caniszczyk

Loading