Rootless managing of additional host groups

[zeehio]

Feature request description

This feature request describes how podman could allow rootless management of
additional host groups using better semantics.

One of the current issues with --group-add keep-groups is that while groups
are added, the groups are not mapped, so when listing the groups or a
mounted volume the overflowgid nobody appears. This is confusing.

Besides, some container images need the users to be mapped to operate, for
instance because they create processes based on user logins into the container and
those created processes do not have the unmapped groups set (because they
can't appear in the container's /etc/groups). As an example the rocker/rstudio image
expects users to login to rstudio server through a web interface, and the web logged in user
does not have those groups kept.

Suggest potential solution

This problem can be addressed if the system administrator is willing to give
some additional trust to users (giving the group as a subordinate GID) and
if podman improves its semantics a bit. This is what this feature request is
about.

Asking permission to the system administrator

The system administrator is able to subordinate arbitrary GID usage to a user
through /etc/subgid.

The most common usage of subordinate GIDs is for the sysadmin to give a large
non-overlapping chunk of unused GIDs to each user. Large GIDs are usually available
so we can easily give them away to users.

However, the sysadmin may consider giving as well an existing GID as subordinate to
several users. These users can act as if they owned that GID.

$ cat /etc/subgid

alice:100000:50000
alice:1001:1

Reads as:

The sysadmin grants a block of 50k GIDs, starting at 100000 to user alice.
The sysadmin grants GID 1001 to user alice.

This has security implications, as it essentially means that user alice can
"impersonate" GID 1001.

Telling rootless podman to map GID 1001

To run a container, we need to be able to act as if we had many users and groups
available. For instance, container images may include files owned by different
users and groups and as a regular user we should be able to create and manage
those files. The way to do this is for us (regular users) to create a mapping
between the UIDs and GIDs our container needs (or may need) and a subordinate
UID (GID) range we have.

By default, we may also want to map the root user and root group to our UID/GID
at the host.

This is what rootless podman essentially does:

container GID	host GID	Comment
0	$(id -g)	Map our GID to the root GID in the container
1	1st subid	Any GID our container needs to use goes to
2	2nd subid	our subordinate block of GIDs
n	nth subid

There is no need for podman to map all the subordinate GIDs we have, as long
as all the GIDs the container needs are mapped to some GID we have.

What we would like to do is to have control over host GID 1001. However, we
would like to map it not to an arbitrary GID in the 1-n range, but to a higher stable
GID the container does not actually use.

By choosing to map to a high GID (e.g. container GID 100000+1001), we avoid the risk of
overlapping with GIDs the container needs.

container GID	host GID	Comment
0	$(id -g)	Map our GID to the root GID in the container
1	1st subid	Any GID our container needs to use goes to
2	2nd subid	our 50k subordinate block of GIDs
n	nth subid
101001	1001	A host group we want to access and we have

Podman should add a group entry into the containers /etc/groups, with
the same group name as the host group 1001, with group id 101001 and with
the root user (or whatever user we have mapped onto) as a member, if we are
members of group 1001 in the host.

Suggested improvement

Semantically, it would be helpful defining something in containers.conf that
would tell podman to use host GID 1001 as a "higher container GID", that would not
collide with existing GIDs in the container image, and let podman add the
corresponding entry in /etc/groups.

For instance, the containers.conf could have a field like:

[containers]

# List the subordinate group IDs that should be mapped as additional groups
# in the container

local_group_mappings = [
 1001
]

local_group_offset = 100000

If 1001 is in the subgid for the user running the container, this setting would tell podman to map
1001 to local_group_offset+1001.

Have you considered any alternatives?

Currently podman treats all subordinate GIDs the same. This means that with
the /etc/subgid defined above, podman would map the host group 1001 to
the container GID 1, and the rest of subordinate GIDs would follow.

This is confusing, because GID1 is usually assigned in many containers and it
is defined as the bin or daemon group or some other system group.

Besides, as proven in the reproducible example below, the usage of --gidmap is not invariant to changes on subgids, making the scripts of the rootless user unnecessarily dependant on the subordinate GIDs available.

Additional context

No response

[Luap99]

There is --uidmap and --gidmap, so you can already set any mapping you would like.

[zeehio]

There is --uidmap and --gidmap, so you can already set any mapping you would like.

Kind of, but not really:

According to https://docs.podman.io/en/latest/markdown/podman-run.1.html#uidmap-container-uid-from-uid-amount, --uidmap and --gidmap when used on rootless podman, they only modify the intermediate UID->container UID mapping, but not the host UID -> intermediate UID mapping.

My point here is that, as a rootless podman user:

• The --gidmap I need to provide to mount a specific host GID (e.g GID 2000) into a specific container GID (e.g. 100000) depends on the intermediate UID mapping, which I do not control.
• The intermediate mapping is set by podman, based on the full range of subordinate GIDs available.
• The subordinate GIDs available to me depend on my sysadmin, and they may change (e.g. because the sysadmin subordinates an additional GID to me, of a secondary group of GID 1500). My intermediate GIDs will all shift as a result making all my scripts invalid.

Since I can't rely on the first intermediate mapping being stable (because my sysadmin may trust me with some extra subordinate ids), my usage of --gidmap is not stable and its usability becomes very poor.

Reproducible example:

# Create a sample directory tree
mkdir /tmp/test1
mkdir /tmp/test1/folder_gid_1001
mkdir /tmp/test1/folder_gid_1002
sudo chgrp 1001 /tmp/test1/folder_gid_1001
sudo chgrp 1002 /tmp/test1/folder_gid_1002
ls -lisan /tmp/test1

$cat /etc/subgid
sergio:100000:65536
sergio:1002:1

List files through podman using uidmap and gidmap:

$ podman run \
  --rm \
  -v /tmp/test1:/workdir \
  --uidmap "0:0:65535" \
  --gidmap "0:0:1" \
  --gidmap "100000:1:1" \
  --gidmap "1:2:65534" \
  alpine ls -lisan /workdir

• host gid 1001 is mapped to nobody (65534)
• host gid 1002 is mapped to container gid 100000 as intended (since 100000:1:1 refers to the "first intermediate GID")

My sysadmin adds gid 1001 as subordinate GID

$ cat /etc/subgid
sergio:100000:65536

sergio:1002:1
sergio:1001:1

My script is now invalid, since the gidmap has changed:

• host gid 1001 is mapped to container gid 100000
• host gid 1002 is mapped to container gid 1

[Luap99]

I am pretty sure the intermediate mapping is deterministic and depends on the order in /etc/sub?id, you can always check with podman unshare cat /proc/self/gid_map.

Since there is only ever one intermediate namespace (pause process) there is no way to check for an image at the point where we create the namespace to see that the mapping will not conflict with an existing groups in the image.

@giuseppe WDYT?

[zeehio]

Thanks for the feedback @Luap99. I added a reproducible example to hopefully clarify the issue.

I see your point. I can use podman unshare cat /proc/self/gid_map to check what's the current intermediate mapping.

However dealing with groups in this way in a script seems a bit error prone to me 😓

And besides I seem to need to provide the full list of --gidmap and --uidmap. I have learned how to do this, but in my humble opinion it is not very user friendly for a regular rootless user.

[giuseppe]

@giuseppe WDYT?

that is correct. The user namespace is used as the parent for any Podman operation, so it has no knowledge of the images that are you going to use. You need to parse the /etc/subgid in the podman unshare environment if you want to map to a specific value on the host

[zeehio]

My point is that the first mapping podman makes by default is not very reasonable when there are low subordinated ids.

It is reasonable for podman to assume that subordinate ids above 99999 are meant to be unused at the host and available to be used as low IDs at the container. But I do not think it is reasonable for podman to assume that with low subordinate ids, since they are usually used at the host for something.

Low IDs typically correspond to some host user or group, and therefore they should be mapped to predictable container IDs. I argue that a good default range for that mapping is to offset the host ID by a large number (e.g. 100 000). This yields a symmetric behaviour between mappings at the host and at the container, because 100 000 is also the default starting range for assigning unused subordinate ids:

Host IDs	Container IDs	Reason
root	nobody	We do not want rootless containers to impersonate root in the host
user	root	We want the user to have root-like behaviour in the container
≥ 100 000	<100 000	Containers need to use low IDs, and high host IDs are unused. Mapping order is not usually that relevant since assigned chunks don't change often
<100 000	≥ 100 000	We want to give access to the container to a host ID. Assign containerID=hostID+100000 so the mapping is systematic

I do not have go experience and I am not familiar with podman's source code, but I would be willing to do my best to attempt to provide a pull request if you think this feature makes sense. Some directions would be very welcome

[giuseppe]

it is too late to change the mappings we perform, some users might be depending on that, and it is the same mapping done by unshare -r --map-auto.

We usually need to have a root inside the user namespace because otherwise all the files from the image owned by root will show up as owned by nobody.

I suggest you take a look at the --userns=nomap that won't map your user inside the container

[zeehio]

it is too late to change the mappings we perform, some users might be depending on that,

I understand your desire to preserve backwards compatibility. Please consider that:

• Most of the users only have one subordinate block with high host IDs and therefore would not be affected by this
• The rest of the users have manually added their subordinate IDs (e.g. adding an additional block) and may in the future change their subordinate IDs again (they have done it once already at least once so they may need to do something similar in the future). When that happens, they will find that all of their --gidmaps may become messed up. And they may not even get an error or a warning... Just wrong IDs being mapped into their containers. Not a nice behaviour in my opinion.

and it is the same mapping done by unshare -r --map-auto.

I haven't verified this (my unshare version is not recent enough) but the docs available at https://man7.org/linux/man-pages/man1/unshare.1.html state that (bold is mine)

  --map-auto
      Map the **first block** of user IDs owned by the effective user
      from /etc/subuid to a block starting at user ID 0. In the
      same manner, also map the **first block** of group IDs owned by
      the effective group from /etc/subgid to a block starting at
      group ID 0. This option is intended to handle the common case
      where the first block of subordinate user and group IDs can
      map the whole user and group ID space. This option is
      equivalent to specifying --map-users=auto and
      --map-groups=auto.

If that is true, then unshare -r --map-auto will map the current user to root and the first block of GIDs to low GIDs, but it will not map other subordinate id blocks, such as my additionally delegated group.

So podman's behaviour of mapping all subordinate blocks and not just the first one is actually different from the one of unshare. I'm starting to think that the current behaviour is actually buggy and not a feature. And if it's a bug then it may be worth fixing it, even if it changes behaviour...

Maybe I would be more convincing if I went to util-linux and convinced the unshare developers of providing an option to map "subordinate blocks different than the first one" to "predictable high IDs" as I am suggesting here? If raising this issue there helps I am willing to go for it.

We usually need to have a root inside the user namespace because otherwise all the files from the image owned by root will show up as owned by nobody.

Yes, that's perfectly fine. No plans to change that. I don't understand why anyone would want to change that. Maybe I did not explain something well.

I suggest you take a look at the --userns=nomap that won't map your user inside the container

I want my user mapped inside the container as root. I want additional groups in the host mapped inside the container to predictable IDs, and that's something podman does not seem to be able to do by default since the mapping and the gidmap I have to provide depend on which groups I have subordinated to my user.

[giuseppe]

If that is true, then unshare -r --map-auto will map the current user to root and the first block of GIDs to low GIDs, but it will not map other subordinate id blocks, such as my additionally delegated group.

So podman's behaviour of mapping all subordinate blocks and not just the first one is actually different from the one of unshare. I'm starting to think that the current behaviour is actually buggy and not a feature. And if it's a bug then it may be worth fixing it, even if it changes behaviour...

I've used unshare only as an example. Podman does also another thing, it currently sorts the entries, so it doesn't follow the same order as specified in the /etc/subuid and /etc/subgid files.

I don't think the current behavior is buggy, it is just accommodating the most common case. Usually, additional IDs are added when the user realized there are not enough IDs available, so the most common case is to use them all contiguously so that we can pull and deal with images having higher IDs. e.g. I expect that the following add all the listed IDs, not that the second line means mapping 165536 to 165536.

foo:100000:65536
foo:165536:65536

so I am against treating the additional entries with a different meaning.

We could think of adding a function like --uidmap 0:@1000:1000 where the 1000 in @1000 is looked up in the parent user namespace instead of using the current one, but I don't think we should aim for predictable mappings just to make simpler this very specific use case.

[zeehio]

We could think of adding a function like --uidmap 0:@1000:1000 where the 1000 in @1000 is looked up in the parent user namespace instead of using the current one, but I don't think we should aim for predictable mappings just to make simpler this very specific use case.

This would make things simpler and the syntax is straightforward to understand. I like it a lot.

I have also noticed that when I provide a --gidmap I have to provide the full mapping (see the three --gidmap and the --uidmap as well) instead of "extending" the default mapping. Do you know if it would be possible to use something like --append-gidmap to avoid specifying the other mappings?

So instead of:

podman run \
  --rm \
  -v /tmp/test1:/workdir \
  --uidmap "0:0:65535" \
  --gidmap "0:0:1" \
  --gidmap "100000:1:1" \
  --gidmap "1:2:65534" \
  alpine ls -lisan /workdir

I can use:

podman run \
  --rm \
  -v /tmp/test1:/workdir \
  --append-gidmap "100000:@1001:1" \
  alpine ls -lisan /workdir

This would make group mapping in rootless containers much easier to use and understand for many people in my environment.

I don't know if I will be capable of implementing this in a pull request, but if we agree on the design then we can see how to implement it.

Thanks for all the time you are spending on this.

[giuseppe]

there are no default mappings added when you specify --uidmap (or --gidmap). What would be your expectation when you do something like --append-gidmap "100000:@1001:1"? What should the final mapping be?

[zeehio]

I would expect this command:

podman run \
  --rm \
  --append-gidmap "100000:1:1" \
  alpine

To behave like this:

podman run \
  --rm \
  --uidmap "0:0:65535" \
  --gidmap "0:0:1" \
  --gidmap "100000:1:1" \
  --gidmap "1:2:65534" \
  alpine

When there is no --uidmap and no --gidmap specified podman has a default mapping. I like that default mapping but I want to make just one change to it (e.g. mapping intermediate GID 1 to container GID 100000).

Currently, if I use --gidmap 100000:1:1 I must also specify (1) the identity for --uidmap (0:0:65535) (2) a mapping for gid 0 to my gid --gidmap 0:0:1 and (3) all the other container GIDs to "the rest of my subordinate GIDs" (excluding intermediate GID 1).

[giuseppe]

this could probably also be expressed using some special annotation for the --gidmap command, e.g.

 --gidmap ">100000:1:1" 

to insert that mapping and break the existing mappings (if any)

[zeehio]

Being able to use

--gidmap ">100000:@1001:1"

When running a container would make a whole community around me (and myself) really happy.

In summary, the features proposed after this discussion are two extensions to the --uidmap and --gidmap mapping syntax. The current syntax mapping is container_id:intermediate_id:map_length.

• If intermediate_id is of the form @<number>, then <number> is interpreted as the id of the parent namespace.
• If the mapping starts with a >, then the mapping is meant to be inserted and breaking existing mappings (if any).

I don't know if I will be able to implement these two things, but if nobody else has the bandwidth to do it I suppose that it's either me doing this or letting this linger. I don't have experience with go, but I have been able to follow instructions and build podman myself, so that's a start. I believe I may need to modify the containers/storage repository, since that's where I believe the parsing of id mappings happens. I'm feeling however this may be quite daunting as a first issue for me so your help (or anyone with more experience or time) taking over would make things easier, I guess. Advice and suggestions are very much welcome.

[zeehio]

I'll go step by step, so I will start by exploring the @<id> syntax.

Supporting @<id> in user/group mappings:

To implement support for @<id> I guess I can leverage GetAvailableIDMaps()to get the host->intermediate ID mappings from /proc/self/*id_map.

I should pass those "parent mappings" to the parsing function, and edit the parsing function so when it finds the @ symbol it finds the corresponding parent mapping and replaces the @1001 with whatever the parent mapping is pointing 1001 to.

I believe the parsing function being called is ParseIDMapping() that after some calls ends up calling ParseIDMapFilter.

If I try to grep for ParseIDMapping in containers/podman source code I see a bit more than 20 entries, scattered in several files. I am not familiar with them to tell precisely which of them need to be modified to support this.

Besides, I believe I'd need to change the containers/storage repository to accept the parent mappings as an optional extra argument and to change the parsing so it accepts the @ notation. Apparently the actual parsing is at opencontainers/runc,libcontainer/user/user.go so changes may be needed there as well.

I'm barely capable of doing anything in go, so I am far from knowing precisely if that's the way to go, or if I should just copy some functions from runc into storage and modify them here locally, since this notation seems to be podman specific, rather than spanning all of runc...

I am a bit lost, so getting help here would be very much appreciated.

[rhatdan]

Use the + sign. We have presedent for the - sign already.

--gidmap "+100000:@1001:1"

[zeehio]

I managed to provide an implementation at #18713. It lacks unit testing and documentation, because I have to figure out where do they go.

Comments and feedback are very welcome.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[zeehio]

Thanks bot. I have been busy with work but I haven't forgotten this, I just need a bit more time

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[rhatdan]

@zeehio Reminder.