Nix build processes on non-NixOS have nogroup in their supplementary groups

[Twey]

Nix on non-NixOS seems to add nogroup to the build process's supplementary groups, even though nixbld isn't actually a member of nogroup in the sandbox.

NixOS (note the lack of 64434(nogroup) at the end of the id line):

$ nix-build -E 'with import <nixpkgs> { }; runCommand "foo" { } "id"'
these derivations will be built:
  /nix/store/cw1c5slf25gjjzpmlwdv44gwdx8kqivc-foo.drv
building '/nix/store/cw1c5slf25gjjzpmlwdv44gwdx8kqivc-foo.drv'...
uid=1000(nixbld) gid=100(nixbld) groups=100(nixbld)

Arch (thanks @paddygord):

$ nix-build -E 'with import <nixpkgs> { }; runCommand "foo" { } "id"'
these derivations will be built:
  /nix/store/scia1pcb1bpwsb026dr9vjzcr4qvc1zr-foo.drv
building '/nix/store/scia1pcb1bpwsb026dr9vjzcr4qvc1zr-foo.drv'...
uid=1000(nixbld) gid=100(nixbld) groups=100(nixbld),65534(nogroup)

Ubuntu 18.04 (thanks @tiagolam):

$ nix-build -E 'with import <nixpkgs> { }; runCommand "foo" { } "id"'
these derivations will be built:
  /nix/store/pqjsbxhs0nmgnxfz3l5yn7jdycvvsp4l-foo.drv
building '/nix/store/pqjsbxhs0nmgnxfz3l5yn7jdycvvsp4l-foo.drv'...
uid=1000(nixbld) gid=100(nixbld) groups=100(nixbld),65534(nogroup)

We're all using Nix 2.3.1.

This breaks tests for Go (e.g. go_bootstrap in nixpkgs): the Go test suite includes a test TestChown that attempts to change the group for each supplementary group given by getgroups, but this fails for nogroup. The failure looks like so:

--- FAIL: TestChown (0.00s)
        os_unix_test.go:51: gid: 100
        os_unix_test.go:63: groups:  [65534 65534 65534 65534 65534 65534 65534 100]
        os_unix_test.go:66: chown /tmp/_Go_TestChown460997763 -1 65534: chown /tmp/_Go_TestChown460997763: invalid argument
FAIL
FAIL    os      0.507s

/etc/passwd in the sandbox looks like this (on both NixOS and non-NixOS):

root:x:0:0:Nix build user:/build:/noshell
nixbld:x:1000:100:Nix build user:/build:/noshell
nobody:x:65534:65534:Nobody:/:/noshell

and /etc/group looks like this:

root:x:0:
nixbld:!:100:
nogroup:x:65534:

[zimbatm]

Can you also cat /etc/group and cat /etc/passwd in the derivation?

[Twey]

I edited the description to add some more information about the test failure and sandbox environment. :)

[7c6f434c]

It kind of looks like before entering the sandbox there was an extra group (or maybe nixbld-outside is not the same as inside?) that is not mapped/doesn't correspond to anything (on the GID level) inside the sandbox user NS.

This is a debugging direction proposal, not a real answer. of course.

[Twey]

Sorry, it looks like when I edited the description most recently GitHub actually had me edit an old version, and some information was lost. I've put it back.

[7c6f434c]

Are you running Nix in a single-user mode? How many groups you user id has?

[Twey]

No @7c6f434c, all of the above are in multi-user mode (for the non-NixOS users bog-standard Nix installs, by running the installer script from the site).

I (NixOS) have six; the Arch user has four; the Ubuntu user has nine.

[Twey]

(nobody is in nogroup, of course)

The Nix daemon shouldn't know anything about our user groups, though, right?

[edolstra]

I can't reproduce this on Ubuntu 18.04 or 19.04.

Note however that this is expected behaviour when using a chroot store as a non-root user. To quote from build.cc:

            /* Drop additional groups here because we can't do it
               after we've created the new user namespace.  FIXME:
               this means that if we're not root in the parent
               namespace, we can't drop additional groups; they will
               be mapped to nogroup in the child namespace. There does
               not seem to be a workaround for this. (But who can tell
               from reading user_namespaces(7)?)
               See also https://lwn.net/Articles/621612/. */
            if (getuid() == 0 && setgroups(0, 0) == -1)
                throw SysError("setgroups failed");

[7c6f434c]

Hm, seven copies of nogroup does kind of matches one-main and six-additional groups… but if you are using Nix-daemon…

For further debugging I would consider starting some very slow build (sleep 1d or something), enumerating all the visible processes in the nix-build and nix-daemon process trees, and doing 'grep '^G' /proc/[PID]/status' for each. For bonus points, maybe do the same with all the processes visible inside the build environment…

[yshui]

This is because group mapping is not properly setup in a UID namespace.

[weeezes]

Thanks for the tips in this issue 👍. What I ended up doing was take my user out of all groups except its primary one just to work around this for now. Seems like there's an issue also open in the Go repository, which gave me the tip that these specific tests break because of the nogroup group. IMO seems like it's more of an issue with the tests in Go, and this just happens to trip on a bug in it.

[stale]

I marked this as stale due to inactivity. → More info

[yshui]

Still hitting this after 3 years...

nix should drop the supplementary groups with setgroups, or probably setup gid map to map them to something. (see below)

[yshui]

After some researching on user namespace, i think this should be closed as wontfix. In summary, for security reason, an unprivileged process is not allowed to drop groups, or setup gid mappings inside the namespace for anything other than it's primary group. (references 1 and 2)

I think the best course of action is to just warn the user when they are not building using the nix daemon.