Support per-package dummy root-owned files in sandboxed builds

[Shados]

Motivation

There are package-included tests that effectively have dependencies on the contents of root-owned, non-store files inside the sandboxed environment (e.g. /etc/passwd), including files where a universal dummy file may not make sense (or there may be no clear defaults). Mostly integration tests.

There may be builds that have these kinds of dependencies as well, although I don't know of any, and based on nixpkgs I doubt there would be many of them if there are any.

Example Use Case

A fairly minimal example is that libssh2's integration tests against opensshd fails in a sandboxed build on NixOS, as below.
libssh2-test.nix:

with import <nixpkgs> {};

pkgs.libssh2.overrideAttrs (oldAttrs: {
  doCheck = true;
  preConfigure = ''
    # gives `sshd -dd` output for proper debugging of integration test failure
    export DEBUG=true

    # configure the sshd integration test to run correctly
    export USER=$(id -un)
    export ac_cv_path_SSHD=${openssh}/bin/sshd
    ./buildconf
  '';
  nativeBuildInputs = [
    autoconf automake libtool which
  ];
  checkInputs = [
    openssh
  ];
})

Relevant nix-build libssh2-test.nix output, if nix.useSandbox = true;:

...
debug1: userauth-request for user nixbld service ssh-connection method none [preauth]
debug1: attempt 0 failures 0 [preauth]
debug2: parse_server_config: config reprocess config len 1
User nixbld not allowed because shell /noshell does not exist
debug2: monitor_read: 8 used once, disabling now
debug2: monitor_read: 4 used once, disabling now
debug2: input_userauth_request: try method none [preauth]
debug1: userauth-request for user nixbld service ssh-connection method publickey [preauth]
debug1: attempt 1 failures 0 [preauth]
debug2: input_userauth_request: try method publickey [preauth]
debug2: userauth_pubkey: disabled because of invalid user [preauth]
Received disconnect from 127.0.0.1 port 47812:11: Normal Shutdown [preauth]
Disconnected from invalid user nixbld 127.0.0.1 port 47812 [preauth]
debug1: do_cleanup [preauth]
debug1: monitor_read_log: child log fd closed
debug1: do_cleanup
debug1: Killing privsep child 6836
Fingerprint: 86 AD B2 21 33 60 65 3D 9A 29 86 DE 22 99 DA 18 CC BA D3 AC
Authentication methods: publickey,password,keyboard-interactive
        Authentication by public key failed!

============================================================================
Testsuite summary for libssh2 -
============================================================================
# TOTAL: 3
# PASS:  2
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0
============================================================================
See tests/test-suite.log
Please report to [email protected]
============================================================================
make[3]: *** [Makefile:723: test-suite.log] Error 1
make[3]: Leaving directory '/build/libssh2-1.8.0/tests'
make[2]: *** [Makefile:831: check-TESTS] Error 2
make[2]: Leaving directory '/build/libssh2-1.8.0/tests'
make[1]: *** [Makefile:916: check-am] Error 2
make[1]: Leaving directory '/build/libssh2-1.8.0/tests'
make: *** [Makefile:545: check-recursive] Error 1
builder for '/nix/store/h25kni38az4d7yc8wvmva0klvb5nwhjp-libssh2-1.8.0.drv' failed with exit code 2
error: build of '/nix/store/h25kni38az4d7yc8wvmva0klvb5nwhjp-libssh2-1.8.0.drv' failed

Note: User nixbld not allowed because shell /noshell does not exist

The underlying issue is that openssh will refuse to auth a user if their shell (which is ultimately resolved by the NSS mechanism) does not actually exist. This basic test works fine with nix.useSandbox = false; on NixOS because the real nixbld users have nologin as their shell, which does happen to exist.

Of course, more complex tests involving actually running commands over the connection would still fail even there, but that's just more reason for this functionality -- it could let you run such tests, albeit only in a sandboxed build.

If the builder could specify the contents of the dummy /etc/passwd file, this could be worked around pretty trivially. Without that, it would require a nasty kludge of one kind or another (e.g. patch openssh, run it with an LD_PRELOAD to intercept the NSS resolution, use a different ssh server binary with different behaviour).

Related Issues

• Give Nix deeper understanding of tests vs. builders #874 suggests separating out package tests from the builder script, in which case it might make sense to only support this functionality for the tests, assuming it really is never relevant during builds
• Most of the suggestions from Inconsistent treatment of /usr/bin/env in build sandbox vs. NixOS #1205 could possibly be implemented by this, on a per-package, as-needed basis
• Expose /etc/nsswitch.conf and /etc/protocols etc to sandboxed nix builds? #1238 could be solved by this, without needing to punch holes in the sandbox
• If some version of [RFC] nixpkgs/tests: create nixos-independent PoC tests nixpkgs#36842 does end up going ahead, it will almost certainly need this functionality for some tests

Questions

• Are there any security implications of allowing derivations to specify a different view of root-owned files within the sandbox environment by swapping some out for dummy files?
• If there are, is the set of problematic files well-known enough to manage with a blacklist, or must a whilelist of allowed dummy paths be defined instead?

[7c6f434c]

I would say, there is also a question whether we want ot have builds that are guaranteed to fail without sandbox builds.

If you assume NixOS-like configuration of everything, you can use unshare during the build, and if not — you are not completely guaranteed to have every kernel feature needed for sandboxing.

[Shados]

An alternative to having builds that fail without sandbox would be to have the dis/use of the sandbox be an input to the tests, although that might motivate going further than #874 suggests and actually separate tests out into their own derivation, so as to keep the same hash for the actual build output.

[stale]

I marked this as stale due to inactivity. → More info