glibc: Add an environment variable for specifying an NSS search path

[edolstra]

With this, you can set $NIX_GLIBC_NSS_PATH to a colon-separated list of directories in which glibc will look for NSS modules. In environments where nscd cannot be used, this allows dealing with non-standard NSS modules in a less sledgehammer-y way than setting $LD_LIBRARY_PATH (which affects everything, and in particular could cause ABI incompatibilities).

The default value is

/nix/run/glibc-nss-path/${out-hash}:/nix/run/glibc-nss-path

where out-hash is the hash part of glibc.out (e.g. /nix/run/glibc-nss-path/0cjq75a1cgwd7ccxsp9warzjax1kr7ag). Thus,
glibc will look for an NSS module named libnss_mymachines.so.2 at

/nix/run/glibc-nss-path/0cjq75a1cgwd7ccxsp9warzjax1kr7ag/libnss_mymachines.so.2

and

/nix/run/glibc-nss-path/libnss_mymachines.so.2

These are tried after the default search locations (i.e. $LD_LIBRARY_PATH and glibc.out/lib) so they don't override any current behaviour.

The use of /nix/run rather than /run is because the user may have write access to /nix but not to /run. On NixOS, it's intended than /nix/run is a symlink / bind-mount to /run.

Motivation for this change

This does for NSS modules what /run/opengl-driver does for our OpenGL support. In particular it's intended for non-NixOS, non-nscd environments that require non-standard NSS modules.

Maybe instead of NIX_GLIBC_NSS_PATH we could use a generic environment variable as suggested in https://gist.github.com/Infinisil/3366e7dfc9a01f6eeb25b5cb475cc585 (and then look for out-hash/nss and nss relative to the directories in that variable).

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[ghost]

This would really help me, what is needed to make this happen?

[ghost]

I was able to make the google-oslogin test work without nscd with a version of this.

The default value is

/nix/run/glibc-nss-path/${out-hash}:/nix/run/glibc-nss-path

where out-hash is the hash part of glibc.out (e.g. /nix/run/glibc-nss-path/0cjq75a1cgwd7ccxsp9warzjax1kr7ag).

This creates problems for extra-utils, since it will be interpreted as a reference by Nix but nuke-refs can't remove it.

The use of /nix/run rather than /run is because the user may have write access to /nix but not to /run. On NixOS, it's intended than /nix/run is a symlink / bind-mount to /run.

Hmm, I think it might be better to keep it in line with /run/opengl-driver for now, and maybe change both over to /nix/run/... later. Is this even a problem non-NixOS systems?

[r-burns]

Looks like the patch doesn't apply anymore after glibc moved some stuff around: bminor/glibc@f8847d8.

Moving the fallback module loading logic after this line seems to work:
https://github.com/bminor/glibc/blob/f8847d83e17774ed5e9c0f75ef693680b91bcae4/nss/nss_module.c#L135

[flokli]

#138178 re-rolls parts of this, by loading nss modules from /run/nss-modules. I didn't add the complicated "multiple paths, separated by colons, read from an environment variable".

We can probably get along with a static set of paths to look in.

In particular it's intended for non-NixOS, non-nscd environments that require non-standard NSS modules.

I'd propose adding a second path in a followup to #138178, that looks in /usr/lib, which is where other distros put their NSS modules.

I don't think we should use /nix/run or anything like that on non-NixOS, as NSS modules are provided by the distribution, and already installed in /usr/lib.

Using that approach, nix-built binaries running on non-NixOS, without nscd installed will just use whatever NSS modules are configured by the other distro itself, without the need to also manually install them through Nix.

[r-burns]

I'd propose adding a second path in a followup to #138178, that looks in /usr/lib, which is where other distros put their NSS modules.

Not necessarily, they might be in e.g. /lib/x86_64-linux-gnu.

I would say to stick to your guns and just do /run/nss-modules. Whatever solution non-NixOS users are using for /run/opengl-drivers/lib would apply here, whether that's manually creating the directory and symlinking the distro's files there, or mounting it with nix-user-chroot.

[SuperSandro2000]

I would say to stick to your guns and just do /run/nss-modules. Whatever solution non-NixOS users are using for /run/opengl-drivers/lib would apply here, whether that's manually creating the directory and symlinking the distro's files there, or mounting it with nix-user-chroot.

No should do be doing that. /run is a tmpfs and it is way easier to just use nixGL which collects env variables which point to the correct shared objects which in the end is similar to this PR.

Alternatively you can also just wrap the final binary and set the correct env variables. I am doing that to get mpv hardware acceleration working with home-manager on Debian.

[Mic92]

Is memcpy not better here given that we already calulate the string length?

[Mic92]

Alternative would be good old snprintf...

[edolstra]

I'm just following the surrounding code here, didn't want to deviate from the coding style.

[Mic92]

How about:

Suggested change

	+ char shlib_path[1024];
	+ char shlib_path[PATH_MAX];

Not that it PATH_MAX is not a lie..

[r-burns]

Okay, #138178 (comment) explains the per-glibc out-hash path very well. I noticed that the new self-reference in libc.so causes issues when building a NixOS VM. I think it could cause issues for bootstrap-tools as well. So I guess we need to either truncate/rot13 the hash to thwart Nix's dependency detection, or nuke the hash in places like this.

[tomberek]

Maybe instead of NIX_GLIBC_NSS_PATH we could use a generic environment variable as suggested in https://gist.github.com/Infinisil/3366e7dfc9a01f6eeb25b5cb475cc585 (and then look for out-hash/nss and nss relative to the directories in that variable).

As this need keeps coming up, it makes sense to have a single well-understood escape hatch that is common for each use-case. I'd rather avoid the proliferation of NIX_SOMETHING_PATH env vars for each thing. A single NIX_PIVOT_PATH=/some/path with the ${out-hash} mechanism that can be utilized as a search path for locales, opengl, nss, certificates seems good. A system admin on non-NixOS can add this to /etc/environment as needed. The tooling to create these dirs is a good barrier to limit its accidental use. (ref: https://gist.github.com/Infinisil/3366e7dfc9a01f6eeb25b5cb475cc585)

The use of /nix/run rather than /run is because the user may have write access to /nix but not to /run. On NixOS, it's intended than /nix/run is a symlink / bind-mount to /run.

Random thought about the default value: With the generic pivot approach one can imagine a default re-using the specialisation mechanism at /nix/var/nix/profiles/system/specialisation/HASH (pivots sound like a specialisation for a specific package). A bit less hacky is to use a new directory in that profile at /nix/var/nix/profiles/system/pivot/HASH. So NixOS can declare these if desired, or can do ln -s /run $out/pivot/HASH. Non-NixOS can create this special system profile and subdirs using any profile generation tooling with Nix, or it can be manually symlink'd in /nix directory by adventurous sysadmins. Example for NixOS:

  system.extraSystemBuilderCmds = with builtins; let
    glibcPath = elemAt (split "-" (baseNameOf pkgs.glibc.outPath)) 0;
  in ''
      mkdir $out/pivot
      ln -s ${pkgs.nss_ldap}/lib $out/pivot/${glibcPath}
    '';
};

This would have the default for NIX_PIVOT_PATH be /nix/var/nix/profiles/system/pivot.

---

Security considerations?

---

This overall issue may need an RFC?

[infinisil]

Just a small note, I think a better name for such an env var is NIX_IMPURE_RUNTIME_DIR or something along those lines.

[stale]

I marked this as stale due to inactivity. → More info

[flokli]

Just a small note, I think a better name for such an env var is NIX_IMPURE_RUNTIME_DIR or something along those lines.

Thie explicitly only affects NSS module loading. I don't think that env var name is very helpful in conveying the message.

[flokli]

As written in https://flokli.de/posts/2022-11-18-nsncd/, I still think we should get something around the lines of this patch into glibc, either here in nixpkgs, or upstream.

[Izorkin]

There is some progress?