glibc: allow load libnss tcb module

[Izorkin]

Description of changes

Allow load libnss tcb module.
It is currently not possible to load third party libnss modules from a custom directory.

Before PR:

strace -ff passwd 2>&1 | grep libnss
openat(AT_FDCWD, "/nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/libnss_tcb.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

After PR:

[pid  2592] openat(AT_FDCWD, "/nix/store/8dvsdby0w84kjc1g14190w15wyycqia1-glibc-2.35-163/lib/libnss_tcb.so.2", O_RDONLY|O_CLOEXEC) = 4
openat(AT_FDCWD, "/nix/store/8dvsdby0w84kjc1g14190w15wyycqia1-glibc-2.35-163/lib/libnss_tcb.so.2", O_RDONLY|O_CLOEXEC) = 4

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 23.05 Release Notes (or backporting 22.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[NickCao]

This looks too much a hack to me, I would definitely not accept this change, for now you may set $LD_LIBRARY_PATH system wide to load custom nss modules.

[Izorkin]

This looks too much a hack to me, I would definitely not accept this change, for now you may set $LD_LIBRARY_PATH system wide to load custom nss modules.

At the moment I have not found another way :(
#111194 - this variant has not yet been implemented.

[NickCao]

Or, is there any specific reason that you cannot resort to nscd?

[Izorkin]

Or, is there any specific reason that you cannot resort to nscd?

This variant doesn't work.
The nss libraries are only looked for in the glibc directory.

[NickCao]

I mean setting $LD_LIBRARY_PATH on the nscd systemd unit.

[Izorkin]

I mean setting $LD_LIBRARY_PATH on the nscd systemd unit.

Not working:

systemctl cat nscd | grep LD_LIBRARY_PATH
Environment="LD_LIBRARY_PATH=/nix/store/s6w6az734vw7n9fzb51a9adr0xfxlhw6-tcb-1.2/lib"

strace -ff passwd 2>&1 | grep libnss
[pid  5001] openat(AT_FDCWD, "/nix/store/8dvsdby0w84kjc1g14190w15wyycqia1-glibc-2.35-163/lib/libnss_tcb.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/nix/store/8dvsdby0w84kjc1g14190w15wyycqia1-glibc-2.35-163/lib/libnss_tcb.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

[flokli]

This is not how this works. You want to add a non-glibc-provided NSS module to the lookup path.

To accomplish this, set system.nssModules and system.nssDatabases accordingly.

The nixos tooling will make sure nscd/nsncd will be appropriately restarted to be able to find the specified NSS module.

Check the following example:

nixpkgs/nixos/modules/security/google_oslogin.nix

Line 53 in 4053a8d

# enable the nss module, so user lookups etc. work

Make sure nscd / nsncd is being used, and not disabled.

[Izorkin]

Doesn't work either

    system.nssModules = [ pkgs.tcb.out ];
    system.nssDatabases.shadow = lib.mkForce [ "tcb" ];

...
setting up /etc...
reloading user units for root...
su: Insufficient credentials to access authentication data
...

After that, it is not possible to log in.

[flokli]

Can you provide a more complete example, like a VM test?

su: Insufficient credentials to access authentication data

Where is this message coming from?

[Izorkin]

Can you provide a more complete example, like a VM test?

I did not write a test, I use the draft patch - #109457 (comment)

Where is this message coming from?

This warning occurs after a rebuild system:

building Nix...
building the system configuration...
stopping the following units: resolvconf.service
NOT restarting the following changed units: systemd-fsck@dev-disk-by\x2duuid-1A15\x2dEF77.service
activating the configuration...
reviving group 'nscd' with GID 996
reviving user 'nscd' with UID 997
setting up /etc...
reloading user units for user...
su: Insufficient credentials to access authentication data
(Ignored)
setting up tmpfiles
reloading the following units: dbus.service
starting the following units: resolvconf.service
the following new units were started: nscd.service, nss-lookup.target, nss-user-lookup.target

[flokli]

This PR as-is can definitely not land. We can't randomly dlopen .so files from /run/current-system, and even if we'd decide to, #111194 would be the more correct patch. I'll close this PR.

In any case, I don't think this NSS module is so different dispatching NSS lookups via ns(n)cd would not work, so this is the wrong approach.

---

I suspect a misconfiguration elsewhere.

su: Insufficient credentials to access authentication data

Where is this message coming from?

I did some digging, the message is coming from PAM.

---

I'll comment in your issue #109457 with how this could be approached.

[Izorkin]

@flokli @NickCao allow variant with use /etc/ld.so.cache file? Need to patch glibc to look for a file in /etc/ld.so.cache instead of /nix/store/...-glibc-2.35-224/etc/ld.so.cache

[NickCao]

That would break nix packages on foreign systems, as system libraries would be loaded instead of nix ones.

[Izorkin]

@NickCao change to /etc/ld-nix.so.conf and /etc/ld-nix.so.cache?

[NickCao]

Sounds a little better, but I would still prefer the cleaner nscd approach, I could imagine these ad-hoc fixes causing a lot of headache later.

[Izorkin]

I couldn't find another working variant :(

[NickCao]

I don't know what's going on with the specific combination of tcb and nscd, @flokli seems to be following that closely, hope he could provide you with more ideas.

[Izorkin]

I checked draft variant - it works!