build(nix): create si-fs-standalone Nix pkg for a Nix-free binary

[fnichol]

This change introduces a new Nix derivation helper
(standaloneBinaryDerivation) which uses a binDerivation-built
package and prepares its main binary to be runnable without the full Nix
package closure.

Note that at the moment, this very much depends on the
dynamic libraries at play and whether or not the executing system has
the correct versions. At the moment, we're targetting the si-fs binary
which only depends on libc and libgcc. Future work will attempt to
make these binaries more flexible and portable where possible.

[github-actions]

Dependency Review

✅ No vulnerabilities or OpenSSF Scorecard issues found.

OpenSSF Scorecard

Package	Version	Score	Details

Scanned Files

[fnichol]

I was testing this directly with nix build -L .#si-fs-standalone which builds the new .#si-fs package and then uses that to build the .#si-fs-standalone package, which you can check out in ./result/bin/si-fs and see the manipulated ELF headers with:

readelf -ld ./result/bin/si-fs

or possibly with some more helpful calls to patchelf:

# should report a linker under /lib64 and not /nix/store
patchelf --print-interpreter ./result/bin/si-fs

# should report the typical libc/libm/libgcc/ld-linux gang with nothing extra
patchelf --print-needed ./result/bin/si-fs

# should be empty/not set
patchelf --print-rpath ./result/bin/si-fs

[fnichol]

The merge-queue pipeline will rebuild all artifacts and should be reasonable proof that everything remains buildable as before. The changes here didn't change existing behavior although there were formatting updates.

[sprutton1]

To clarify my understanding, the idea here is to undo the autopatching done in the binDerivation to just use the system's libc bits, yeah? Out of curiostiy, why do we need a separate derivation for this?

[fnichol]

Essentially yeah. The default in Nix is to use patchelf to hardcode the linker path, set RPATH (i.e. where to look for libs), etc., and that's what we want to un-do. At some level I need a way to specify that I only want certain package targets to mangle themselves like this, not all packages (as we are relying on properly built Nix packages in our omnibus packages). At the moment this other derivation is using the output of a prior built package, but I may change this in the future to build is entirely (i.e. perform the buck2 build) but may play with which libs are present at build time (that is, can we provide an older Glibc at build time to help with backwards compat, etc).

And how did I arrive here? I partially resurrected the tricks and strategies used for the old launcher (

si/flake.nix

Lines 241 to 286 in 5534c19

	# Builds a standalone binary that is divorced from the Nix build and
	# runtime environment
	si-binary = buck2Derivation {
	pathPrefix = "bin";
	pkgName = "si";
	stdBuildPhase = ''
	# TODO(fnichol): we currently have a dynamically linked library of
	# libc++abi being looked for in a `/nix/store` path when this
	# binary is compiled on macOS, so for the meantime until this is
	# fixed, we're going to perform a Cargo build of the binary (it
	# does not suffer from this issue).
	cargo build --bin "$name" --release
	cp -pv "target/release/$name" "build/$name-$system"
	'';
	installPhase = ''
	mkdir -pv "$out/bin"
	cp -pv "build/$name-$system" "$out/bin/$name"
	'';
	dontPatchELF = true;
	dontAutoPatchELF = true;
	postFixup =
	""
	+ pkgs.lib.optionalString (pkgs.stdenv.isDarwin) ''
	nix_lib="$(otool -L "$out/bin/$name" \
	| grep libiconv.dylib \
	| awk '{print $1}'
	)"
	install_name_tool \
	-change \
	"$nix_lib" \
	/usr/lib/libiconv.2.dylib \
	"$out/bin/$name" \
	2>/dev/null
	''
	+ pkgs.lib.optionalString (pkgs.stdenv.isLinux) ''
	patchelf \
	--set-interpreter "${systemInterpreter}" \
	--remove-rpath \
	"$out/bin/$name"
	''
	+ ''
	mkdir -pv "$out/tarballs"
	tar cvf "$out/tarballs/$name-$system.tar" -C "$out/bin" "$name"
	gzip -9 "$out/tarballs/$name-$system.tar"
	'';
	};

)

[sprutton1]

Ah, okay. Totally makes sense.