stdenv: add uutils-coreutils stdenv

[SuperSandro2000]

To try this use the pkgsUutils overlay and replace python3 with any package:

nix-build -A pkgsUutils.python3

ToDo:

• merge coreutils: disable tests which fail in sandbox #116241
• wait for 1f831f0 to be in master

Issues preventing us from using it in a stdenv:

• autotools does not work because the died not handle escape codes tr does not handle escape sequences correct uutils/coreutils#1817
• install does not work when installing files from directors install: does not remove path from files uutils/coreutils#1823
• head does not work with null bytes head: missing --zero-terminated uutils/coreutils#1822
• touch cannot parse some date touch -m -h -d 2009-10-10 link is failing with "Unable to parse date" uutils/coreutils#2311
• cp to many files open cp Too many open files (os error 24) uutils/coreutils#2312
• cp main panic cp --reflink=auto panics when copying nested symlinks with restrict permissions uutils/coreutils#2314
• converting absolute symlinks to relative ones Converting absolute to relative symlink errors with No such file or directory uutils/coreutils#2335
• ln multiple --symbolic ln: The argument '--symbolic' was provided more than once, but cannot be used multiple times uutils/coreutils#2359
• missing symlink from test to [ make with MULTICALL is missing symlink from test to [ uutils/coreutils#2420
• tr posix character classes tr does not support posix character class  uutils/coreutils#2421
• cp dangling or nested symlinks cp: copying dangling or nested symlink results in error uutils/coreutils#2422
• cp permission denied cp: permission denied uutils/coreutils#2423
• cut not working in rustc cut: -d= works differently than gnu-cut uutils/coreutils#2424
• Fix installation of [ symlink https://github.com/Fix installation of [ symlink uutils/coreutils#2491
• Create completion directories incase they do not exist Create completion directories incase they do not exist uutils/coreutils#2490

Motivation for this change

• Try it out!
• https://discourse.nixos.org/t/rust-coreutils-replacement/11867

Things done

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/rust-coreutils-replacement/11867/4

[kvtb]

Is it ...practical?
Rust is funny, but uutils-coreutils is 80MB vs. 1.5MB of coreutils

[SuperSandro2000]

Is it ...practical?
Rust is funny, but uutils-coreutils is 80MB vs. 1.5MB of coreutils

The single binary is much smaller and when rust supports lining to shared find it will be reduced even further. uutils/coreutils#747

[siraben]

Changing the hash (temporary workaround) GNU Hello compiles and runs:

$ nix-build --show-trace -E 'with import ./. { }; hello.override { stdenv = stdenvUutilsCoreutils; }' && ./result/bin/hello
[nix-shell:~/nixpkgs]$ nix-build --show-trace -E 'with import ./. { }; hello.override { stdenv = stdenvUutilsCoreutils; }' && ./result/bin/hello
/nix/store/jxklig35p42bxhp9yzk0gkgz2d5i8vml-hello-2.10
Hello, world!

[siraben]

Checked as of 2021-06-16

• ed failed to build

shrinking RPATHs of ELF executables and libraries in /nix/store/19n3pm10rr79445054shvid8m49mab43-ed-1.17
shrinking /nix/store/19n3pm10rr79445054shvid8m49mab43-ed-1.17/bin/ed
gzipping man pages under /nix/store/19n3pm10rr79445054shvid8m49mab43-ed-1.17/share/man/
strip is /nix/store/b8y61srrw1m2n87dr5snfaxy0rqgk6qy-binutils-2.35.1/bin/strip
stripping (with command strip and flags -S) in /nix/store/19n3pm10rr79445054shvid8m49mab43-ed-1.17/bin
patching script interpreter paths in /nix/store/19n3pm10rr79445054shvid8m49mab43-ed-1.17
/nix/store/19n3pm10rr79445054shvid8m49mab43-ed-1.17/bin/red: interpreter directive changed from "#! /bin/sh" to "/nix/store/3xqkrab9zqpv91bw1xp8551sjk7qrc34-bash-4.4-p23/bin/sh"
thread 'main' panicked at 'Unable to parse date
Invalid day.', src/uu/touch/src/touch.rs:248:19
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
builder for '/nix/store/yh6a7878qfg3qqi24mk61jaax47y2jpa-ed-1.17.drv' failed with exit code 101
error: build of '/nix/store/yh6a7878qfg3qqi24mk61jaax47y2jpa-ed-1.17.drv' failed

• vim failed to build

these derivations will be built:
  /nix/store/7wna7whwryrrmsm6fpf02sqpsaw4jpv9-vim-8.2.2567.drv
building '/nix/store/7wna7whwryrrmsm6fpf02sqpsaw4jpv9-vim-8.2.2567.drv'...
unpacking sources
unpacking source archive /nix/store/dvh1x2m7gvp1iv5jjlvrf5ry5g3ay23k-source
cp: error: '/nix/store/dvh1x2m7gvp1iv5jjlvrf5ry5g3ay23k-source/src/normal.c' -> 'source/src/normal.c': Too many open files (os error 24)
do not know how to unpack source archive /nix/store/dvh1x2m7gvp1iv5jjlvrf5ry5g3ay23k-source
builder for '/nix/store/7wna7whwryrrmsm6fpf02sqpsaw4jpv9-vim-8.2.2567.drv' failed with exit code 1
error: build of '/nix/store/7wna7whwryrrmsm6fpf02sqpsaw4jpv9-vim-8.2.2567.drv' failed

Compiling other packages with the uutils-based stdenv:

• vim (symlink converting fails)
• coreutils-gnu (failed with error: The argument '--symbolic' was provided more than once, but cannot be used multiple times)
• tinycc (failed)
• chez
• pforth
• nawk
• gawk
• guile
• chicken (failed, unable to parse date)
• miranda (cp: '/nix/store/dl6ahdcycmw4yry2zpmslq7w7i2mdlvk-source/miralib/ex/box.m' -> 'source/miralib/ex/box.m': Permission denied (os error 13))
• rustc (error: Found argument '--libdir' which wasn't expected, or isn't valid in this context)
• bash
• dash
• tmux
• screen
• go (failed, unable to parse date)
• htop (cp: '/nix/store/21zp1xfq78bp4rgvdlvd5r2d0rmmp55n-source/README' -> 'source/README': Permission denied (os error 13))
• uutils-coreutils
• fish
• ghc (mkdir: ./utils/ghc-cabal/dist/build/tmp//.: No such file or directory (os error 2))
• openjdk (error: cannot find symbol)
• julia (failed, failing test in LinearAlgebra/triangular)
• tcl (failed, ln: error: No such file or directory (os error 2))
• sbcl (failed, unable to parse date)
• emacs (failed, unable to parse date)
• binutils-unwrapped
• the-powder-toy
• gnumake
• cmake
• ripgrep
• gnupg
• git (test failure)

[siraben]

Unfortunately it will take some digging to figure out exactly what went wrong in the builds that failed, so that upstream has a clear view on what incompatibilities they need to address.

[vcunat]

Nit: copy&paste "Musl".

[sylvestre]

Any chance this pr gets merged? Thanks

[SuperSandro2000]

Any chance this pr gets merged? Thanks

No, because it is not practical and uutils is not fully compatible with gnu coreutils.

[AndersonTorres]

Closing this as sadly abandoned.

[06kellyjac]

Could draft it until uutils is more matute

[SuperSandro2000]

Please re-open. I have no mechanic to not delete this branch from my fork.

[sylvestre]

I think it is now mature enough to replace gnu.

[nyabinary]

0.0.22 released

[siraben]

Can we rebase again?

[theoparis]

If this is still being worked on - what about using uutils findutils and uutils diffutils as well? I'm not sure how stable they are compared to uutils coreutils though...

[SuperSandro2000]

I think it is now mature enough to replace gnu.

No, it isn't. eg autoconf relies on many very exact behaviors and breaks without them.

If this is still being worked on - what about using uutils findutils and uutils diffutils as well?

Currently not and I haven't thought about this.

[sylvestre]

autoconf relies on many very exact behaviors and breaks without them.

Do you have example of this? I haven't seen such issues for the last year
I think many of these have been fixed

[SuperSandro2000]

See the PR title with many examples. I did not try this recently as it is quite home intensive.