Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cross compilation from x86_64 to x86_64 with custom gcc arch/tune fails #268767

Closed
jys1670 opened this issue Nov 20, 2023 · 3 comments
Closed

Cross compilation from x86_64 to x86_64 with custom gcc arch/tune fails #268767

jys1670 opened this issue Nov 20, 2023 · 3 comments
Labels
0.kind: build failure A package fails to build

Comments

@jys1670
Copy link

jys1670 commented Nov 20, 2023

Steps to reproduce

I have small VPS which could benefit greatly from AES-NI and other fancy instructions. It can't be used for building due to load limitations, so I've attempted to do compilation on local machine and push that on remote target:

nixos-rebuild --flake .#myServer \
  --target-host servername \
  --build-host localhost \
  boot

Wiki suggest tweaking configuration for such scenario in this way:

nixpkgs = {
    # building ON
    localSystem = {
        system = "x86_64-linux";
    };
    # building FOR
    crossSystem = {
        gcc.arch = "znver2";
        gcc.tune = "znver2";
        system = "x86_64-linux";
    };
};

However it fails while trying to compile bash:

checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu

Beginning configuration for bash-5.2-release for x86_64-unknown-linux-gnu

checking for x86_64-unknown-linux-gnu-gcc... x86_64-unknown-linux-gnu-gcc
checking whether the C compiler works... no
configure: error: in `/build/bash-5.2':
configure: error: C compiler cannot create executables

After that I've tried alternative approach that doesn't touch localSystem / crossSystem and modifies compiler wrapper instead:

crossOverlays = [
    (final: prev: {
        wrapCC = cc: prev.wrapCCWith {
            inherit cc;
            extraBuildCommands = ''
                echo "-O3 -march=znver2 -mtune=znver2" >> "$out/nix-support/cc-cflags"
            '';
        };
    })
];

Sadly that failed on python (bash worked though):

Found duplicated packages in closure for dependency 'setuptools':
        setuptools 68.2.2.post0 (/nix/store/rk7kwr8c0v8dwk64442q8c0lp7car7qn-python3.11...
        setuptools 68.2.2.post0 (/nix/store/y8yq448xka665bqyd5gvdhajqdbsnyys-python3.11...

Notify maintainers

I've seen @amjoseph-nixpkgs working on cross compilation a lot, I assume it's his area of expertise

Metadata

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.62, NixOS, 23.11 (Tapir), 23.11.20231117.c757e9b`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.1`
 - channels(root): `""`
 - nixpkgs: `/nix/store/rj6pqzx66pliiwdfk2wjnnlk40rm3mpd-source`
@jys1670 jys1670 added the 0.kind: build failure A package fails to build label Nov 20, 2023
@Artturin
Copy link
Member

Dupe of #243164

localSystem should work

https://github.com/search?q=%2Fgcc%5C.tune%2F+lang%3Anix+NOT+is%3Afork&type=code

There's a simpler way to add flags than what you're doing

withCFlags = compilerFlags: stdenv:

https://github.com/search?q=withCFlags+lang%3Anix+NOT+is%3Afork&type=code

@jys1670
Copy link
Author

jys1670 commented Nov 23, 2023

@Artturin stdenv adapters stuff seems to be broken. Doing it this way in latest nixos-unstable (global overlay):

self: super: {
    stdenv = super.withCFlags [ "-O3" "-march=znver2" "-mtune=znver2" "-flto" ] super.stdenv;
}

always results in error

error: The ‘env’ attribute set cannot contain any attributes passed to derivation. The following attributes are overlapping: NIX_CFLAGS_COMPILE`

I've tried other adapters mentioned here, it's all the same

As for localSystem, I think last time I tried it just built optimized packages locally, but normal packages were copied to target. Same goes for overlays (that's why crossOverlays). It could be that I simply forgot to reboot though, I am going to retest.

But anyways, man nixos-rebuild has such line:

--target-host host
...
Note that nixos-rebuild honors the nixpkgs.crossSystem setting of the given configuration but disregards the  true  architecture  of  the  target host. Hence the nixpkgs.crossSystem setting has to match the target platform or else activation will fail.

It implies that crossSystem should be used, not localSystem.

@Artturin
Copy link
Member

Artturin commented Nov 24, 2023

@Artturin stdenv adapters stuff seems to be broken. Doing it this way in latest nixos-unstable (global overlay):

self: super: {
    stdenv = super.withCFlags [ "-O3" "-march=znver2" "-mtune=znver2" "-flto" ] super.stdenv;
}

always results in error

error: The ‘env’ attribute set cannot contain any attributes passed to derivation. The following attributes are overlapping: NIX_CFLAGS_COMPILE`

This is because something is setting NIX_CFLAGS_COMPILE instead of env.NIX_CFLAGS_COMPILE, it would probably be possible to choose which one to use with some conditionals, but I'm not inclined to do it because people should switch to using the env attrset for environmental variables within builds to support structuredAttrs.

Found a issue #269539

Answer before realizing the error was about overlapping attrs

Works fine here

nix build --expr '(import ./. { overlays = [(self: super: {stdenv = super.withCFlags [ "-O3" "-march=znver2" "-mtune=znver2" "-flto" ] super.stdenv;  })]; }).zsh' --impure --keep-failed

because toString is run on it

env = (args.env or {}) // { NIX_CFLAGS_COMPILE = toString (args.env.NIX_CFLAGS_COMPILE or "") + " ${toString compilerFlags}"; };

however it fails with stuff like

expand-response-params> cc1: error: bad value ('znver2') for '-march=' switch
expand-response-params> cc1: note: valid arguments to '-march=' switch are: nocona core2 nehalem corei7 westmere sandybridge corei7-avx ivybridge core-avx-i haswell core-avx2 broadwell skylake skylake-avx512 cannonlake icelake-client icelake-server bonnell atom silvermont slm knl knm x86-64 eden-x2 nano nano-1000 nano-2000 nano-3000 nano-x2 eden-x4 nano-x4 k8 k8-sse3 opteron opteron-sse3 athlon64 athlon64-sse3 athlon-fx amdfam10 barcelona bdver1 bdver2 bdver3 bdver4 znver1 btver1 btver2 native; did you mean 'znver1'?

(likely related #253713)

To avoid the bootstrapping stuff do import ./. { config = { replaceStdenv = ({ pkgs }: pkgs.withCFlags [ "-O3" "-march=znver2" "-mtune=znver2" "-flto" ] pkgs.stdenv); }; };

This still fails with errors like

> g++ -g -O2  -o gperf version.o positions.o options.o keyword.o keyword-list.o input.o bool-array.o hash-table.o search.o output.o main.o ../lib/libgp.a -lm
> /nix/store/1fn92b0783crypjcxvdv6ycmvi27by0j-binutils-2.40/bin/ld: /build/ccPAfibH.ltrans0.ltrans.o: in function `Input::read_input()':
> /build/gperf-3.1/src/./input.cc:254: undefined reference to `get_delim(char**, unsigned long*, int, _IO_FILE*)'
> /nix/store/1fn92b0783crypjcxvdv6ycmvi27by0j-binutils-2.40/bin/ld: /build/ccPAfibH.ltrans0.ltrans.o: in function `Hash_Table::insert(KeywordExt*)':
> /build/gperf-3.1/src/./hash-table.cc:143: undefined reference to `hashpjw(unsigned char const*, unsigned int)'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: build failure A package fails to build
Projects
None yet
Development

No branches or pull requests

2 participants