[v2] rust: Write to .cargo/config.toml instead of .cargo/config #331167

mmlb · 2024-07-30T21:57:11Z

Description of changes

Changes the rust infra to write cargo config to .cargo/config.toml insted of .cargo/config since the latter is deprecated (since 1.38?) and is now warning on stderr.

Things done

Add a 👍 reaction to pull requests you find important.

Changes since v1:

Avoid changing cargoHash for FOD
Build/fixup more packages

mmlb · 2024-07-30T22:15:46Z

This time I've verified that no cargoHash changes to FOD happens (note, this was from when I based on master, waiting on building-most-of-the-world now that I rebased on staging):

[18:14:12]-[~/c/g/n/nixpkgs]-[manny@c3mv18nix]
git status
On branch rust-write-config.toml-not-config
nothing to commit, working tree clean
[18:14:18]-[~/c/g/n/nixpkgs]-[manny@c3mv18nix]
git rev-parse HEAD
c935d4b1ba9800c06629b7d3c39f156d52c32c9d
[18:14:20]-[~/c/g/n/nixpkgs]-[manny@c3mv18nix]
git grep cargoHash pkgs/by-name/co/commitmsgfmt/package.nix
pkgs/by-name/co/commitmsgfmt/package.nix:  cargoHash = "sha256-jTRB9ogFQGVC4C9xpGxsJYV3cnWydAJLMcjhzUPULTE=";
[18:14:24]-[~/c/g/n/nixpkgs]-[manny@c3mv18nix]
sed -i '/cargoHash/ s|".*"|""|' pkgs/by-name/co/commitmsgfmt/package.nix
[18:14:53]-[~/c/g/n/nixpkgs]-[manny@c3mv18nix]
nix-build -A commitmsgfmt 2>&1 | tail -n 5
calling 'postFixup' function hook '_multioutPropagateDev'
error: hash mismatch in fixed-output derivation '/nix/store/k0sfjvbjx1binzpm5b36n02a030rzdy9-commitmsgfmt-1.6.0-vendor.tar.gz.drv':
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
            got:    sha256-jTRB9ogFQGVC4C9xpGxsJYV3cnWydAJLMcjhzUPULTE=
error: 1 dependencies of derivation '/nix/store/x925hf14jii50vzlniamja7yb2726dzd-commitmsgfmt-1.6.0.drv' failed to build
[--:--:--]-[~/c/g/n/nixpkgs]-[manny@c3mv18nix]

mmlb · 2024-07-30T22:26:20Z

This should be the proper fix to close #320294

mmlb · 2024-07-31T00:23:52Z

And same steps to verify no change to cargoHash now off of staging:

[20:23:02]-[~/c/g/n/nixpkgs]-[manny@c3mv18nix]
git status
On branch rust-write-config.toml-not-config
nothing to commit, working tree clean
[20:23:04]-[~/c/g/n/nixpkgs]-[manny@c3mv18nix]
git rev-parse HEAD
4d384b1cf0afb65a5b1c3e7ecb4754a292662bcb
[20:23:08]-[~/c/g/n/nixpkgs]-[manny@c3mv18nix]
git grep cargoHash pkgs/by-name/co/commitmsgfmt/package.nix
pkgs/by-name/co/commitmsgfmt/package.nix:  cargoHash = "sha256-jTRB9ogFQGVC4C9xpGxsJYV3cnWydAJLMcjhzUPULTE=";
[20:23:10]-[~/c/g/n/nixpkgs]-[manny@c3mv18nix]
sed -i '/cargoHash/ s|".*"|""|' pkgs/by-name/co/commitmsgfmt/package.nix
[20:23:14]-[~/c/g/n/nixpkgs]-[manny@c3mv18nix]
nix-build -A commitmsgfmt 2>&1 | tail -n 5
calling 'postFixup' function hook '_multioutPropagateDev'
error: hash mismatch in fixed-output derivation '/nix/store/y58pyqfr0ad4pwwkwn4mlnr1bvn7xzsf-commitmsgfmt-1.6.0-vendor.tar.gz.drv':
         specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
            got:    sha256-jTRB9ogFQGVC4C9xpGxsJYV3cnWydAJLMcjhzUPULTE=
error: 1 dependencies of derivation '/nix/store/vxhj2yakd59ciqip8my6zz39pahj83vd-commitmsgfmt-1.6.0.drv' failed to build

winterqt · 2024-08-09T06:07:06Z

I'm not sure if this will just time out or not, but let's see.

@ofborg build fd

mmlb · 2024-08-16T21:52:54Z

@winterqt looks like it succeeded (FOD being FOD, its not much a signal though... right?). Any other worries? Also ping ping @zowoq @figsoda @Mic92 @mbalatsko

emilazy

LGTM in principle.

emilazy · 2024-09-03T16:27:00Z

pkgs/build-support/rust/hooks/cargo-setup-hook.sh

@@ -22,17 +22,17 @@ cargoSetupPostUnpackHook() {
        mkdir .cargo
    fi

-    config="$cargoDepsCopy/.cargo/config";
+    config="$cargoDepsCopy/.cargo/config.toml";


Non‐blocking nit: If we’re changing this hook anyway we could get rid of this unnecessary ;.

updated/removed

emilazy · 2024-09-03T16:29:10Z

This is an improvement over the status quo either way, but: this doesn’t adjust the fetchCargoTarball behaviour, which is good, but that means it’s still writing to the deprecated location. Doesn’t that mean we’ll still get warnings if we don’t mv or ln it in the build hooks? Do we not use those files at all somehow?

VuiMuich · 2024-09-04T04:33:28Z

Would we want to symlink ln -s .cargo/config.toml .cargo/config for compatibility to cargo 1.38 and earlier proactively, or should we wait for some occurrence?

Closes #334857

emilazy · 2024-09-04T04:42:04Z

From discussion on Matrix I think we decided that we don’t need to support 5+ year old compilers in Nixpkgs.

VuiMuich · 2024-09-04T14:50:03Z

From discussion on Matrix I think we decided that we don’t need to support 5+ year old compilers in Nixpkgs.

Fair enough. And if there would be a package that could absolutely not be built on a newer rust, they could create the symlink in their derivation anyways.

mmlb · 2024-09-05T15:36:09Z

This is an improvement over the status quo either way, but: this doesn’t adjust the fetchCargoTarball behaviour, which is good, but that means it’s still writing to the deprecated location. Doesn’t that mean we’ll still get warnings if we don’t mv or ln it in the build hooks? Do we not use those files at all somehow?

code comments say this only applies when depending on a git repo. I'm not sure if I built one of those explicitly, let me check (currently rebuilding everything as I ran a gc and did not cache any of this remotely ...)

Honestly I'm not sure why its not warning any more even without the mv or ln, as you suggest. I suspect it has to do with difference between $CARGO_HOME/config vs $CARGO_HOME/config.toml and where fetchCargoTarball writes the config to $name/.cargo/config. Looks like cargo is not warning about the latter case. I have a hunch that will change one day and then I'm not sure what we can do w/o causing the FOD hashes to be wrong.

Seeing the following new warnings pop up on stderr when cargo was bumped to 1.78: ``` warning: `/build/.cargo/config` is deprecated in favor of `config.toml` note: if you need to support cargo 1.38 or earlier, you can symlink `config` to `config.toml` ``` which happens to break commitmsgfmt builds in nix (NixOS#320294). closes NixOS#320294

mmlb · 2024-09-12T01:21:28Z

Rebased on latest staging to handle the conflict, looks like I'm rebuilding the world ...

mmlb · 2024-09-12T02:27:57Z

commitmsgfmt FOD hash checks out as no changes after rebase.

emilazy

I think let’s land this and worry about if the path in fetchCargoTarball affects anything another time. Thanks!

github-actions bot added 6.topic: rust 6.topic: ruby labels Jul 30, 2024

mmlb changed the base branch from master to staging July 30, 2024 22:16

mmlb force-pushed the rust-write-config.toml-not-config branch from c935d4b to f2da47c Compare July 30, 2024 22:16

github-actions bot removed 6.topic: python 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: GNOME GNOME desktop environment and its underlying platform labels Jul 30, 2024

ofborg bot requested a review from mbalatsko July 31, 2024 03:16

ofborg bot added 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-linux: 501+ 10.rebuild-linux: 5001+ labels Jul 31, 2024

alyssais mentioned this pull request Sep 3, 2024

pkgs/build-support/rust: fix warning "build/.cargo/config is deprecated" #339281

Closed

13 tasks

emilazy approved these changes Sep 3, 2024

View reviewed changes

mmlb force-pushed the rust-write-config.toml-not-config branch from 4d384b1 to 5b36343 Compare September 5, 2024 18:42

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 10, 2024

mmlb force-pushed the rust-write-config.toml-not-config branch from 5b36343 to 17b3df2 Compare September 12, 2024 02:26

mmlb requested a review from emilazy September 12, 2024 02:28

emilazy approved these changes Sep 12, 2024

View reviewed changes

emilazy merged commit bce3d51 into NixOS:staging Sep 12, 2024
8 of 9 checks passed

ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Sep 12, 2024

emilazy mentioned this pull request Sep 13, 2024

Nixpkgs Rust build machinery no longer supports Cargo 1.38 and older oxalica/rust-overlay#186

Closed

mmlb deleted the rust-write-config.toml-not-config branch September 13, 2024 15:22

TomaSajt mentioned this pull request Oct 18, 2024

rustPlatform.fetchCargoVendor: init #349360

Merged

13 tasks

alyssais mentioned this pull request Oct 22, 2024

buildRustPackage: use the same config file name for cargo-setup-hook and fetch-cargo-tarball #350446

Open

13 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[v2] rust: Write to .cargo/config.toml instead of .cargo/config #331167

[v2] rust: Write to .cargo/config.toml instead of .cargo/config #331167

mmlb commented Jul 30, 2024 •

edited

Loading

mmlb commented Jul 30, 2024 •

edited

Loading

mmlb commented Jul 30, 2024

mmlb commented Jul 31, 2024

winterqt commented Aug 9, 2024

mmlb commented Aug 16, 2024

emilazy left a comment

emilazy Sep 3, 2024

mmlb Sep 5, 2024

mmlb Sep 5, 2024

emilazy commented Sep 3, 2024

VuiMuich commented Sep 4, 2024

emilazy commented Sep 4, 2024

VuiMuich commented Sep 4, 2024

mmlb commented Sep 5, 2024 •

edited

Loading

mmlb commented Sep 12, 2024

mmlb commented Sep 12, 2024

emilazy left a comment

[v2] rust: Write to .cargo/config.toml instead of .cargo/config #331167

[v2] rust: Write to .cargo/config.toml instead of .cargo/config #331167

Conversation

mmlb commented Jul 30, 2024 • edited Loading

Description of changes

Things done

mmlb commented Jul 30, 2024 • edited Loading

mmlb commented Jul 30, 2024

mmlb commented Jul 31, 2024

winterqt commented Aug 9, 2024

mmlb commented Aug 16, 2024

emilazy left a comment

Choose a reason for hiding this comment

emilazy Sep 3, 2024

Choose a reason for hiding this comment

mmlb Sep 5, 2024

Choose a reason for hiding this comment

mmlb Sep 5, 2024

Choose a reason for hiding this comment

emilazy commented Sep 3, 2024

VuiMuich commented Sep 4, 2024

emilazy commented Sep 4, 2024

VuiMuich commented Sep 4, 2024

mmlb commented Sep 5, 2024 • edited Loading

mmlb commented Sep 12, 2024

mmlb commented Sep 12, 2024

emilazy left a comment

Choose a reason for hiding this comment

mmlb commented Jul 30, 2024 •

edited

Loading

mmlb commented Jul 30, 2024 •

edited

Loading

mmlb commented Sep 5, 2024 •

edited

Loading