Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stdenv.darwin: bootstrap darwin using updated tools #295558

Merged
1 commit merged into from Mar 28, 2024
Merged

stdenv.darwin: bootstrap darwin using updated tools #295558

1 commit merged into from Mar 28, 2024

Conversation

ghost
Copy link

@ghost ghost commented Mar 13, 2024

testing:

  • built stdenv for aarch64 and x64 with tools from hydra

how to request a seed update

  1. get approval for PR
  2. request hydra build uploaded to tarballs.nixos.org

once tools are uploaded will switch to ready to review so the PR doesn't get merged prematurely.

Description of changes

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

- update the hashes and tools needed to extract the bootstrap-tools archive
- unify the x64 and aarch64 unpack process
- unpin libxml for python-minimal
@ghost ghost requested a review from wegank March 22, 2024 13:34
Copy link
Contributor

@toonn toonn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (didn't rerun a stdenv build though).

@ghost
Copy link
Author

ghost commented Mar 23, 2024

cc @lovesegfault for uploading new bootstrap-tools to tarballs.nixos.org

aarch64-apple-darwin

$ sha256sum /nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools/on-server/*
63041266fcdb3a8db03829dc06c9e969ca3b660f755101c4950530798fdac454  /nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
77593ef6c27aa4a8272799f046427864b3359871de64a3f7fba6dd5e2b7f12d8  /nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools/on-server/unpack.nar.xz

Suggested commands to upload files to 'tarballs.nixos.org':

nix-store --realize /nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools
aws s3 cp --recursive --acl public-read /nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools/on-server/ s3://nixpkgs-tarballs/stdenv/aarch64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e
aws s3 cp --recursive s3://nixpkgs-tarballs/stdenv/aarch64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e ./
sha256sum bootstrap-tools.tar.xz unpack.nar.xz

x86_64-apple-darwin

$ sha256sum /nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools/on-server/*
5d44af17dea8350954b9b6daa8c2883bcb2f2f1633c1987d264b20032091cb3d  /nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
9d863677c42daffc1a7b2d9f775debd37b9ab4804d167a2d56f61c3f80d28dc4  /nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools/on-server/unpack.nar.xz

Suggested commands to upload files to 'tarballs.nixos.org':

nix-store --realize /nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools
aws s3 cp --recursive --acl public-read /nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools/on-server/ s3://nixpkgs-tarballs/stdenv/x86_64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e
aws s3 cp --recursive s3://nixpkgs-tarballs/stdenv/x86_64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e ./
sha256sum bootstrap-tools.tar.xz unpack.nar.xz
sha256sum /nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools/on-server/*

@ghost
Copy link
Author

ghost commented Mar 27, 2024

ping @lovesegfault #295558 (comment) for updating the tools into tarball.nixos.org

@ghost ghost mentioned this pull request Mar 27, 2024
13 tasks
@lovesegfault
Copy link
Member

@a-n-n-a-l-e-e I'm happy to help with uploading the tarballs, but I'd like to see an approval from @reckenrode and for this to be un-drafted before that :)

@ghost
Copy link
Author

ghost commented Mar 27, 2024

@a-n-n-a-l-e-e I'm happy to help with uploading the tarballs, but I'd like to see an approval from @reckenrode and for this to be un-drafted before that :)

this is in draft mode so it doesn't get committed by accident, as noted in the PR comment, to avoid incidents like #255451

@reckenrode
Copy link
Contributor

I’ll take a look at it this evening. It’d definitely be nice to update the bootstrap tools and simplify the stdenv bootstrap.

Copy link
Contributor

@reckenrode reckenrode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. I successfully built the stdenv on both aarch64-darwin and x86_64-darwin.

@ghost ghost marked this pull request as ready for review March 28, 2024 12:37
@ghost
Copy link
Author

ghost commented Mar 28, 2024

ok @lovesegfault, i think this meets your criteria for uploading the tarballs, i'd like to get this completed before the next staging cycle begins. thx.

@lovesegfault
Copy link
Member

Uploaded:

$ nix-store --realize /nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools
this path will be fetched (78.61 MiB download, 276.71 MiB unpacked):
  /nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools
copying path '/nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools' from 'https://cache.nixos.org'...
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools

$ sha256sum /nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools/on-server/*
63041266fcdb3a8db03829dc06c9e969ca3b660f755101c4950530798fdac454  /nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
77593ef6c27aa4a8272799f046427864b3359871de64a3f7fba6dd5e2b7f12d8  /nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools/on-server/unpack.nar.xz

$ aws s3 cp --recursive --acl public-read /nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools/on-server/ s3://nixpkgs-tarballs/stdenv/aarch64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e
upload: ../../../../nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools/on-server/unpack.nar.xz to s3://nixpkgs-tarballs/stdenv/aarch64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e/unpack.nar.xz
upload: ../../../../nix/store/sxc61dx6sng17lpp2rvv1q2apxj7bvq2-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz to s3://nixpkgs-tarballs/stdenv/aarch64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e/bootstrap-tools.tar.xz

$ aws s3 cp --recursive s3://nixpkgs-tarballs/stdenv/aarch64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e ./
download: s3://nixpkgs-tarballs/stdenv/aarch64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e/unpack.nar.xz to ./unpack.nar.xz
download: s3://nixpkgs-tarballs/stdenv/aarch64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e/bootstrap-tools.tar.xz to ./bootstrap-tools.tar.xz

$ sha256sum bootstrap-tools.tar.xz unpack.nar.xz
63041266fcdb3a8db03829dc06c9e969ca3b660f755101c4950530798fdac454  bootstrap-tools.tar.xz
77593ef6c27aa4a8272799f046427864b3359871de64a3f7fba6dd5e2b7f12d8  unpack.nar.xz

$ nix-store --realize /nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools
this path will be fetched (90.25 MiB download, 298.00 MiB unpacked):
  /nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools
copying path '/nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools' from 'https://cache.nixos.org'...
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
/nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools

$ sha256sum /nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools/on-server/*
5d44af17dea8350954b9b6daa8c2883bcb2f2f1633c1987d264b20032091cb3d  /nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
9d863677c42daffc1a7b2d9f775debd37b9ab4804d167a2d56f61c3f80d28dc4  /nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools/on-server/unpack.nar.xz

$ aws s3 cp --recursive --acl public-read /nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools/on-server/ s3://nixpkgs-tarballs/stdenv/x86_64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e
upload: ../../../../nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools/on-server/unpack.nar.xz to s3://nixpkgs-tarballs/stdenv/x86_64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e/unpack.nar.xz
upload: ../../../../nix/store/z2gdi0ha3w9scbgl3jv6431hd52d4b3j-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz to s3://nixpkgs-tarballs/stdenv/x86_64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e/bootstrap-tools.tar.xz

$ aws s3 cp --recursive s3://nixpkgs-tarballs/stdenv/x86_64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e ./
download: s3://nixpkgs-tarballs/stdenv/x86_64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e/unpack.nar.xz to ./unpack.nar.xz
download: s3://nixpkgs-tarballs/stdenv/x86_64-apple-darwin/d03ae62edd9402b6ee0471cb1543e1e19585025e/bootstrap-tools.tar.xz to ./bootstrap-tools.tar.xz

$ sha256sum bootstrap-tools.tar.xz unpack.nar.xz
5d44af17dea8350954b9b6daa8c2883bcb2f2f1633c1987d264b20032091cb3d  bootstrap-tools.tar.xz
9d863677c42daffc1a7b2d9f775debd37b9ab4804d167a2d56f61c3f80d28dc4  unpack.nar.xz

@ghost ghost merged commit e7079af into NixOS:staging Mar 28, 2024
19 checks passed
@ghost ghost deleted the darwin-bootstrap-update branch March 28, 2024 14:46
@ghost
Copy link
Author

ghost commented Mar 28, 2024

thanks!

@ghost
Copy link
Author

ghost commented Mar 29, 2024

reverting due to xz vulnerability included in the tools.
https://www.openwall.com/lists/oss-security/2024/03/29/4

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: darwin Running or building packages on Darwin 6.topic: stdenv Standard environment 10.rebuild-darwin: 501+ 10.rebuild-darwin: 5001+ 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants