Move some more bootstrap logic from python to rust #92260

jyn514 · 2021-12-24T19:10:30Z

Same rationale as #76544; it would be nice to make python entirely optional at some point.

This also removes $ROOT as an option for the build directory; I haven't been using it, and like Alex
said in #76544 (comment) it seems like a misfeature.

This allows running cargo run from src/bootstrap, although that still gives
lots of compile errors if you don't use the beta toolchain. It's not exactly the same as using x.py, since it won't have BOOTSTRAP_DOWNLOAD_RUSTC set, but it's pretty close. Doing this from the top-level directory requires rust-lang/cargo#7290 to be fixed, or using cargo run -p bootstrap.

The next steps for making python optional are to move download-ci-llvm and download-rustc support into rustbuild, likely be shelling out as the python scripts do today.

It would also be nice (although not required) to move submodule support there, but that would require taking bootstrap out of the workspace to avoid errors from crates that haven't been cloned yet.

r? @Mark-Simulacrum

jyn514 · 2021-12-24T20:06:12Z

I am confused. canonicalize() is failing because /checkout/build doesn't exist yet in CI, but creating it fails because /checkout is read-only. How did this possibly work before? Something needs to create the build directory or you can't build the compiler.

ehuss · 2021-12-25T18:06:00Z

The build-dir directory is relative to the current working directory (not the checkout directory). The docker scripts set the workdir to /checkout/obj and use relative paths like python3 ../x.py. The default build-dir of build then becomes /checkout/obj/build. The /checkout/obj volume is mounted read-write.

bors · 2022-01-02T03:57:55Z

☔ The latest upstream changes (presumably #92482) made this pull request unmergeable. Please resolve the merge conflicts.

jyn514 · 2022-02-06T23:49:57Z

It's not exactly the same as using x.py, since it won't have BOOTSTRAP_CONFIG or BOOTSTRAP_DOWNLOAD_RUSTC set, but it's pretty close.

actually, let me switch rustbuild to using RUST_BOOTSTRAP_CONFIG instead of its own custom env var while I'm at it, seems simple enough and I don't see how it could break anything

jyn514 · 2022-02-07T00:14:53Z

    # Read from `RUST_BOOTSTRAP_CONFIG`, then `--config`, then fallback to `config.toml` (if it
    # exists).
    toml_path = os.getenv('RUST_BOOTSTRAP_CONFIG') or args.config

uhhh it seems pretty strange to me that the env variable would take precedence over --config, but I don't want to change it here

bors · 2022-03-08T17:00:11Z

💥 Test timed out

Mark-Simulacrum · 2022-03-08T17:04:36Z

@bors retry test various timed out

bors · 2022-03-08T17:04:48Z

⌛ Testing commit 477cae3 with merge 64187b8...

rust-log-analyzer · 2022-03-08T17:13:28Z

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

Aaron1011 · 2022-03-08T19:06:50Z

src/bootstrap/util.rs

@@ -440,3 +440,112 @@ fn fail(s: &str) -> ! {
    println!("\n\n{}\n\n", s);
    std::process::exit(1);
 }
+
+/// Copied from `std::path::absolute` until it stabilizes.


Couldn't we just have bootstrap use the nightly feature until it's stabilized?

bjorn3 · 2022-03-08T19:10:57Z

#92260 (comment)

No, bootstrap shouldn't use unstable features, it causes problems when we are switching between self-bootstrap (i.e., building 1.x with 1.x) and regular bootstrap; most code avoids that problem through switching cfg(bootstrap) in rustbuild off and on depending on whether we're self bootstrapping but since rustbuild (in the long run) wants to be buildable with just a beta/stable compiler, we want to avoid unstable feature use since there's no one to do that switching. Copying the code from std should be fine (it's not a lot of code and isn't particularly likely to change).

bors · 2022-03-08T19:25:12Z

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing 64187b8 to master...

rust-timer · 2022-03-08T20:55:20Z

Finished benchmarking commit (64187b8): comparison url.

Summary: This benchmark run did not return any relevant results.

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

This was part of Mark's original PR in rust-lang@ecb424f, but I missed it when writing rust-lang#92260.

…Simulacrum Allow `cargo run` instead of `cargo run -p bootstrap` This was part of `@Mark-Simulacrum` 's original PR in rust-lang@ecb424f, but I missed it when writing rust-lang#92260. This also has the side effect of allowing `cargo build --bins` instead of `cargo build -p bootstrap --bins`. I'm not sure when you would want to run cargo build/check/test without going through bootstrap, but this still allows you to do so as long as you pass `-p` for all the crates you want to build.

When I implemented rust-only bootstrapping in rust-lang#92260, I neglected to test stage0 tools - it turns out they were broken because they couldn't find the sysroot of the initial bootstrap compiler. This fixes stage0 tools by using `rustc --print sysroot` instead of assuming rustc is already in a sysroot and hard-coding the relative directory.

…acrum Fix `cargo run tidy` When I implemented rust-only bootstrapping in rust-lang#92260, I neglected to test stage0 tools - it turns out they were broken because they couldn't find the sysroot of the initial bootstrap compiler. This fixes stage0 tools by using `rustc --print sysroot` instead of assuming rustc is already in a sysroot and hard-coding the relative directory. Fixes rust-lang#94797 (properly, without having to change rustup).

jyn514 added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Dec 24, 2021

rust-highfive assigned Mark-Simulacrum Dec 24, 2021

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Dec 24, 2021

jyn514 force-pushed the less-python-logic branch from d201ec7 to 7b4aef9 Compare December 24, 2021 19:11

This comment has been minimized.

Sign in to view

jyn514 force-pushed the less-python-logic branch from 7b4aef9 to ca3825c Compare December 24, 2021 19:41

This comment has been minimized.

Sign in to view

jyn514 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 25, 2021

JohnCSimon added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Jan 23, 2022

jyn514 force-pushed the less-python-logic branch 2 times, most recently from eb96446 to b3929be Compare February 6, 2022 23:48

jyn514 added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Feb 6, 2022

jyn514 force-pushed the less-python-logic branch from b3929be to d68021d Compare February 6, 2022 23:51

jyn514 added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 6, 2022

This comment has been minimized.

Sign in to view

jyn514 force-pushed the less-python-logic branch from d68021d to f2426ba Compare February 7, 2022 00:12

jyn514 force-pushed the less-python-logic branch from f2426ba to c91ca29 Compare February 7, 2022 00:15

jyn514 added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Feb 7, 2022

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Mar 8, 2022

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 8, 2022

Aaron1011 reviewed Mar 8, 2022

View reviewed changes

bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 8, 2022

bors merged commit 64187b8 into rust-lang:master Mar 8, 2022

rustbot added this to the 1.61.0 milestone Mar 8, 2022

jyn514 deleted the less-python-logic branch March 8, 2022 19:53

jyn514 added a commit to jyn514/rust that referenced this pull request Mar 10, 2022

Allow cargo run instead of cargo run -p bootstrap

687e53e

This was part of Mark's original PR in rust-lang@ecb424f, but I missed it when writing rust-lang#92260.

This was referenced Mar 10, 2022

Allow cargo run instead of cargo run -p bootstrap #94796

Merged

cargo run -p bootstrap -- test tidy fails because it can't find the sysroot #94797

Closed

jyn514 mentioned this pull request Mar 10, 2022

Fix cargo run tidy #94806

Merged

jyn514 mentioned this pull request Mar 10, 2022

Tracking Issue for removing python as the main bootstrap entry point #94829

Open

15 tasks

jyn514 mentioned this pull request Mar 22, 2022

cargo run doesn't rebuild bootstrap binaries #95141

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move some more bootstrap logic from python to rust #92260

Move some more bootstrap logic from python to rust #92260

jyn514 commented Dec 24, 2021 •

edited by Mark-Simulacrum

Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

jyn514 commented Dec 24, 2021

ehuss commented Dec 25, 2021

bors commented Jan 2, 2022

jyn514 commented Feb 6, 2022

This comment has been minimized.

jyn514 commented Feb 7, 2022

bors commented Mar 8, 2022

Mark-Simulacrum commented Mar 8, 2022

bors commented Mar 8, 2022

rust-log-analyzer commented Mar 8, 2022

Aaron1011 Mar 8, 2022

bjorn3 commented Mar 8, 2022

bors commented Mar 8, 2022

rust-timer commented Mar 8, 2022

Move some more bootstrap logic from python to rust #92260

Move some more bootstrap logic from python to rust #92260

Conversation

jyn514 commented Dec 24, 2021 • edited by Mark-Simulacrum Loading

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

jyn514 commented Dec 24, 2021

ehuss commented Dec 25, 2021

bors commented Jan 2, 2022

jyn514 commented Feb 6, 2022

This comment has been minimized.

jyn514 commented Feb 7, 2022

bors commented Mar 8, 2022

Mark-Simulacrum commented Mar 8, 2022

bors commented Mar 8, 2022

rust-log-analyzer commented Mar 8, 2022

Aaron1011 Mar 8, 2022

Choose a reason for hiding this comment

bjorn3 commented Mar 8, 2022

bors commented Mar 8, 2022

rust-timer commented Mar 8, 2022

jyn514 commented Dec 24, 2021 •

edited by Mark-Simulacrum

Loading