[x.py] Option to not rebuild compiler

[Havvy]

One of the biggest paper cuts when working on Rust documentation is that x.py is really really eager to rebuild the compiler, even when the only thing that is changing one line in documentation or update a test.

As such, it would be extremely beneficial to those writing docs and tests (including doctests!) if we could have a flag where we as the end user of the flag promise we're not changing actual compiler code, so please use the previous still good compiler to rebuild the docs with our changes. Note that the option doesn't have to actually rebuild the documentation, as ideally we'd just combine it with x.py doc and x.py test src/libstd as needed.

cc @rust-lang/infra

[steveklabnik]

I've often wondered if incremental recompilation fixes this, as no code has changed.

[kennytm]

Have you tried

./x.py test --stage 1 --keep-stage 1 src/libstd
#                     ^^^^^^^^^^^^^^

[steveklabnik]

Without changes:

> python x.py test --stage 1 src/libstd
Updating submodules
    Finished dev [unoptimized] target(s) in 0.0 secs
Building stage0 std artifacts (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
    Finished release [optimized] target(s) in 0.0 secs
Copying stage0 std from stage0 (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc / x86_64-pc-windows-msvc)
Building stage0 test artifacts (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
    Finished release [optimized] target(s) in 0.0 secs
Copying stage0 test from stage0 (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc / x86_64-pc-windows-msvc)
Building stage0 compiler artifacts (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
    Finished release [optimized] target(s) in 0.2 secs
Copying stage0 rustc from stage0 (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc / x86_64-pc-windows-msvc)
Assembling stage1 compiler (x86_64-pc-windows-msvc)
Building stage1 std artifacts (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
    Finished release [optimized] target(s) in 0.0 secs
Copying stage1 std from stage1 (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc / x86_64-pc-windows-msvc)
Building stage1 test artifacts (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
    Finished release [optimized] target(s) in 0.0 secs
Copying stage1 test from stage1 (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc / x86_64-pc-windows-msvc)
Building rustdoc for stage1 (x86_64-pc-windows-msvc)
    Finished release [optimized] target(s) in 0.0 secs
Testing libstd stage1 (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
    Finished release [optimized] target(s) in 0.0 secs
     Running build\x86_64-pc-windows-msvc\stage1-std\x86_64-pc-windows-msvc\release\deps\std-b0d966d629cbd139.exe

running 778 tests
<snip>

Then, add one line to libstd's main doc block:

> vim .\src\libstd\lib.rs

Then with --keep-stage:

> python x.py test --stage 1 --keep-stage 1 src/libstd
Updating submodules
    Finished dev [unoptimized] target(s) in 0.0 secs
Copying stage0 std from stage0 (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc / x86_64-pc-windows-msvc)
Copying stage0 test from stage0 (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc / x86_64-pc-windows-msvc)
Copying stage0 rustc from stage0 (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc / x86_64-pc-windows-msvc)
Assembling stage1 compiler (x86_64-pc-windows-msvc)
Building stage1 std artifacts (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
   Compiling std v0.0.0 (file:///C:/Users/steve/src/rs/src/libstd)
    Finished release [optimized] target(s) in 20.34 secs
Copying stage1 std from stage1 (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc / x86_64-pc-windows-msvc)
Building stage1 test artifacts (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
   Compiling getopts v0.2.15
   Compiling term v0.0.0 (file:///C:/Users/steve/src/rs/src/libterm)
   Compiling test v0.0.0 (file:///C:/Users/steve/src/rs/src/libtest)
    Finished release [optimized] target(s) in 17.9 secs
Copying stage1 test from stage1 (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc / x86_64-pc-windows-msvc)
Building stage0 std artifacts (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
   Compiling std v0.0.0 (file:///C:/Users/steve/src/rs/src/libstd)
    Finished release [optimized] target(s) in 16.51 secs
Building stage0 test artifacts (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
   Compiling term v0.0.0 (file:///C:/Users/steve/src/rs/src/libterm)
   Compiling getopts v0.2.15
   Compiling test v0.0.0 (file:///C:/Users/steve/src/rs/src/libtest)
    Finished release [optimized] target(s) in 12.67 secs
Building stage0 compiler artifacts (x86_64-pc-windows-msvc -> x86_64-pc-windows-msvc)
   Compiling bitflags v1.0.0
   Compiling fmt_macros v0.0.0 (file:///C:/Users/steve/src/rs/src/libfmt_macros)
   Compiling rustc_driver v0.0.0 (file:///C:/Users/steve/src/rs/src/librustc_driver)
   Compiling cfg-if v0.1.2
   Compiling rustc_platform_intrinsics v0.0.0 (file:///C:/Users/steve/src/rs/src/librustc_platform_intrinsics)
   Compiling rustc-serialize v0.3.24
   Compiling cc v1.0.3

aka, it still rebuilds.

[ollie27]

I'm pretty sure --keep-stage is completely broken right now: #44737. Also I think you'd want to use --keep-stage 0.

[leoyvens]

Does test even care what stage you pass?

[camelid]

What's the status of this?

[jyn514]

I don't think this is actionable - either there's a bug in --keep-stage (which I don't think is the case), or it was user error passing --keep-stage 1 instead of --keep-stage 0. I highly recommend reading through https://rustc-dev-guide.rust-lang.org/building/bootstrapping.html to see what the difference is.

I've often wondered if incremental recompilation fixes this, as no code has changed.

See #47389 - the rebuilds are not actually spurious, and once the compiler is rebuilt, all its artifacts have to be rebuilt in order to be sound (see also #76720 which intentionally does not rebuild and thus regularly has ICEs that can be fixed by clearing the build directory). There's a bit more discussion at https://rustc-dev-guide.rust-lang.org/building/bootstrapping.html#stages-and-std (I thought there was somewhere that discussed ABIs more thoroughly, but I can't find it now).

So I think this should be closed.

[jyn514]

I'm going to close this since it's not actually a bug. I do want to mention that if you're only changing documentation, you can use download-rustc = true which doesn't have to do a full build even once.