Check the result cache before the DepGraph when ensuring queries #81855

cjgillot · 2021-02-07T15:25:03Z

Split out of #70951

Calling ensure on already forced queries is a common operation.
Looking at the results cache first is faster than checking the DepGraph for a green node.

rust-highfive · 2021-02-07T15:25:06Z

r? @lcnr

(rust-highfive has picked a reviewer for you, use r? to override)

cjgillot · 2021-02-07T15:25:35Z

@bors try @rust-timer queue

rust-timer · 2021-02-07T15:25:36Z

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

bors · 2021-02-07T15:25:45Z

⌛ Trying commit aaf78a81486855e9c5868c06d2294530e0dd27ad with merge bcdde8a8e45640ba22bf2d913ac678ac46c14291...

bors · 2021-02-07T16:13:59Z

☀️ Try build successful - checks-actions
Build commit: bcdde8a8e45640ba22bf2d913ac678ac46c14291 (bcdde8a8e45640ba22bf2d913ac678ac46c14291)

rust-timer · 2021-02-07T16:14:01Z

Queued bcdde8a8e45640ba22bf2d913ac678ac46c14291 with parent ae00b62, future comparison URL.

rust-timer · 2021-02-07T17:52:12Z

Finished benchmarking try commit (bcdde8a8e45640ba22bf2d913ac678ac46c14291): comparison url.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. Please note that if the perf results are neutral, you should likely undo the rollup=never given below by specifying rollup- to bors.

Importantly, though, if the results of this run are non-neutral do not roll this PR up -- it will mask other regressions or improvements in the roll up.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf

tgnottingham · 2021-02-07T18:23:35Z

Much respect for isolating the effect of this. :)

lcnr · 2021-02-09T18:59:20Z

LGTM

am not familiar enough to merge this myself though, reassigning for a final review

r? @michaelwoerister maybe 🤔

cjgillot · 2021-02-14T12:43:17Z

cc @oli-obk

oli-obk · 2021-02-14T19:03:31Z

@bors r+

bors · 2021-02-14T19:03:32Z

📌 Commit 3fc8ed6 has been approved by oli-obk

bors · 2021-02-15T06:45:54Z

⌛ Testing commit 3fc8ed6 with merge 41e9d49d0242b77073d4e98e72bd542bac0eaa25...

bors · 2021-02-15T11:00:55Z

💥 Test timed out

rust-log-analyzer · 2021-02-15T11:32:35Z

A job failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

oli-obk · 2021-02-15T11:51:14Z

@bors retry (all builders but one succeeded)

bors · 2021-02-15T12:12:04Z

⌛ Testing commit 3fc8ed6 with merge d1206f9...

bors · 2021-02-15T15:14:24Z

☀️ Test successful - checks-actions
Approved by: oli-obk
Pushing d1206f9 to master...

Mark-Simulacrum · 2021-02-16T16:25:33Z

Interestingly, on final merge benchmark results show a regression in instruction counts (and at least somewhat on wall times too...): https://perf.rust-lang.org/compare.html?start=9503ea19edbf01b9435e80e17d60ce1b88390116&end=d1206f950ffb76c76e1b74a19ae33c2b7d949454&stat=instructions:u is final merge vs. https://perf.rust-lang.org/compare.html?start=ae00b62ceb7eaf1f02f5289ab233bf7e0e8060d5&end=bcdde8a8e45640ba22bf2d913ac678ac46c14291 for the PR run.

Maybe something merged in the meantime that caused some delta there? The bootstrap time win is still pretty significant (thanks :), so this seems good to stay in, but seems like it didn't end up being the win we were expecting on many benchmarks at least.

cjgillot · 2021-02-17T12:49:56Z

Perf mismatch resolved by #82197
https://perf.rust-lang.org/compare.html?start=8fe989dd768f5dfdb0fc90933f3f74fa4579fefd&end=ee88f46bb5e27c4d9f30326e69ff2298dcbeecb1

Package changes: * bump bootstraps to 1.51.0. * adjust patches and cargo checksums as required * 1.51 failed to build natively on 32-bit armv7, there is hope that this is fixed with 1.52. (1.51 can be built with netbsd32 emulation on a aarch64 system). Upsteream changes: Version 1.52.0 (2021-05-06) ============================ Language -------- - [Added the `unsafe_op_in_unsafe_fn` lint, which checks whether the unsafe code in an `unsafe fn` is wrapped in a `unsafe` block.][79208] This lint is allowed by default, and may become a warning or hard error in a future edition. - [You can now cast mutable references to arrays to a pointer of the same type as the element.][81479] Compiler -------- - [Upgraded the default LLVM to LLVM 12.][81451] Added tier 3\* support for the following targets. - [`s390x-unknown-linux-musl`][82166] - [`riscv32gc-unknown-linux-musl` & `riscv64gc-unknown-linux-musl`][82202] - [`powerpc-unknown-openbsd`][82733] \* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [`OsString` now implements `Extend` and `FromIterator`.][82121] - [`cmp::Reverse` now has `#[repr(transparent)]` representation.][81879] - [`Arc<impl Error>` now implements `error::Error`.][80553] - [All integer division and remainder operations are now `const`.][80962] Stabilised APIs ------------- - [`Arguments::as_str`] - [`char::MAX`] - [`char::REPLACEMENT_CHARACTER`] - [`char::UNICODE_VERSION`] - [`char::decode_utf16`] - [`char::from_digit`] - [`char::from_u32_unchecked`] - [`char::from_u32`] - [`slice::partition_point`] - [`str::rsplit_once`] - [`str::split_once`] The following previously stable APIs are now `const`. - [`char::len_utf8`] - [`char::len_utf16`] - [`char::to_ascii_uppercase`] - [`char::to_ascii_lowercase`] - [`char::eq_ignore_ascii_case`] - [`u8::to_ascii_uppercase`] - [`u8::to_ascii_lowercase`] - [`u8::eq_ignore_ascii_case`] Rustdoc ------- - [Rustdoc lints are now treated as a tool lint, meaning that lints are now prefixed with `rustdoc::` (e.g. `#[warn(rustdoc::non_autolinks)]`).][80527] Using the old style is still allowed, and will become a warning in a future release. - [Rustdoc now supports argument files.][82261] - [Rustdoc now generates smart punctuation for documentation.][79423] - [You can now use "task lists" in Rustdoc Markdown.][81766] E.g. ```markdown - [x] Complete - [ ] Todo ``` Misc ---- - [You can now pass multiple filters to tests.][81356] E.g. `cargo test -- foo bar` will run all tests that match `foo` and `bar`. - [Rustup now distributes PDB symbols for the `std` library on Windows, allowing you to see `std` symbols when debugging.][82218] Internal Only ------------- These changes provide no direct user facing benefits, but represent significant improvements to the internals and overall performance of rustc and related tools. - [Check the result cache before the DepGraph when ensuring queries][81855] - [Try fast_reject::simplify_type in coherence before doing full check][81744] - [Only store a LocalDefId in some HIR nodes][81611] - [Store HIR attributes in a side table][79519] Compatibility Notes ------------------- - [Cargo build scripts are now forbidden from setting `RUSTC_BOOTSTRAP`.] [cargo/9181] - [Removed support for the `x86_64-rumprun-netbsd` target.][82594] - [Deprecated the `x86_64-sun-solaris` target in favor of `x86_64-pc-solaris`.] [82216] - [Rustdoc now only accepts `,`, ` `, and `\t` as delimiters for specifying languages in code blocks.][78429] - [Rustc now catches more cases of `pub_use_of_private_extern_crate`][80763] - [Changes in how proc macros handle whitespace may lead to panics when used with older `proc-macro-hack` versions. A `cargo update` should be sufficient to fix this in all cases.][84136] [84136]: rust-lang/rust#84136 [80763]: rust-lang/rust#80763 [82166]: rust-lang/rust#82166 [82121]: rust-lang/rust#82121 [81879]: rust-lang/rust#81879 [82261]: rust-lang/rust#82261 [82218]: rust-lang/rust#82218 [82216]: rust-lang/rust#82216 [82202]: rust-lang/rust#82202 [81855]: rust-lang/rust#81855 [81766]: rust-lang/rust#81766 [81744]: rust-lang/rust#81744 [81611]: rust-lang/rust#81611 [81479]: rust-lang/rust#81479 [81451]: rust-lang/rust#81451 [81356]: rust-lang/rust#81356 [80962]: rust-lang/rust#80962 [80553]: rust-lang/rust#80553 [80527]: rust-lang/rust#80527 [79519]: rust-lang/rust#79519 [79423]: rust-lang/rust#79423 [79208]: rust-lang/rust#79208 [78429]: rust-lang/rust#78429 [82733]: rust-lang/rust#82733 [82594]: rust-lang/rust#82594 [cargo/9181]: rust-lang/cargo#9181 [`char::MAX`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.MAX [`char::REPLACEMENT_CHARACTER`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.REPLACEMENT_CHARACTER [`char::UNICODE_VERSION`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.UNICODE_VERSION [`char::decode_utf16`]: https://doc.rust-lang.org/std/primitive.char.html#method.decode_utf16 [`char::from_u32`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_u32 [`char::from_u32_unchecked`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_u32_unchecked [`char::from_digit`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_digit [`Peekable::next_if`]: https://doc.rust-lang.org/stable/std/iter/struct.Peekable.html#method.next_if [`Peekable::next_if_eq`]: https://doc.rust-lang.org/stable/std/iter/struct.Peekable.html#method.next_if_eq [`Arguments::as_str`]: https://doc.rust-lang.org/stable/std/fmt/struct.Arguments.html#method.as_str [`str::split_once`]: https://doc.rust-lang.org/stable/std/primitive.str.html#method.split_once [`str::rsplit_once`]: https://doc.rust-lang.org/stable/std/primitive.str.html#method.rsplit_once [`slice::partition_point`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.partition_point [`char::len_utf8`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.len_utf8 [`char::len_utf16`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.len_utf16 [`char::to_ascii_uppercase`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.to_ascii_uppercase [`char::to_ascii_lowercase`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.to_ascii_lowercase [`char::eq_ignore_ascii_case`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.eq_ignore_ascii_case [`u8::to_ascii_uppercase`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ascii_uppercase [`u8::to_ascii_lowercase`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ascii_lowercase [`u8::eq_ignore_ascii_case`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.eq_ignore_ascii_case

Pkgsrc changes: * Bump bootstrap kit version to 1.51.0. * Adjust patches as needed. * Update checksum adjustments. * Fix syntax error in commands adjusting libserde_derive for Darwin Upstream changes: Version 1.52.1 (2021-05-10) ============================ This release disables incremental compilation, unless the user has explicitly opted in via the newly added RUSTC_FORCE_INCREMENTAL=1 environment variable. This is due to the widespread, and frequently occuring, breakage encountered by Rust users due to newly enabled incremental verification in 1.52.0. Notably, Rust users **should** upgrade to 1.52.0 or 1.52.1: the bugs that are detected by newly added incremental verification are still present in past stable versions, and are not yet fixed on any channel. These bugs can lead to miscompilation of Rust binaries. These problems only affect incremental builds, so release builds with Cargo should not be affected unless the user has explicitly opted into incremental. Debug and check builds are affected. See [84970] for more details. [84970]: rust-lang/rust#84970 Version 1.52.0 (2021-05-06) ============================ Language -------- - [Added the `unsafe_op_in_unsafe_fn` lint, which checks whether the unsafe code in an `unsafe fn` is wrapped in a `unsafe` block.][79208] This lint is allowed by default, and may become a warning or hard error in a future edition. - [You can now cast mutable references to arrays to a pointer of the same type as the element.][81479] Compiler -------- - [Upgraded the default LLVM to LLVM 12.][81451] Added tier 3\* support for the following targets. - [`s390x-unknown-linux-musl`][82166] - [`riscv32gc-unknown-linux-musl` & `riscv64gc-unknown-linux-musl`][82202] - [`powerpc-unknown-openbsd`][82733] \* Refer to Rust's [platform support page][platform-support-doc] for more information on Rust's tiered platform support. Libraries --------- - [`OsString` now implements `Extend` and `FromIterator`.][82121] - [`cmp::Reverse` now has `#[repr(transparent)]` representation.][81879] - [`Arc<impl Error>` now implements `error::Error`.][80553] - [All integer division and remainder operations are now `const`.][80962] Stabilised APIs ------------- - [`Arguments::as_str`] - [`char::MAX`] - [`char::REPLACEMENT_CHARACTER`] - [`char::UNICODE_VERSION`] - [`char::decode_utf16`] - [`char::from_digit`] - [`char::from_u32_unchecked`] - [`char::from_u32`] - [`slice::partition_point`] - [`str::rsplit_once`] - [`str::split_once`] The following previously stable APIs are now `const`. - [`char::len_utf8`] - [`char::len_utf16`] - [`char::to_ascii_uppercase`] - [`char::to_ascii_lowercase`] - [`char::eq_ignore_ascii_case`] - [`u8::to_ascii_uppercase`] - [`u8::to_ascii_lowercase`] - [`u8::eq_ignore_ascii_case`] Rustdoc ------- - [Rustdoc lints are now treated as a tool lint, meaning that lints are now prefixed with `rustdoc::` (e.g. `#[warn(rustdoc::non_autolinks)]`).][80527] Using the old style is still allowed, and will become a warning in a future release. - [Rustdoc now supports argument files.][82261] - [Rustdoc now generates smart punctuation for documentation.][79423] - [You can now use "task lists" in Rustdoc Markdown.][81766] E.g. ```markdown - [x] Complete - [ ] Todo ``` Misc ---- - [You can now pass multiple filters to tests.][81356] E.g. `cargo test -- foo bar` will run all tests that match `foo` and `bar`. - [Rustup now distributes PDB symbols for the `std` library on Windows, allowing you to see `std` symbols when debugging.][82218] Internal Only ------------- These changes provide no direct user facing benefits, but represent significant improvements to the internals and overall performance of rustc and related tools. - [Check the result cache before the DepGraph when ensuring queries][81855] - [Try fast_reject::simplify_type in coherence before doing full check][81744] - [Only store a LocalDefId in some HIR nodes][81611] - [Store HIR attributes in a side table][79519] Compatibility Notes ------------------- - [Cargo build scripts are now forbidden from setting `RUSTC_BOOTSTRAP`.][cargo/9181] - [Removed support for the `x86_64-rumprun-netbsd` target.][82594] - [Deprecated the `x86_64-sun-solaris` target in favor of `x86_64-pc-solaris`.][82216] - [Rustdoc now only accepts `,`, ` `, and `\t` as delimiters for specifying languages in code blocks.][78429] - [Rustc now catches more cases of `pub_use_of_private_extern_crate`][80763] - [Changes in how proc macros handle whitespace may lead to panics when used with older `proc-macro-hack` versions. A `cargo update` should be sufficient to fix this in all cases.][84136] [84136]: rust-lang/rust#84136 [80763]: rust-lang/rust#80763 [82166]: rust-lang/rust#82166 [82121]: rust-lang/rust#82121 [81879]: rust-lang/rust#81879 [82261]: rust-lang/rust#82261 [82218]: rust-lang/rust#82218 [82216]: rust-lang/rust#82216 [82202]: rust-lang/rust#82202 [81855]: rust-lang/rust#81855 [81766]: rust-lang/rust#81766 [81744]: rust-lang/rust#81744 [81611]: rust-lang/rust#81611 [81479]: rust-lang/rust#81479 [81451]: rust-lang/rust#81451 [81356]: rust-lang/rust#81356 [80962]: rust-lang/rust#80962 [80553]: rust-lang/rust#80553 [80527]: rust-lang/rust#80527 [79519]: rust-lang/rust#79519 [79423]: rust-lang/rust#79423 [79208]: rust-lang/rust#79208 [78429]: rust-lang/rust#78429 [82733]: rust-lang/rust#82733 [82594]: rust-lang/rust#82594 [cargo/9181]: rust-lang/cargo#9181 [`char::MAX`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.MAX [`char::REPLACEMENT_CHARACTER`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.REPLACEMENT_CHARACTER [`char::UNICODE_VERSION`]: https://doc.rust-lang.org/std/primitive.char.html#associatedconstant.UNICODE_VERSION [`char::decode_utf16`]: https://doc.rust-lang.org/std/primitive.char.html#method.decode_utf16 [`char::from_u32`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_u32 [`char::from_u32_unchecked`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_u32_unchecked [`char::from_digit`]: https://doc.rust-lang.org/std/primitive.char.html#method.from_digit [`Peekable::next_if`]: https://doc.rust-lang.org/stable/std/iter/struct.Peekable.html#method.next_if [`Peekable::next_if_eq`]: https://doc.rust-lang.org/stable/std/iter/struct.Peekable.html#method.next_if_eq [`Arguments::as_str`]: https://doc.rust-lang.org/stable/std/fmt/struct.Arguments.html#method.as_str [`str::split_once`]: https://doc.rust-lang.org/stable/std/primitive.str.html#method.split_once [`str::rsplit_once`]: https://doc.rust-lang.org/stable/std/primitive.str.html#method.rsplit_once [`slice::partition_point`]: https://doc.rust-lang.org/stable/std/primitive.slice.html#method.partition_point [`char::len_utf8`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.len_utf8 [`char::len_utf16`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.len_utf16 [`char::to_ascii_uppercase`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.to_ascii_uppercase [`char::to_ascii_lowercase`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.to_ascii_lowercase [`char::eq_ignore_ascii_case`]: https://doc.rust-lang.org/stable/std/primitive.char.html#method.eq_ignore_ascii_case [`u8::to_ascii_uppercase`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ascii_uppercase [`u8::to_ascii_lowercase`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.to_ascii_lowercase [`u8::eq_ignore_ascii_case`]: https://doc.rust-lang.org/stable/std/primitive.u8.html#method.eq_ignore_ascii_case

rust-highfive assigned lcnr Feb 7, 2021

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Feb 7, 2021

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Feb 7, 2021

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Feb 7, 2021

jyn514 added A-query-system Area: The rustc query system (https://rustc-dev-guide.rust-lang.org/query.html) T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Feb 8, 2021

cjgillot mentioned this pull request Feb 8, 2021

Move the query engine out of rustc_middle #70951

Merged

rust-highfive assigned michaelwoerister and unassigned lcnr Feb 9, 2021

cjgillot added 7 commits February 13, 2021 21:14

Merge {get,ensure}_query.

8684e9e

Introduce query_stored module.

4b42a6d

Introduce query_storage.

f8ab649

Return a Result for query cache.

9f46259

Separate the query cache from the query state.

15b0bc6

Drop the cache lock earlier.

280a286

Check query cache before calling into the query engine.

3fc8ed6

cjgillot force-pushed the ensure-cache branch from aaf78a8 to 3fc8ed6 Compare February 14, 2021 12:15

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 14, 2021

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Feb 15, 2021

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Feb 15, 2021

bors added the merged-by-bors This PR was explicitly merged by bors. label Feb 15, 2021

bors merged commit d1206f9 into rust-lang:master Feb 15, 2021

rustbot added this to the 1.52.0 milestone Feb 15, 2021

bors mentioned this pull request Feb 15, 2021

Make the Query enum a simple struct. #80891

Merged

cjgillot deleted the ensure-cache branch February 15, 2021 19:13

This was referenced Feb 18, 2021

Refactorings in preparation for #70951 #80452

Closed

Add triage-2021-02-24 rust-lang/rustc-perf#844

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Check the result cache before the DepGraph when ensuring queries #81855

Check the result cache before the DepGraph when ensuring queries #81855

cjgillot commented Feb 7, 2021

rust-highfive commented Feb 7, 2021

cjgillot commented Feb 7, 2021

rust-timer commented Feb 7, 2021

bors commented Feb 7, 2021

bors commented Feb 7, 2021

rust-timer commented Feb 7, 2021

rust-timer commented Feb 7, 2021

tgnottingham commented Feb 7, 2021

lcnr commented Feb 9, 2021

cjgillot commented Feb 14, 2021

oli-obk commented Feb 14, 2021

bors commented Feb 14, 2021

bors commented Feb 15, 2021

bors commented Feb 15, 2021

rust-log-analyzer commented Feb 15, 2021

oli-obk commented Feb 15, 2021

bors commented Feb 15, 2021

bors commented Feb 15, 2021

Mark-Simulacrum commented Feb 16, 2021

cjgillot commented Feb 17, 2021

Check the result cache before the DepGraph when ensuring queries #81855

Check the result cache before the DepGraph when ensuring queries #81855

Conversation

cjgillot commented Feb 7, 2021

rust-highfive commented Feb 7, 2021

cjgillot commented Feb 7, 2021

rust-timer commented Feb 7, 2021

bors commented Feb 7, 2021

bors commented Feb 7, 2021

rust-timer commented Feb 7, 2021

rust-timer commented Feb 7, 2021

tgnottingham commented Feb 7, 2021

lcnr commented Feb 9, 2021

cjgillot commented Feb 14, 2021

oli-obk commented Feb 14, 2021

bors commented Feb 14, 2021

bors commented Feb 15, 2021

bors commented Feb 15, 2021

rust-log-analyzer commented Feb 15, 2021

oli-obk commented Feb 15, 2021

bors commented Feb 15, 2021

bors commented Feb 15, 2021

Mark-Simulacrum commented Feb 16, 2021

cjgillot commented Feb 17, 2021