Tweak crate loading to load less metadata #13384
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This separate crate cache is one factor which is causing libstd to be loaded twice during normal compilation. The crates loaded for syntax extensions have a separate cache than the crates loaded for linking, so all crates are loaded once per #[phase] they're tagged with. This removes the cache and instead uses the CStore structure itself as the cache for loaded crates. This should allow crates loaded during the syntax phase to be shared with the crates loaded during the link phase.
When linking, all crates in the local CStore are used to link the final product. With #[phase(syntax)], crates want to be omitted from this linkage phase, and this was achieved by dumping the entire CStore after loading crates. This causes crates like the standard library to get loaded twice. This loading process is a fairly expensive operation when dealing with decompressing metadata. This commit alters the loading process to never register syntax crates in CStore. Instead, only phase(link) crates ever make their way into the map of crates. The CrateLoader trait was altered to return everything in one method instead of having separate methods for finding information.
This is an optimization which is quite impactful for compiling small crates. Reading libstd's metadata takes about 50ms, and a hello world before this change took about 100ms (this change halves that time). Recent changes made it such that this optimization wasn't performed, but I think it's a better idea do to this for now. See rust-lang#10786 for tracking this issue.
alexcrichton
added a commit
to alexcrichton/rust
that referenced
this pull request
Apr 13, 2014
This bug was introduced in rust-lang#13384 by accident, and this commit continues the work of rust-lang#13384 by finishing support for loading a syntax extension crate without registering it with the local cstore. Closes rust-lang#13495
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Oct 18, 2022
Expand unmatched mbe fragments to reasonable default token trees Currently we expand unmatched fragments by not replacing them at all, leaving us with `$ident`. This trips up the parser or subsequent macro calls. Instead it makes more sense to replace these with some reasonable default depending on the fragment kind which should make more recursive macro calls work better for completions.
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Sep 24, 2024
…, r=llogiq Remove unused `collect_metadata` function Leftover from rust-lang#13221 changelog: none
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
On today's rustc, we're accidentally loading libstd's metadata 4 times when we only really need to load it once!
This takes compiling small crates back down below 50ms (whereas it has creeped back up to ~200ms)