-
Notifications
You must be signed in to change notification settings - Fork 12.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
rustc_ast: Visit tokens stored in AST nodes in mutable visitor #78712
Conversation
That would just affect the computed spans, right? E.g. we might fail to join two spans that now have different |
@bors try @rust-timer queue |
Awaiting bors try build completion |
⌛ Trying commit fcfe3063cbacb070da03eeda8ed88c7bfd24f9de with merge bd450b36446fdb9d6276544cc56174298a890781... |
☀️ Try build successful - checks-actions |
Queued bd450b36446fdb9d6276544cc56174298a890781 with parent 0fb0025, future comparison URL. |
Finished benchmarking try commit (bd450b36446fdb9d6276544cc56174298a890781): comparison url. Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. Please note that if the perf results are neutral, you should likely undo the rollup=never given below by specifying Importantly, though, if the results of this run are non-neutral do not roll this PR up -- it will mask other regressions or improvements in the roll up. @bors rollup=never |
Yes, perhaps some spans will be different. |
Updated. I've also added the mut visitor micro-optimization commit from #78736 which seems relevant and non-controversial. |
@bors r+ |
📌 Commit 8def2fc has been approved by |
☀️ Test successful - checks-actions |
After #77271 token visiting is enabled only for one visitor in
rustc_expand\src\mbe\transcribe.rs
which applies hygiene marks to tokens produced by declarative macros (macro_rules
ormacro
), so this change doesn't affect anything else.When a macro has some interpolated token from an outer macro in its output
we can use the usual interpretation of interpolated tokens in token-based model - a None-delimited group - to write this macro in an equivalent form
When we are expanding the macro
inner
we need to apply hygiene marks to all tokens produced by it, including the tokens inside the group.Before this PR we did this by visiting the AST piece inside the interpolated token and applying marks to all spans in it.
I'm not sure this is 100% correct (ideally we should apply the marks to tokens and then re-parse the AST from tokens), but it's a very good approximation at least.
We didn't however apply the marks to actual tokens stored in the nonterminal, so if we used the nonterminal as a token rather than as an AST piece (e.g. passed it to a proc macro), then we got hygiene bugs.
This PR applies the marks to tokens in addition to the AST pieces thus fixing the issue.
r? @Aaron1011