Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ICE: parser_range.start >= start_pos && parser_range.end >= start_pos #129166

Closed
matthiaskrgr opened this issue Aug 16, 2024 · 2 comments
Closed

ICE: parser_range.start >= start_pos && parser_range.end >= start_pos #129166

matthiaskrgr opened this issue Aug 16, 2024 · 2 comments
Assignees
@nnethercote
Labels
A-parser Area: The parsing of Rust source code to an AST C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@matthiaskrgr
Copy link
Member

auto-reduced (treereduce-rust):

fn main() {
    let _ = #[cfg_eval]
    #[cfg( = , , )]
    0;
}

original:

#![feature(cfg_eval)]
#![feature(stmt_expr_attributes)]

fn main() {
    let _ = #[cfg_eval] #[cfg(name = "...", cfg(FALSE), cfg(FALSE))] 0;
    //~^ ERROR removing an expression is not supported in this position
}

Version information

rustc 1.82.0-nightly (a73bc4a13 2024-08-16)
binary: rustc
commit-hash: a73bc4a131d94eba633c4c572a28e0bf94a67530
commit-date: 2024-08-16
host: x86_64-unknown-linux-gnu
release: 1.82.0-nightly
LLVM version: 19.1.0

Command:
/home/matthias/.rustup/toolchains/master/bin/rustc

Program output 

error[E0658]: attributes on expressions are experimental
 --> /tmp/icemaker_global_tempdir.P8GwvQiqbOKy/rustc_testrunner_tmpdir_reporting.kwFlrMqR8nnW/mvce.rs:2:13
  |
2 |     let _ = #[cfg_eval]
  |             ^^^^^^^^^^^
  |
  = note: see issue #15701 <https://github.com/rust-lang/rust/issues/15701> for more information
  = help: add `#![feature(stmt_expr_attributes)]` to the crate attributes to enable
  = note: this compiler was built on 2024-08-16; consider upgrading it if it is out of date

error: expected unsuffixed literal, found `=`
 --> /tmp/icemaker_global_tempdir.P8GwvQiqbOKy/rustc_testrunner_tmpdir_reporting.kwFlrMqR8nnW/mvce.rs:3:12
  |
3 |     #[cfg( = , , )]
  |            ^

error[E0658]: use of unstable library feature 'cfg_eval': `cfg_eval` is a recently implemented feature
 --> /tmp/icemaker_global_tempdir.P8GwvQiqbOKy/rustc_testrunner_tmpdir_reporting.kwFlrMqR8nnW/mvce.rs:2:15
  |
2 |     let _ = #[cfg_eval]
  |               ^^^^^^^^
  |
  = note: see issue #82679 <https://github.com/rust-lang/rust/issues/82679> for more information
  = help: add `#![feature(cfg_eval)]` to the crate attributes to enable
  = note: this compiler was built on 2024-08-16; consider upgrading it if it is out of date

thread 'rustc' panicked at compiler/rustc_parse/src/parser/mod.rs:241:9:
assertion failed: parser_range.start >= start_pos && parser_range.end >= start_pos
stack backtrace:
   0:     0x766de83ac32d - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::hbb42d93700eab6e5
   1:     0x766de8a05297 - core::fmt::write::hede989e4e7d14b6d
   2:     0x766de998cd51 - std::io::Write::write_fmt::h7a42f27b08b157f5
   3:     0x766de83aea0b - std::panicking::default_hook::{{closure}}::hc8a48bf7f87d4f00
   4:     0x766de83ae67e - std::panicking::default_hook::h722ce6f9c5e65f69
   5:     0x766de7543fb9 - std[5f115fdff94c2120]::panicking::update_hook::<alloc[587b8234a862f141]::boxed::Box<rustc_driver_impl[818abaa6828b5c02]::install_ice_hook::{closure#0}>>::{closure#0}
   6:     0x766de83af327 - std::panicking::rust_panic_with_hook::h6b96b15aea6f391f
   7:     0x766de83aefb3 - std::panicking::begin_panic_handler::{{closure}}::h78f98703de9c52d7
   8:     0x766de83ac7e9 - std::sys::backtrace::__rust_end_short_backtrace::he797a316ee99c3ac
   9:     0x766de83aecb4 - rust_begin_unwind
  10:     0x766de55e1c93 - core::panicking::panic_fmt::hfd35dc75e555f48a
  11:     0x766de54a457c - core::panicking::panic::hec3c35fa0f838a8a
  12:     0x766de8f8e90e - <rustc_parse[5e692a80545cfc79]::parser::Parser>::parse_expr_force_collect
  13:     0x766de73631b6 - <<rustc_builtin_macros[cd55725283bcdb35]::cfg_eval::CfgEval>::configure_annotatable::{closure#5} as core[83194bb5562a103a]::ops::function::FnOnce<(&mut rustc_parse[5e692a80545cfc79]::parser::Parser,)>>::call_once
  14:     0x766de9927604 - rustc_builtin_macros[cd55725283bcdb35]::cfg_eval::cfg_eval
  15:     0x766de73570a5 - <rustc_builtin_macros[cd55725283bcdb35]::cfg_eval::expand as rustc_expand[ffb2cc9a288afecc]::base::MultiItemModifier>::expand
  16:     0x766de50c17e8 - <rustc_expand[ffb2cc9a288afecc]::expand::MacroExpander>::fully_expand_fragment
  17:     0x766de9b7f05f - <rustc_expand[ffb2cc9a288afecc]::expand::MacroExpander>::expand_crate
  18:     0x766de8f70e55 - rustc_interface[aec65d594c25cb1a]::passes::resolver_for_lowering_raw
  19:     0x766de8f703db - rustc_query_impl[4f1661ee14cf0c15]::plumbing::__rust_begin_short_backtrace::<rustc_query_impl[4f1661ee14cf0c15]::query_impl::resolver_for_lowering_raw::dynamic_query::{closure#2}::{closure#0}, rustc_middle[ccb09bcce8b60fe7]::query::erase::Erased<[u8; 16usize]>>
  20:     0x766de8f703c9 - <rustc_query_impl[4f1661ee14cf0c15]::query_impl::resolver_for_lowering_raw::dynamic_query::{closure#2} as core[83194bb5562a103a]::ops::function::FnOnce<(rustc_middle[ccb09bcce8b60fe7]::ty::context::TyCtxt, ())>>::call_once
  21:     0x766de9963dd2 - rustc_query_system[df9560578d31cea0]::query::plumbing::try_execute_query::<rustc_query_impl[4f1661ee14cf0c15]::DynamicConfig<rustc_query_system[df9560578d31cea0]::query::caches::SingleCache<rustc_middle[ccb09bcce8b60fe7]::query::erase::Erased<[u8; 16usize]>>, false, false, false>, rustc_query_impl[4f1661ee14cf0c15]::plumbing::QueryCtxt, false>
  22:     0x766de9963a6d - rustc_query_impl[4f1661ee14cf0c15]::query_impl::resolver_for_lowering_raw::get_query_non_incr::__rust_end_short_backtrace
  23:     0x766de97d7e5e - rustc_interface[aec65d594c25cb1a]::interface::run_compiler::<core[83194bb5562a103a]::result::Result<(), rustc_span[3220205ed180bad]::ErrorGuaranteed>, rustc_driver_impl[818abaa6828b5c02]::run_compiler::{closure#0}>::{closure#1}
  24:     0x766de987a804 - std[5f115fdff94c2120]::sys::backtrace::__rust_begin_short_backtrace::<rustc_interface[aec65d594c25cb1a]::util::run_in_thread_with_globals<rustc_interface[aec65d594c25cb1a]::util::run_in_thread_pool_with_globals<rustc_interface[aec65d594c25cb1a]::interface::run_compiler<core[83194bb5562a103a]::result::Result<(), rustc_span[3220205ed180bad]::ErrorGuaranteed>, rustc_driver_impl[818abaa6828b5c02]::run_compiler::{closure#0}>::{closure#1}, core[83194bb5562a103a]::result::Result<(), rustc_span[3220205ed180bad]::ErrorGuaranteed>>::{closure#0}, core[83194bb5562a103a]::result::Result<(), rustc_span[3220205ed180bad]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[83194bb5562a103a]::result::Result<(), rustc_span[3220205ed180bad]::ErrorGuaranteed>>
  25:     0x766de987ae70 - <<std[5f115fdff94c2120]::thread::Builder>::spawn_unchecked_<rustc_interface[aec65d594c25cb1a]::util::run_in_thread_with_globals<rustc_interface[aec65d594c25cb1a]::util::run_in_thread_pool_with_globals<rustc_interface[aec65d594c25cb1a]::interface::run_compiler<core[83194bb5562a103a]::result::Result<(), rustc_span[3220205ed180bad]::ErrorGuaranteed>, rustc_driver_impl[818abaa6828b5c02]::run_compiler::{closure#0}>::{closure#1}, core[83194bb5562a103a]::result::Result<(), rustc_span[3220205ed180bad]::ErrorGuaranteed>>::{closure#0}, core[83194bb5562a103a]::result::Result<(), rustc_span[3220205ed180bad]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[83194bb5562a103a]::result::Result<(), rustc_span[3220205ed180bad]::ErrorGuaranteed>>::{closure#1} as core[83194bb5562a103a]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
  26:     0x766de987b1eb - std::sys::pal::unix::thread::Thread::new::thread_start::h48acfbfeb014780f
  27:     0x766deafdc39d - <unknown>
  28:     0x766deb06149c - <unknown>
  29:                0x0 - <unknown>

error: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md

note: please make sure that you have updated to the latest nightly

note: rustc 1.82.0-nightly (a73bc4a13 2024-08-16) running on x86_64-unknown-linux-gnu

query stack during panic:
#0 [resolver_for_lowering_raw] getting the resolver for lowering
end of query stack
error: aborting due to 3 previous errors

For more information about this error, try `rustc --explain E0658`.

@rustbot label +F-cfg_eval +F-stmt_expr_attributes
@matthiaskrgr matthiaskrgr added I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. C-bug Category: This is a bug. labels Aug 16, 2024
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Aug 16, 2024
@matthiaskrgr matthiaskrgr added the A-parser Area: The parsing of Rust source code to an AST label Aug 16, 2024
@matthiaskrgr
Copy link
Member Author 

fn main() {
    #[cfg_eval]#[cfg]0
}

@matthiaskrgr
Copy link
Member Author

#128725 cc @nnethercote

@nnethercote nnethercote self-assigned this Aug 19, 2024
@saethlin saethlin removed the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Aug 20, 2024
nnethercote added a commit to nnethercote/rust that referenced this issue Aug 21, 2024
@nnethercote
Return earlier in some cases in collect_token.
fb3ebe1 
This example triggers an assertion failure:
```
fn f() -> u32 {
    #[cfg_eval] #[cfg(not(FALSE))] 0
}
```
The sequence of events:
- `configure_annotatable` calls `parse_expr_force_collect`, which calls
  `collect_tokens`.
- Within that, we end up in `parse_expr_dot_or_call`, which again calls
  `collect_tokens`.
  - The return value of the `f` call is the expression `0`.
  - This inner call collects tokens for `0` (parser range 10..11) and
    creates a replacement covering `#[cfg(not(FALSE))] 0` (parser range
    0..11).
- We return to the outer `collect_tokens` call. The return value of the
  `f` call is *again* the expression `0`, again with the range 10..11,
  but the replacement from earlier covers the range 0..11. The code
  mistakenly assumes that any attributes from an inner `collect_tokens`
  call fit entirely within the body of the result of an outer
  `collect_tokens` call. So it adjusts the replacement parser range
  0..11 to a node range by subtracting 10, resulting in -10..1. This is
  an invalid range and triggers an assertion failure.

It's tricky to follow, but basically things get complicated when an AST
node is returned from an inner `collect_tokens` call and then returned
again from an outer `collect_token` node without being wrapped in any
kind of additional layer.

This commit changes `collect_tokens` to return early in some extra cases,
avoiding the construction of lazy tokens. In the example above, the
outer `collect_tokens` returns earlier because the `0` token already has
tokens and `self.capture_state.capturing` is `Capturing::No`. This early
return avoids the creation of the invalid range and the assertion
failure.

Fixes rust-lang#129166. Note: these invalid ranges have been happening for a long
time. rust-lang#128725 looks like it's at fault only because it introduced the
assertion that catches the invalid ranges.
@nnethercote nnethercote mentioned this issue Aug 21, 2024
Merged
nnethercote added a commit to nnethercote/rust that referenced this issue Aug 21, 2024
@nnethercote
Return earlier in some cases in collect_token.
889ed02 
This example triggers an assertion failure:
```
fn f() -> u32 {
    #[cfg_eval] #[cfg(not(FALSE))] 0
}
```
The sequence of events:
- `configure_annotatable` calls `parse_expr_force_collect`, which calls
  `collect_tokens`.
- Within that, we end up in `parse_expr_dot_or_call`, which again calls
  `collect_tokens`.
  - The return value of the `f` call is the expression `0`.
  - This inner call collects tokens for `0` (parser range 10..11) and
    creates a replacement covering `#[cfg(not(FALSE))] 0` (parser range
    0..11).
- We return to the outer `collect_tokens` call. The return value of the
  `f` call is *again* the expression `0`, again with the range 10..11,
  but the replacement from earlier covers the range 0..11. The code
  mistakenly assumes that any attributes from an inner `collect_tokens`
  call fit entirely within the body of the result of an outer
  `collect_tokens` call. So it adjusts the replacement parser range
  0..11 to a node range by subtracting 10, resulting in -10..1. This is
  an invalid range and triggers an assertion failure.

It's tricky to follow, but basically things get complicated when an AST
node is returned from an inner `collect_tokens` call and then returned
again from an outer `collect_token` node without being wrapped in any
kind of additional layer.

This commit changes `collect_tokens` to return early in some extra cases,
avoiding the construction of lazy tokens. In the example above, the
outer `collect_tokens` returns earlier because the `0` token already has
tokens and `self.capture_state.capturing` is `Capturing::No`. This early
return avoids the creation of the invalid range and the assertion
failure.

Fixes rust-lang#129166. Note: these invalid ranges have been happening for a long
time. rust-lang#128725 looks like it's at fault only because it introduced the
assertion that catches the invalid ranges.
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 21, 2024
@bors
Auto merge of rust-lang#129346 - nnethercote:fix-double-handling-in-c…
8b22b49 
…ollect_tokens, r=<try>

Fix double handling in `collect_tokens`

Double handling of AST nodes can occur in `collect_tokens`. This is when an inner call to `collect_tokens` produces an AST node, and then an outer call to `collect_tokens` produces the same AST node. This can happen in a few places, e.g. expression statements where the statement delegates `HasTokens` and `HasAttrs` to the expression. It will also happen more after rust-lang#124141.

This PR fixes some double handling cases that cause problems, including rust-lang#129166.

r? `@petrochenkov`
nnethercote added a commit to nnethercote/rust that referenced this issue Aug 21, 2024
@nnethercote
Return earlier in some cases in collect_token.
ab795f1 
This example triggers an assertion failure:
```
fn f() -> u32 {
    #[cfg_eval] #[cfg(not(FALSE))] 0
}
```
The sequence of events:
- `configure_annotatable` calls `parse_expr_force_collect`, which calls
  `collect_tokens`.
- Within that, we end up in `parse_expr_dot_or_call`, which again calls
  `collect_tokens`.
  - The return value of the `f` call is the expression `0`.
  - This inner call collects tokens for `0` (parser range 10..11) and
    creates a replacement covering `#[cfg(not(FALSE))] 0` (parser range
    0..11).
- We return to the outer `collect_tokens` call. The return value of the
  `f` call is *again* the expression `0`, again with the range 10..11,
  but the replacement from earlier covers the range 0..11. The code
  mistakenly assumes that any attributes from an inner `collect_tokens`
  call fit entirely within the body of the result of an outer
  `collect_tokens` call. So it adjusts the replacement parser range
  0..11 to a node range by subtracting 10, resulting in -10..1. This is
  an invalid range and triggers an assertion failure.

It's tricky to follow, but basically things get complicated when an AST
node is returned from an inner `collect_tokens` call and then returned
again from an outer `collect_token` node without being wrapped in any
kind of additional layer.

This commit changes `collect_tokens` to return early in some extra cases,
avoiding the construction of lazy tokens. In the example above, the
outer `collect_tokens` returns earlier because the `0` token already has
tokens and `self.capture_state.capturing` is `Capturing::No`. This early
return avoids the creation of the invalid range and the assertion
failure.

Fixes rust-lang#129166. Note: these invalid ranges have been happening for a long
time. rust-lang#128725 looks like it's at fault only because it introduced the
assertion that catches the invalid ranges.
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 22, 2024
@bors
Auto merge of rust-lang#129346 - nnethercote:fix-double-handling-in-c…
a70947b 
…ollect_tokens, r=<try>

Fix double handling in `collect_tokens`

Double handling of AST nodes can occur in `collect_tokens`. This is when an inner call to `collect_tokens` produces an AST node, and then an outer call to `collect_tokens` produces the same AST node. This can happen in a few places, e.g. expression statements where the statement delegates `HasTokens` and `HasAttrs` to the expression. It will also happen more after rust-lang#124141.

This PR fixes some double handling cases that cause problems, including rust-lang#129166.

r? `@petrochenkov`
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 23, 2024
@bors
Auto merge of rust-lang#129346 - nnethercote:fix-double-handling-in-c…
3632378 
…ollect_tokens, r=<try>

Fix double handling in `collect_tokens`

Double handling of AST nodes can occur in `collect_tokens`. This is when an inner call to `collect_tokens` produces an AST node, and then an outer call to `collect_tokens` produces the same AST node. This can happen in a few places, e.g. expression statements where the statement delegates `HasTokens` and `HasAttrs` to the expression. It will also happen more after rust-lang#124141.

This PR fixes some double handling cases that cause problems, including rust-lang#129166.

r? `@petrochenkov`
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 23, 2024
@bors
Auto merge of rust-lang#129346 - nnethercote:fix-double-handling-in-c…
028775a 
…ollect_tokens, r=<try>

Fix double handling in `collect_tokens`

Double handling of AST nodes can occur in `collect_tokens`. This is when an inner call to `collect_tokens` produces an AST node, and then an outer call to `collect_tokens` produces the same AST node. This can happen in a few places, e.g. expression statements where the statement delegates `HasTokens` and `HasAttrs` to the expression. It will also happen more after rust-lang#124141.

This PR fixes some double handling cases that cause problems, including rust-lang#129166.

r? `@petrochenkov`
bors added a commit to rust-lang-ci/rust that referenced this issue Aug 23, 2024
@bors
Auto merge of rust-lang#129346 - nnethercote:fix-double-handling-in-c…
96ad39d 
…ollect_tokens, r=<try>

Fix double handling in `collect_tokens`

Double handling of AST nodes can occur in `collect_tokens`. This is when an inner call to `collect_tokens` produces an AST node, and then an outer call to `collect_tokens` produces the same AST node. This can happen in a few places, e.g. expression statements where the statement delegates `HasTokens` and `HasAttrs` to the expression. It will also happen more after rust-lang#124141.

This PR fixes some double handling cases that cause problems, including rust-lang#129166.

r? `@petrochenkov`
bors added a commit to rust-lang-ci/rust that referenced this issue Sep 8, 2024
@bors
Auto merge of rust-lang#129346 - nnethercote:fix-double-handling-in-c…
6d05f12 
…ollect_tokens, r=petrochenkov

Fix double handling in `collect_tokens`

Double handling of AST nodes can occur in `collect_tokens`. This is when an inner call to `collect_tokens` produces an AST node, and then an outer call to `collect_tokens` produces the same AST node. This can happen in a few places, e.g. expression statements where the statement delegates `HasTokens` and `HasAttrs` to the expression. It will also happen more after rust-lang#124141.

This PR fixes some double handling cases that cause problems, including rust-lang#129166.

r? `@petrochenkov`
@bors bors closed this as completed in 1ae521e Sep 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nnethercote nnethercote

Labels
A-parser Area: The parsing of Rust source code to an AST C-bug Category: This is a bug. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@matthiaskrgr @nnethercote @saethlin @rustbot