assertion failed: bpos.to_u32() >= mbc.pos.to_u32() + mbc.bytes as u32'

[bjorn3]

Reproduction branch: https://github.com/bjorn3/rustc_codegen_cranelift/tree/syntax_assertion_fail

When applying the change from the last commit I get a panic in my code during the call to SourceMap::lookup_char_pos.

$ rustc -V
rustc 1.34.0-nightly (8a57831a4 2019-02-02)
$ uname -a
Darwin iMac.local 18.2.0 Darwin Kernel Version 18.2.0: Mon Nov 12 20:24:46 PST 2018; root:xnu-4903.231.4~2/RELEASE_X86_64 x86_64

Backtrace

thread 'rustc' panicked at 'assertion failed: bpos.to_u32() >= mbc.pos.to_u32() + mbc.bytes as u32', src/libsyntax/source_map.rs:832:17
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::_print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: rustc::util::common::panic_hook
   5: std::panicking::rust_panic_with_hook
   6: std::panicking::begin_panic
   7: syntax::source_map::SourceMap::bytepos_to_file_charpos
   8: syntax::source_map::SourceMap::lookup_char_pos
   9: rustc_codegen_cranelift::debuginfo::FunctionDebugContext::define::{{closure}}
  10: rustc_codegen_cranelift::debuginfo::FunctionDebugContext::define
  11: rustc_codegen_cranelift::base::trans_fn::{{closure}}::{{closure}}
  12: <core::option::Option<T>>::map
  13: rustc_codegen_cranelift::base::trans_fn::{{closure}}
  14: <cranelift_module::module::Module<B>>::define_function_peek_compiled
  15: rustc_codegen_cranelift::base::trans_fn
  16: rustc_codegen_cranelift::base::trans_mono_item
  17: rustc_codegen_cranelift::codegen_mono_items::{{closure}}::{{closure}}
  18: rustc_codegen_cranelift::unimpl::try_unimpl::{{closure}}
  19: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
  20: std::panicking::try::do_call
  21: __rust_maybe_catch_panic
  22: std::panicking::try
  23: std::panic::catch_unwind
  24: rustc_codegen_cranelift::unimpl::try_unimpl
  25: rustc_codegen_cranelift::codegen_mono_items::{{closure}}
  26: rustc_codegen_cranelift::time
  27: rustc_codegen_cranelift::codegen_mono_items
  28: rustc_codegen_cranelift::codegen_cgus
  29: <rustc_codegen_cranelift::CraneliftCodegenBackend as rustc_codegen_utils::codegen_backend::CodegenBackend>::codegen_crate
  30: rustc::util::common::time
  31: rustc_driver::driver::phase_4_codegen
  32: rustc_driver::driver::compile_input::{{closure}}
  33: <std::thread::local::LocalKey<T>>::with
  34: rustc::ty::context::TyCtxt::create_and_enter
  35: rustc_driver::driver::compile_input
  36: rustc_driver::run_compiler_with_pool
  37: <scoped_tls::ScopedKey<T>>::set
  38: rustc_driver::run_compiler
  39: <scoped_tls::ScopedKey<T>>::set

[hellow554]

Cannot reproduce on my linux.
Can you provide a small MCVE? Is that possible? Maybe without all the externals :/

[kentfredric]

I can reproduce a very similar problem (appears to involve the same origin and failing assertion of syntax::source_map::SourceMap::bytepos_to_file_charpos) compiling rust-lang/regex@8977e35
with rustc 1.34.0-nightly (02c4c2892 2019-02-26) running on x86_64-unknown-linux-gnu
and compiler flags:

-Z mir-opt-level=2 -C debuginfo=2 -C incremental --crate-type lib

For me, setting mir-opt-level to 2 or greater is required to present this failure.

backtrace

 RUST_BACKTRACE=1 cargo build
   Compiling regex v1.1.2 (/home/kent/rust/regex)
thread 'rustc' panicked at 'assertion failed: bpos.to_u32() >= mbc.pos.to_u32() + mbc.bytes as u32', src/libsyntax/source_map.rs:834:17
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
             at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
   1: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:70
   2: std::panicking::default_hook::{{closure}}
             at src/libstd/sys_common/backtrace.rs:58
             at src/libstd/panicking.rs:200
   3: std::panicking::default_hook
             at src/libstd/panicking.rs:215
   4: rustc::util::common::panic_hook
   5: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:482
   6: std::panicking::begin_panic
   7: syntax::source_map::SourceMap::bytepos_to_file_charpos
   8: syntax::source_map::SourceMap::lookup_char_pos
   9: >::scope_metadata_for_loc
  10: >::debug_loc
  11: rustc_codegen_ssa::mir::codegen_mir
  12: rustc_codegen_ssa::base::codegen_instance
  13: rustc_codegen_ssa::mono_item::MonoItemExt::define
  14: rustc_codegen_llvm::base::compile_codegen_unit::module_codegen
  15: rustc::dep_graph::graph::DepGraph::with_task
  16: rustc_codegen_llvm::base::compile_codegen_unit
  17: rustc_codegen_ssa::base::codegen_crate
  18: ::codegen_crate
  19: rustc::util::common::time
  20: rustc_driver::driver::phase_4_codegen
  21: rustc_driver::driver::compile_input::{{closure}}
  22: >::with
  23: rustc::ty::context::TyCtxt::create_and_enter
  24: rustc_driver::driver::compile_input
  25: rustc_driver::run_compiler_with_pool
  26: >::set
  27: rustc_driver::run_compiler
  28: syntax::with_globals
  29: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:87
  30: >::call_box
  31: std::sys::unix::thread::Thread::new::thread_start
             at /rustc/02c4c28920ddfdb43e68a363b7fe59ac4c92399c/src/liballoc/boxed.rs:759
             at src/libstd/sys_common/thread.rs:14
             at src/libstd/sys/unix/thread.rs:81
  32: 
  33: clone
query stack during panic:
end of query stack
error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports
note: rustc 1.34.0-nightly (02c4c2892 2019-02-26) running on x86_64-unknown-linux-gnu
note: compiler flags: -Z mir-opt-level=2 -C debuginfo=2 -C incremental --crate-type lib

[dwrensha]

With the help of libfuzzer-sys, I've found a tiny test that reproduces this crash on Linux and Mac:

$ echo I1vQhQ== | base64 -D > main.rs

$ rustc main.rs
error: this file contains an un-closed delimiter
 --> main.rs:1:4
  |
1 | #[Ѕ
  |  - ^
  |  |
  |  un-closed delimiter

thread 'rustc' panicked at 'assertion failed: bpos.to_u32() >= mbc.pos.to_u32() + mbc.bytes as u32', src/libsyntax/source_map.rs:824:17
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
error: aborting due to previous error


error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.36.0 (a53f9df32 2019-07-03) running on x86_64-apple-darwin

[bjorn3]

Got another one:

thread 'rustc' panicked at 'assertion failed: bpos.to_u32() >= mbc.pos.to_u32() + mbc.bytes as u32', src/libsyntax/source_map.rs:867:17
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/backtrace-0.3.35/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/backtrace-0.3.35/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:47
   3: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:36
   4: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:200
   5: std::panicking::default_hook
             at src/libstd/panicking.rs:214
   6: rustc::util::common::panic_hook
   7: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:481
   8: std::panicking::begin_panic
   9: syntax::source_map::SourceMap::bytepos_to_file_charpos
  10: syntax::source_map::SourceMap::lookup_char_pos
  11: rustc_codegen_cranelift::debuginfo::FunctionDebugContext::define::{{closure}}
             at src/debuginfo.rs:293
  12: rustc_codegen_cranelift::debuginfo::FunctionDebugContext::define
             at src/debuginfo.rs:317
  13: rustc_codegen_cranelift::base::trans_fn::{{closure}}
             at src/base.rs:89
  14: core::option::Option<T>::map
             at /rustc/17e73e801a75559eac5c932ff07bd9c8499a1364/src/libcore/option.rs:447
  15: rustc_codegen_cranelift::base::trans_fn
             at src/base.rs:87
  16: rustc_codegen_cranelift::driver::trans_mono_item
             at src/driver.rs:344
  17: rustc_codegen_cranelift::driver::codegen_mono_items::{{closure}}::{{closure}}
             at src/driver.rs:304
  18: rustc_codegen_cranelift::unimpl::try_unimpl::{{closure}}
             at src/unimpl.rs:28
  19: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /rustc/17e73e801a75559eac5c932ff07bd9c8499a1364/src/libstd/panic.rs:315
  20: std::panicking::try::do_call
             at /rustc/17e73e801a75559eac5c932ff07bd9c8499a1364/src/libstd/panicking.rs:296
  21: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:80
  22: std::panicking::try
             at /rustc/17e73e801a75559eac5c932ff07bd9c8499a1364/src/libstd/panicking.rs:275
  23: std::panic::catch_unwind
             at /rustc/17e73e801a75559eac5c932ff07bd9c8499a1364/src/libstd/panic.rs:394
  24: rustc_codegen_cranelift::unimpl::try_unimpl
             at src/unimpl.rs:28
  25: rustc_codegen_cranelift::driver::codegen_mono_items::{{closure}}
             at src/driver.rs:302
  26: rustc_codegen_cranelift::driver::time
             at src/driver.rs:358
  27: rustc_codegen_cranelift::driver::codegen_mono_items
             at src/driver.rs:300
  28: rustc_codegen_cranelift::driver::codegen_cgus
             at src/driver.rs:287
  29: rustc_codegen_cranelift::driver::run_aot
             at src/driver.rs:205
  30: rustc_codegen_cranelift::driver::codegen_crate
             at src/driver.rs:39
  31: <rustc_codegen_cranelift::CraneliftCodegenBackend as rustc_codegen_utils::codegen_backend::CodegenBackend>::codegen_crate
             at src/lib.rs:202
  32: rustc::util::common::time
  33: rustc_interface::passes::BoxedGlobalCtxt::access::{{closure}}
  34: rustc_interface::passes::create_global_ctxt::{{closure}}
  35: rustc_interface::passes::BoxedGlobalCtxt::enter
  36: rustc_interface::queries::Query<T>::compute
  37: rustc_interface::queries::<impl rustc_interface::interface::Compiler>::ongoing_codegen
  38: rustc_interface::interface::run_compiler_in_existing_thread_pool
  39: std::thread::local::LocalKey<T>::with
  40: scoped_tls::ScopedKey<T>::set
  41: syntax::with_globals

Happens when compiling regex tests with -Zmir-opt-level=3 and calling SourceMap::lookup_char_pos on every span found in the mir of the functions needing codegen.

[JohnTitor]

With the help of libfuzzer-sys, I've found a tiny test that reproduces this crash on Linux and Mac:

According to glaicer, the ICE from this example has been gone.

[hellow554]

@bjorn3 can you confirm that your example does not crash on nightly anymore?

[hellow554]

Has been fixed in #66141

[bjorn3]

#66054 is likely the part of the rollup fixing it.

[hellow554]

Shouldn't a testcase be added instead of closing this?

[dwrensha]

A similar crash that still happens: #66473