Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI failure injection_queue_depth #5892

Closed
NobodyXu opened this issue Jul 28, 2023 · 2 comments · Fixed by #6559
Closed

CI failure injection_queue_depth #5892

NobodyXu opened this issue Jul 28, 2023 · 2 comments · Fixed by #6559
Assignees
Labels
A-tokio Area: The main tokio crate C-bug Category: This is a bug. M-runtime Module: tokio/runtime

Comments

@NobodyXu
Copy link
Contributor

This bug is reproduced in the windows CI of #5885

Description

stack backtrace:

running 1 test
test injection_queue_depth ... FAILED

failures:

failures:
    injection_queue_depth

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 19 filtered out; finished in 0.31s


--- STDERR:              tokio::rt_metrics injection_queue_depth ---
thread 'injection_queue_depth' panicked at '0', tokio\tests\rt_metrics.rs:534:5
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library\std\src\panicking.rs:593
   1: core::panicking::panic_fmt
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library\core\src\panicking.rs:67
   2: core::panicking::panic_display<usize>
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\core\src\panicking.rs:150
   3: rt_metrics::injection_queue_depth
             at .\tests\rt_metrics.rs:534
   4: rt_metrics::injection_queue_depth::closure$0
             at .\tests\rt_metrics.rs:491
   5: core::ops::function::FnOnce::call_once<rt_metrics::injection_queue_depth::closure_env$0,tuple$<> >
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\core\src\ops\function.rs:250
   6: core::ops::function::FnOnce::call_once
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library\core\src\ops\function.rs:250
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', tokio\tests\rt_metrics.rs:523:39
stack backtrace:
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', tokio\tests\rt_metrics.rs:516:38
   0: std::panicking::begin_panic_handler
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library\std\src\panicking.rs:593
   1: core::panicking::panic_fmt
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library\core\src\panicking.rs:67
   2: core::result::unwrap_failed
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library\core\src\result.rs:1651
   3: enum2$<core::result::Result<tuple$<>,std::sync::mpsc::RecvError> >::unwrap<tuple$<>,std::sync::mpsc::RecvError>
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\core\src\result.rs:1076
   4: rt_metrics::injection_queue_depth::async_block$3
             at .\tests\rt_metrics.rs:523
   5: tracing::instrument::impl$0::poll<enum2$<rt_metrics::injection_queue_depth::async_block_env$3> >
             at C:\Users\runneradmin\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tracing-0.1.37\src\instrument.rs:272
   6: tokio::runtime::task::core::impl$6::poll::closure$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$3> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\core.rs:334
   7: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$3> > > > >::with_mut
             at .\src\loom\std\unsafe_cell.rs:16
   8: tokio::runtime::task::core::Core<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$3> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::poll<tracing::instrument::Instrumented<enum2$<rt
             at .\src\runtime\task\core.rs:323
   9: tokio::runtime::task::harness::poll_future::closure$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$3> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\harness.rs:485
  10: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$3> >,alloc::sync:
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\core\src\panic\unwind_safe.rs:271
  11: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$3> >,alloc::sync::Arc<tokio::runt
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panicking.rs:500
  12: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$1<tracing::instrument::Instrumented<enum2$<rt_metrics::worker_steal_count::async_block$0::async_block_env$0> >,alloc::sync::Ar
  13: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panicking.rs:464
  14: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$3> >,alloc::sync::Arc<tokio::runtime:
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panic.rs:142
  15: tokio::runtime::task::harness::poll_future<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$3> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\harness.rs:473
  16: tokio::runtime::task::harness::Harness<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$3> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::poll_inner<tracing::instrument::Instrument
             at .\src\runtime\task\harness.rs:208
  17: tokio::runtime::task::harness::Harness<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$3> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::poll<tracing::instrument::Instrumented<enu
             at .\src\runtime\task\harness.rs:153
  18: tokio::runtime::task::raw::poll<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$3> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\raw.rs:276
  19: tokio::runtime::task::raw::RawTask::poll
             at .\src\runtime\task\raw.rs:200
  20: tokio::runtime::task::LocalNotified<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::run<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\mod.rs:400
  21: tokio::runtime::scheduler::multi_thread::worker::impl$1::run_task::closure$0
             at .\src\runtime\scheduler\multi_thread\worker.rs:576
  22: tokio::runtime::coop::with_budget
             at .\src\runtime\coop.rs:107
  23: tokio::runtime::coop::budget
             at .\src\runtime\coop.rs:73
  24: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at .\src\runtime\scheduler\multi_thread\worker.rs:575
  25: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at .\src\runtime\scheduler\multi_thread\worker.rs:526
  26: tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure$0
             at .\src\runtime\scheduler\multi_thread\worker.rs:491
  27: tokio::runtime::context::scoped::Scoped<enum2$<tokio::runtime::scheduler::Context> >::set<enum2$<tokio::runtime::scheduler::Context>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0,tuple$<> >
             at .\src\runtime\context\scoped.rs:40
  28: tokio::runtime::context::set_scheduler::closure$0<tuple$<>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0>
             at .\src\runtime\context.rs:176
  29: std::thread::local::LocalKey<tokio::runtime::context::Context>::try_with<tokio::runtime::context::Context,tokio::runtime::context::set_scheduler::closure_env$0<tuple$<>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0>,tuple$
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\thread\local.rs:270
  30: std::thread::local::LocalKey<tokio::runtime::context::Context>::with<tokio::runtime::context::Context,tokio::runtime::context::set_scheduler::closure_env$0<tuple$<>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0>,tuple$<> >
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\thread\local.rs:246
  31: tokio::runtime::context::set_scheduler<tuple$<>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0>
             at .\src\runtime\context.rs:176
  32: tokio::runtime::scheduler::multi_thread::worker::run::closure$0
             at .\src\runtime\scheduler\multi_thread\worker.rs:486
  33: tokio::runtime::context::runtime::enter_runtime<tokio::runtime::scheduler::multi_thread::worker::run::closure_env$0,tuple$<> >
             at .\src\runtime\context\runtime.rs:65
  34: tokio::runtime::scheduler::multi_thread::worker::run
             at .\src\runtime\scheduler\multi_thread\worker.rs:478
  35: tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure$0
             at .\src\runtime\scheduler\multi_thread\worker.rs:447
  36: tokio::runtime::blocking::task::impl$2::poll<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0,tuple$<> >
             at .\src\runtime\blocking\task.rs:42
  37: tracing::instrument::impl$0::poll<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >
             at C:\Users\runneradmin\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tracing-0.1.37\src\instrument.rs:272
  38: tokio::runtime::task::core::impl$6::poll::closure$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::Blocki
             at .\src\runtime\task\core.rs:334
  39: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> > > > >::with
             at .\src\loom\std\unsafe_cell.rs:16
  40: tokio::runtime::task::core::Core<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedule>::poll<t
             at .\src\runtime\task\core.rs:323
  41: tokio::runtime::task::harness::poll_future::closure$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::Bloc
             at .\src\runtime\task\harness.rs:485
  42: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::mul
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\core\src\panic\unwind_safe.rs:271
  43: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worke
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panicking.rs:500
  44: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$7::drop::closure_env$0<tuple$<> > >,tuple$<> >
  45: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runti
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panicking.rs:464
  46: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::i
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panic.rs:142
  47: tokio::runtime::task::harness::poll_future<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedul
             at .\src\runtime\task\harness.rs:473
  48: tokio::runtime::task::harness::Harness<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedule>::
             at .\src\runtime\task\harness.rs:208
  49: tokio::runtime::task::harness::Harness<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedule>::
             at .\src\runtime\task\harness.rs:153
  50: tokio::runtime::task::raw::poll<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedule>
             at .\src\runtime\task\raw.rs:276
  51: tokio::runtime::task::raw::RawTask::poll
             at .\src\runtime\task\raw.rs:200
  52: tokio::runtime::task::UnownedTask<tokio::runtime::blocking::schedule::BlockingSchedule>::run<tokio::runtime::blocking::schedule::BlockingSchedule>
             at .\src\runtime\task\mod.rs:437
  53: tokio::runtime::blocking::pool::Task::run
             at .\src\runtime\blocking\pool.rs:159
  54: tokio::runtime::blocking::pool::Inner::run
             at .\src\runtime\blocking\pool.rs:513
  55: tokio::runtime::blocking::pool::impl$6::spawn_thread::closure$0
             at .\src\runtime\blocking\pool.rs:471
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: PoisonError { .. }', tokio\tests\rt_metrics.rs:523:23
   0: std::panicking::begin_panic_handler
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library\std\src\panicking.rs:593
   1: core::panicking::panic_fmt
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library\core\src\panicking.rs:67
   2: core::result::unwrap_failed
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library\core\src\result.rs:1651
   3: enum2$<core::result::Result<tuple$<>,std::sync::mpsc::RecvError> >::unwrap<tuple$<>,std::sync::mpsc::RecvError>
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\core\src\result.rs:1076
   4: rt_metrics::injection_queue_depth::async_block$1
             at .\tests\rt_metrics.rs:516
   5: tracing::instrument::impl$0::poll<enum2$<rt_metrics::injection_queue_depth::async_block_env$1> >
             at C:\Users\runneradmin\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tracing-0.1.37\src\instrument.rs:272
   6: tokio::runtime::task::core::impl$6::poll::closure$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$1> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\core.rs:334
   7: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$1> > > > >::with_mut
             at .\src\loom\std\unsafe_cell.rs:16
   8: tokio::runtime::task::core::Core<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$1> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::poll<tracing::instrument::Instrumented<enum2$<rt
             at .\src\runtime\task\core.rs:323
   9: tokio::runtime::task::harness::poll_future::closure$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$1> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\harness.rs:485
  10: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$1> >,alloc::sync:
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\core\src\panic\unwind_safe.rs:271
  11: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$1> >,alloc::sync::Arc<tokio::runt
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panicking.rs:500
  12: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$1<tracing::instrument::Instrumented<enum2$<rt_metrics::worker_steal_count::async_block$0::async_block_env$0> >,alloc::sync::Ar
  13: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panicking.rs:464
  14: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$1> >,alloc::sync::Arc<tokio::runtime:
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panic.rs:142
  15: tokio::runtime::task::harness::poll_future<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$1> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\harness.rs:473
  16: tokio::runtime::task::harness::Harness<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$1> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::poll_inner<tracing::instrument::Instrument
             at .\src\runtime\task\harness.rs:208
  17: tokio::runtime::task::harness::Harness<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$1> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::poll<tracing::instrument::Instrumented<enu
             at .\src\runtime\task\harness.rs:153
  18: tokio::runtime::task::raw::poll<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$1> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\raw.rs:276
  19: tokio::runtime::task::raw::RawTask::poll
             at .\src\runtime\task\raw.rs:200
  20: tokio::runtime::task::LocalNotified<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::run<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\mod.rs:400
  21: tokio::runtime::scheduler::multi_thread::worker::impl$1::run_task::closure$0
             at .\src\runtime\scheduler\multi_thread\worker.rs:576
  22: tokio::runtime::coop::with_budget
             at .\src\runtime\coop.rs:107
  23: tokio::runtime::coop::budget
             at .\src\runtime\coop.rs:73
  24: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at .\src\runtime\scheduler\multi_thread\worker.rs:575
  25: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at .\src\runtime\scheduler\multi_thread\worker.rs:526
  26: tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure$0
             at .\src\runtime\scheduler\multi_thread\worker.rs:491
  27: tokio::runtime::context::scoped::Scoped<enum2$<tokio::runtime::scheduler::Context> >::set<enum2$<tokio::runtime::scheduler::Context>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0,tuple$<> >
             at .\src\runtime\context\scoped.rs:40
  28: tokio::runtime::context::set_scheduler::closure$0<tuple$<>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0>
             at .\src\runtime\context.rs:176
  29: std::thread::local::LocalKey<tokio::runtime::context::Context>::try_with<tokio::runtime::context::Context,tokio::runtime::context::set_scheduler::closure_env$0<tuple$<>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0>,tuple$
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\thread\local.rs:270
  30: std::thread::local::LocalKey<tokio::runtime::context::Context>::with<tokio::runtime::context::Context,tokio::runtime::context::set_scheduler::closure_env$0<tuple$<>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0>,tuple$<> >
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\thread\local.rs:246
  31: tokio::runtime::context::set_scheduler<tuple$<>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0>
             at .\src\runtime\context.rs:176
  32: tokio::runtime::scheduler::multi_thread::worker::run::closure$0
             at .\src\runtime\scheduler\multi_thread\worker.rs:486
  33: tokio::runtime::context::runtime::enter_runtime<tokio::runtime::scheduler::multi_thread::worker::run::closure_env$0,tuple$<> >
             at .\src\runtime\context\runtime.rs:65
  34: tokio::runtime::scheduler::multi_thread::worker::run
             at .\src\runtime\scheduler\multi_thread\worker.rs:478
  35: tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure$0
             at .\src\runtime\scheduler\multi_thread\worker.rs:447
  36: tokio::runtime::blocking::task::impl$2::poll<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0,tuple$<> >
             at .\src\runtime\blocking\task.rs:42
  37: tracing::instrument::impl$0::poll<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >
             at C:\Users\runneradmin\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tracing-0.1.37\src\instrument.rs:272
  38: tokio::runtime::task::core::impl$6::poll::closure$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::Blocki
             at .\src\runtime\task\core.rs:334
  39: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> > > > >::with
             at .\src\loom\std\unsafe_cell.rs:16
  40: tokio::runtime::task::core::Core<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedule>::poll<t
             at .\src\runtime\task\core.rs:323
  41: tokio::runtime::task::harness::poll_future::closure$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::Bloc
             at .\src\runtime\task\harness.rs:485
  42: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::mul
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\core\src\panic\unwind_safe.rs:271
  43: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worke
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panicking.rs:500
  44: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$7::drop::closure_env$0<tuple$<> > >,tuple$<> >
  45: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runti
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panicking.rs:464
  46: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::i
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panic.rs:142
  47: tokio::runtime::task::harness::poll_future<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedul
             at .\src\runtime\task\harness.rs:473
  48: tokio::runtime::task::harness::Harness<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedule>::
             at .\src\runtime\task\harness.rs:208
  49: tokio::runtime::task::harness::Harness<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedule>::
             at .\src\runtime\task\harness.rs:153
  50: tokio::runtime::task::raw::poll<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedule>
             at .\src\runtime\task\raw.rs:276
  51: tokio::runtime::task::raw::RawTask::poll
             at .\src\runtime\task\raw.rs:200
  52: tokio::runtime::task::UnownedTask<tokio::runtime::blocking::schedule::BlockingSchedule>::run<tokio::runtime::blocking::schedule::BlockingSchedule>
             at .\src\runtime\task\mod.rs:437
  53: tokio::runtime::blocking::pool::Task::run
             at .\src\runtime\blocking\pool.rs:159
  54: tokio::runtime::blocking::pool::Inner::run
             at .\src\runtime\blocking\pool.rs:513
  55: tokio::runtime::blocking::pool::impl$6::spawn_thread::closure$0
             at .\src\runtime\blocking\pool.rs:471
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: RecvError', tokio\tests\rt_metrics.rs:517:38
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library\std\src\panicking.rs:593
   1: core::panicking::panic_fmt
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library\core\src\panicking.rs:67
   2: core::result::unwrap_failed
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library\core\src\result.rs:1651
   3: enum2$<core::result::Result<tuple$<>,std::sync::mpsc::RecvError> >::unwrap<tuple$<>,std::sync::mpsc::RecvError>
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\core\src\result.rs:1076
   4: rt_metrics::injection_queue_depth::async_block$2
             at .\tests\rt_metrics.rs:517
   5: tracing::instrument::impl$0::poll<enum2$<rt_metrics::injection_queue_depth::async_block_env$2> >
             at C:\Users\runneradmin\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tracing-0.1.37\src\instrument.rs:272
   6: tokio::runtime::task::core::impl$6::poll::closure$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$2> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\core.rs:334
   7: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$2> > > > >::with_mut
             at .\src\loom\std\unsafe_cell.rs:16
   8: tokio::runtime::task::core::Core<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$2> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::poll<tracing::instrument::Instrumented<enum2$<rt
             at .\src\runtime\task\core.rs:323
   9: tokio::runtime::task::harness::poll_future::closure$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$2> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\harness.rs:485
  10: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$2> >,alloc::sync:
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\core\src\panic\unwind_safe.rs:271
  11: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$2> >,alloc::sync::Arc<tokio::runt
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panicking.rs:500
  12: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$1<tracing::instrument::Instrumented<enum2$<rt_metrics::worker_steal_count::async_block$0::async_block_env$0> >,alloc::sync::Ar
  13: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panicking.rs:464
  14: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$2> >,alloc::sync::Arc<tokio::runtime:
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panic.rs:142
  15: tokio::runtime::task::harness::poll_future<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$2> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\harness.rs:473
  16: tokio::runtime::task::harness::Harness<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$2> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::poll_inner<tracing::instrument::Instrument
             at .\src\runtime\task\harness.rs:208
  17: tokio::runtime::task::harness::Harness<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$2> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::poll<tracing::instrument::Instrumented<enu
             at .\src\runtime\task\harness.rs:153
  18: tokio::runtime::task::raw::poll<tracing::instrument::Instrumented<enum2$<rt_metrics::injection_queue_depth::async_block_env$2> >,alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\raw.rs:276
  19: tokio::runtime::task::raw::RawTask::poll
             at .\src\runtime\task\raw.rs:200
  20: tokio::runtime::task::LocalNotified<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >::run<alloc::sync::Arc<tokio::runtime::scheduler::multi_thread::handle::Handle> >
             at .\src\runtime\task\mod.rs:400
  21: tokio::runtime::scheduler::multi_thread::worker::impl$1::run_task::closure$0
             at .\src\runtime\scheduler\multi_thread\worker.rs:576
  22: tokio::runtime::coop::with_budget
             at .\src\runtime\coop.rs:107
  23: tokio::runtime::coop::budget
             at .\src\runtime\coop.rs:73
  24: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
             at .\src\runtime\scheduler\multi_thread\worker.rs:575
  25: tokio::runtime::scheduler::multi_thread::worker::Context::run
             at .\src\runtime\scheduler\multi_thread\worker.rs:526
  26: tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure$0
             at .\src\runtime\scheduler\multi_thread\worker.rs:491
  27: tokio::runtime::context::scoped::Scoped<enum2$<tokio::runtime::scheduler::Context> >::set<enum2$<tokio::runtime::scheduler::Context>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0,tuple$<> >
             at .\src\runtime\context\scoped.rs:40
  28: tokio::runtime::context::set_scheduler::closure$0<tuple$<>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0>
             at .\src\runtime\context.rs:176
  29: std::thread::local::LocalKey<tokio::runtime::context::Context>::try_with<tokio::runtime::context::Context,tokio::runtime::context::set_scheduler::closure_env$0<tuple$<>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0>,tuple$
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\thread\local.rs:270
  30: std::thread::local::LocalKey<tokio::runtime::context::Context>::with<tokio::runtime::context::Context,tokio::runtime::context::set_scheduler::closure_env$0<tuple$<>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0>,tuple$<> >
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\thread\local.rs:246
  31: tokio::runtime::context::set_scheduler<tuple$<>,tokio::runtime::scheduler::multi_thread::worker::run::closure$0::closure_env$0>
             at .\src\runtime\context.rs:176
  32: tokio::runtime::scheduler::multi_thread::worker::run::closure$0
             at .\src\runtime\scheduler\multi_thread\worker.rs:486
  33: tokio::runtime::context::runtime::enter_runtime<tokio::runtime::scheduler::multi_thread::worker::run::closure_env$0,tuple$<> >
             at .\src\runtime\context\runtime.rs:65
  34: tokio::runtime::scheduler::multi_thread::worker::run
             at .\src\runtime\scheduler\multi_thread\worker.rs:478
  35: tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure$0
             at .\src\runtime\scheduler\multi_thread\worker.rs:447
  36: tokio::runtime::blocking::task::impl$2::poll<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0,tuple$<> >
             at .\src\runtime\blocking\task.rs:42
  37: tracing::instrument::impl$0::poll<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >
             at C:\Users\runneradmin\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tracing-0.1.37\src\instrument.rs:272
  38: tokio::runtime::task::core::impl$6::poll::closure$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::Blocki
             at .\src\runtime\task\core.rs:334
  39: tokio::loom::std::unsafe_cell::UnsafeCell<enum2$<tokio::runtime::task::core::Stage<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> > > > >::with
             at .\src\loom\std\unsafe_cell.rs:16
  40: tokio::runtime::task::core::Core<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedule>::poll<t
             at .\src\runtime\task\core.rs:323
  41: tokio::runtime::task::harness::poll_future::closure$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::Bloc
             at .\src\runtime\task\harness.rs:485
  42: core::panic::unwind_safe::impl$23::call_once<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::mul
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\core\src\panic\unwind_safe.rs:271
  43: std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worke
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panicking.rs:500
  44: std::panicking::try::do_catch<core::panic::unwind_safe::AssertUnwindSafe<std::thread::impl$7::drop::closure_env$0<tuple$<> > >,tuple$<> >
  45: std::panicking::try<enum2$<core::task::poll::Poll<tuple$<> > >,core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runti
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panicking.rs:464
  46: std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<tokio::runtime::task::harness::poll_future::closure_env$0<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::i
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225\library\std\src\panic.rs:142
  47: tokio::runtime::task::harness::poll_future<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedul
             at .\src\runtime\task\harness.rs:473
  48: tokio::runtime::task::harness::Harness<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedule>::
             at .\src\runtime\task\harness.rs:208
  49: tokio::runtime::task::harness::Harness<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedule>::
             at .\src\runtime\task\harness.rs:153
  50: tokio::runtime::task::raw::poll<tracing::instrument::Instrumented<tokio::runtime::blocking::task::BlockingTask<tokio::runtime::scheduler::multi_thread::worker::impl$0::launch::closure_env$0> >,tokio::runtime::blocking::schedule::BlockingSchedule>
             at .\src\runtime\task\raw.rs:276
  51: tokio::runtime::task::raw::RawTask::poll
             at .\src\runtime\task\raw.rs:200
  52: tokio::runtime::task::UnownedTask<tokio::runtime::blocking::schedule::BlockingSchedule>::run<tokio::runtime::blocking::schedule::BlockingSchedule>
             at .\src\runtime\task\mod.rs:437
  53: tokio::runtime::blocking::pool::Task::run
             at .\src\runtime\blocking\pool.rs:159
  54: tokio::runtime::blocking::pool::Inner::run
             at .\src\runtime\blocking\pool.rs:513
  55: tokio::runtime::blocking::pool::impl$6::spawn_thread::closure$0
             at .\src\runtime\blocking\pool.rs:471
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
@NobodyXu NobodyXu added A-tokio Area: The main tokio crate C-bug Category: This is a bug. labels Jul 28, 2023
@Darksonn Darksonn added the M-runtime Module: tokio/runtime label Jul 28, 2023
@carllerche carllerche self-assigned this Aug 22, 2023
@Darksonn
Copy link
Contributor

Seen again on macOS ci.

@Darksonn
Copy link
Contributor

Darksonn commented Jan 2, 2024

Seen again in Windows CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-tokio Area: The main tokio crate C-bug Category: This is a bug. M-runtime Module: tokio/runtime
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants