Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ASAN_X] heap-buffer-overflow in l1t::L1GTSingleCollectionCut::checkObject (checkEtaDependentCuts) #47194

Open
iarspider opened this issue Jan 28, 2025 · 11 comments
Open

[ASAN_X] heap-buffer-overflow in l1t::L1GTSingleCollectionCut::checkObject (checkEtaDependentCuts) #47194

iarspider opened this issue Jan 28, 2025 · 11 comments
Labels
l1-pending pending-signatures upgrade-pending

Comments

@iarspider
Copy link
Contributor

In CMSSW_15_0_ASAN_X_2025-01-27-2300, several RelVals failed with heap-buffer-overflow in l1t::L1GTSingleCollectionCut::checkObject:

==3854934==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x602000624a58 at pc 0x151e2ad1071d bp 0x151e35e4e3d0 sp 0x151e35e4e3c8
READ of size 4 at 0x602000624a58 thread T2
    #0 0x151e2ad1071c in l1t::L1GTSingleCollectionCut::checkObject(l1t::P2GTCandidate const&) const (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/pluginL1TriggerPhase2L1GTAuto.so+0x11071c)
    #1 0x151e2aea20ba in L1GTQuadObjectCond::filter(edm::StreamID, edm::Event&, edm::EventSetup const&) const (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/pluginL1TriggerPhase2L1GTAuto.so+0x2a20ba)
    #2 0x151e94655bde in edm::global::EDFilterBase::doEvent(edm::EventTransitionInfo const&, edm::ActivityRegistry*, edm::ModuleCallingContext const*) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0xa55bde)
    #3 0x151e9463dbd8 in edm::WorkerT<edm::global::EDFilterBase>::implDo(edm::EventTransitionInfo const&, edm::ModuleCallingContext const*) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0xa3dbd8)
    #4 0x151e94259ea9 in decltype ({parm#1}()) edm::convertException::wrap<edm::Worker::runModule<edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}>(edm::Worker::runModule<edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1> >(edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::Context const*)::{lambda()#1}) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0x659ea9)
    #5 0x151e9425a5ab in std::__exception_ptr::exception_ptr edm::Worker::runModuleAfterAsyncPrefetch<edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1> >(std::__exception_ptr::exception_ptr, edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::TransitionInfoType const&, edm::StreamID, edm::ParentContext const&, edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1>::Context const*) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0x65a5ab)
    #6 0x151e9426ae0d in edm::Worker::RunModuleTask<edm::OccurrenceTraits<edm::EventPrincipal, (edm::BranchActionType)1> >::execute() (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0x66ae0d)
    #7 0x151e9527e59e in tbb::detail::d1::function_task<edm::WaitingTaskList::announce()::{lambda()#1}>::execute(tbb::detail::d1::execution_data&) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreConcurrency.so+0x1559e)
    #8 0x151e95129b3a in tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::outermost_worker_waiter>(tbb::detail::d1::task*, tbb::detail::r1::outermost_worker_waiter&) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/task_dispatcher.h:322
    #9 0x151e95129b3a in tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::outermost_worker_waiter>(tbb::detail::d1::task*, tbb::detail::r1::outermost_worker_waiter&) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/task_dispatcher.h:458
    #10 0x151e95129b3a in tbb::detail::r1::arena::process(tbb::detail::r1::thread_data&) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/arena.cpp:137
    #11 0x151e95129b3a in tbb::detail::r1::market::process(rml::job&) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/market.cpp:599
    #12 0x151e9512bced in tbb::detail::r1::rml::private_worker::run() /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/private_server.cpp:271
    #13 0x151e9512bced in tbb::detail::r1::rml::private_worker::thread_routine(void*) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/private_server.cpp:221
    #14 0x151e922061c9 in start_thread (/lib64/libpthread.so.0+0x81c9)
    #15 0x151e928638d2 in __GI___clone (/lib64/libc.so.6+0x398d2)

0x602000624a58 is located 0 bytes to the right of 8-byte region [0x602000624a50,0x602000624a58)
allocated by thread T3 here:
    #0 0x151e94ab96d8 in operator new(unsigned long) ../../../../libsanitizer/asan/asan_new_delete.cpp:95
    #1 0x151e2ad0d92b in std::vector<int, std::allocator<int> > l1t::getParamVector<int, double>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::ParameterSet const&, std::function<int (double)>) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/pluginL1TriggerPhase2L1GTAuto.so+0x10d92b)
    #2 0x151e2ad1af51 in l1t::L1GTSingleCollectionCut::L1GTSingleCollectionCut(edm::ParameterSet const&, edm::ParameterSet const&, l1t::L1GTScales const&) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/pluginL1TriggerPhase2L1GTAuto.so+0x11af51)
    #3 0x151e2ae9eecb in L1GTQuadObjectCond::L1GTQuadObjectCond(edm::ParameterSet const&) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/pluginL1TriggerPhase2L1GTAuto.so+0x29eecb)
    #4 0x151e2aeb59fb in edm::WorkerMaker<L1GTQuadObjectCond>::makeModule(edm::ParameterSet const&) const (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/pluginL1TriggerPhase2L1GTAuto.so+0x2b59fb)
    #5 0x151e9460567f in edm::Maker::makeModule(edm::MakeModuleParams const&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) const (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0xa0567f)
    #6 0x151e941638d9 in edm::Factory::makeModule(edm::MakeModuleParams const&, edm::ModuleTypeResolverMaker const*, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) const (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0x5638d9)
    #7 0x151e941d2b00 in edm::ModuleRegistry::getModule(edm::MakeModuleParams const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0x5d2b00)
    #8 0x151e94617336 in edm::WorkerRegistry::getWorker(edm::WorkerParams const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0xa17336)
    #9 0x151e9460ac44 in edm::WorkerManager::getWorker(edm::ParameterSet&, edm::SignallingProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0xa0ac44)
    #10 0x151e944d8847 in edm::(anonymous namespace)::getWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::ParameterSet&, edm::WorkerManager&, edm::SignallingProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0x8d8847)
    #11 0x151e944fcff1 in edm::StreamSchedule::fillWorkers(edm::ParameterSet&, edm::SignallingProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, std::vector<edm::WorkerInPath, std::allocator<edm::WorkerInPath> >&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, edm::ConditionalTaskHelper const&, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0x8fcff1)
    #12 0x151e94500f72 in edm::StreamSchedule::fillTrigPath(edm::ParameterSet&, edm::SignallingProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<edm::HLTGlobalStatus>, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, edm::ConditionalTaskHelper const&, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0x900f72)
    #13 0x151e94506206 in edm::StreamSchedule::StreamSchedule(std::shared_ptr<edm::TriggerResultInserter>, std::vector<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> > > >&, std::vector<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> > > >&, std::shared_ptr<edm::ModuleRegistry>, edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::SignallingProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration const>, edm::StreamID, edm::ProcessContext const*) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0x906206)
    #14 0x151e9441559f in edm::Schedule::Schedule(edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::SignallingProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration const>, edm::PreallocationConfiguration const&, edm::ProcessContext const*, edm::ModuleTypeResolverMaker const*) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0x81559f)
    #15 0x151e94492a18 in edm::ScheduleItems::initModules(edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::ProcessContext const*, edm::ModuleTypeResolverMaker const*) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0x892a18)
    #16 0x151e93f2e535 in tbb::detail::d1::function_task<edm::EventProcessor::init(std::shared_ptr<edm::ProcessDesc>&, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy)::{lambda()#1}>::execute(tbb::detail::d1::execution_data&) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/libFWCoreFramework.so+0x32e535)
    #17 0x151e95129b3a in tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::outermost_worker_waiter>(tbb::detail::d1::task*, tbb::detail::r1::outermost_worker_waiter&) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/task_dispatcher.h:322
    #18 0x151e95129b3a in tbb::detail::d1::task* tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::outermost_worker_waiter>(tbb::detail::d1::task*, tbb::detail::r1::outermost_worker_waiter&) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/task_dispatcher.h:458
    #19 0x151e95129b3a in tbb::detail::r1::arena::process(tbb::detail::r1::thread_data&) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/arena.cpp:137
    #20 0x151e95129b3a in tbb::detail::r1::market::process(rml::job&) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/market.cpp:599
    #21 0x151e9512bced in tbb::detail::r1::rml::private_worker::run() /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/private_server.cpp:271
    #22 0x151e9512bced in tbb::detail::r1::rml::private_worker::thread_routine(void*) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/private_server.cpp:221

Thread T2 created by T0 here:
    #0 0x151e94a4a136 in __interceptor_pthread_create ../../../../libsanitizer/asan/asan_interceptors.cpp:207
    #1 0x151e9512b34f in tbb::detail::r1::rml::internal::thread_monitor::launch(void* (*)(void*), void*, unsigned long) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/rml_thread_monitor.h:208
    #2 0x151e9512b34f in tbb::detail::r1::rml::private_worker::wake_or_launch() /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/private_server.cpp:305
    #3 0x151e9512b34f in tbb::detail::r1::rml::private_server::wake_some(int) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/private_server.cpp:412

Thread T3 created by T0 here:
    #0 0x151e94a4a136 in __interceptor_pthread_create ../../../../libsanitizer/asan/asan_interceptors.cpp:207
    #1 0x151e9512b34f in tbb::detail::r1::rml::internal::thread_monitor::launch(void* (*)(void*), void*, unsigned long) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/rml_thread_monitor.h:208
    #2 0x151e9512b34f in tbb::detail::r1::rml::private_worker::wake_or_launch() /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/private_server.cpp:305
    #3 0x151e9512b34f in tbb::detail::r1::rml::private_server::wake_some(int) /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-de1d0edd23fc7bdc98ffd0075777fffa/tbb-v2021.9.0/src/tbb/private_server.cpp:412

SUMMARY: AddressSanitizer: heap-buffer-overflow (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02874/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_ASAN_X_2025-01-27-2300/lib/el8_amd64_gcc12/pluginL1TriggerPhase2L1GTAuto.so+0x11071c) in l1t::L1GTSingleCollectionCut::checkObject(l1t::P2GTCandidate const&) const
Shadow bytes around the buggy address:
  0x0c04800bc8f0: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 04
  0x0c04800bc900: fa fa 00 fa fa fa 00 00 fa fa 00 04 fa fa 00 fa
  0x0c04800bc910: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 00
  0x0c04800bc920: fa fa 00 00 fa fa 02 fa fa fa 00 00 fa fa 00 00
  0x0c04800bc930: fa fa 00 00 fa fa 02 fa fa fa 00 00 fa fa 00 fa
=>0x0c04800bc940: fa fa 00 fa fa fa 02 fa fa fa 00[fa]fa fa 00 00
  0x0c04800bc950: fa fa 02 fa fa fa 00 00 fa fa 00 00 fa fa 00 00
  0x0c04800bc960: fa fa 00 00 fa fa 00 04 fa fa 00 00 fa fa 00 04
  0x0c04800bc970: fa fa 00 00 fa fa 00 00 fa fa 00 00 fa fa 00 fa
  0x0c04800bc980: fa fa 00 fa fa fa 00 00 fa fa 02 fa fa fa 00 fa
  0x0c04800bc990: fa fa 00 fa fa fa 00 00 fa fa 00 00 fa fa 00 fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==3854934==ABORTING

Running locally after building with -g indicates that the failure happens on line 211, when index == 2 and regionsMinPt_.size() == 2
@iarspider
Copy link
Contributor Author

assign L1Trigger/Phase2L1GT

@cmsbuild
Copy link
Contributor

New categories assigned: l1,upgrade

@aloeliger,@epalencia,@Moanwar,@srimanob,@subirsarkar you have been requested to review this Pull request/Issue and eventually sign? Thanks

@cmsbuild
Copy link
Contributor

cms-bot internal usage

@cmsbuild
Copy link
Contributor

A new Issue was created by @iarspider.

@Dr15Jones, @antoniovilela, @makortel, @mandrenguyen, @rappoccio, @sextonkennedy, @smuzaffar can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

@iarspider
Copy link
Contributor Author

Log examples: RelVal 24834.0, RelVal 29634.0, RelVal 24834.911.

@mmusich
Copy link
Contributor

mmusich commented Jan 28, 2025

#46489 is likely the cause. @artlbv FYI

@artlbv
Copy link
Contributor

artlbv commented Jan 28, 2025

Thanks for pinging! FYI @HaarigerHarald @qvyz

Is this a concrete workflow where this appears? How can one reproduce this?
I’m surprised it did not show up in the PR tests.

@artlbv
Copy link
Contributor

artlbv commented Jan 28, 2025

To precise: I see the logs of the failed workflows but aren’t these also run for the PR tests? What is the difference?

@Dr15Jones
Copy link
Contributor

Dr15Jones commented Jan 28, 2025 •
edited
Loading

The ASAN builds (address sanitizer) link to a special library which tracks memory behavior. This library is rather slow so we do not include it in PR tests. This problem was only uncovered by the ASAN library.

To be more accurate, ASAN builds our code in a different way in order to utilize the library.

@mmusich
Copy link
Contributor

mmusich commented Jan 28, 2025

@artlbv

Is this a concrete workflow where this appears? How can one reproduce this?

cmsrel CMSSW_15_0_ASAN_X_2025-01-27-2300
cd CMSSW_15_0_ASAN_X_2025-01-27-2300/src
cmsenv
runTheMatrix.py -l 24834.0, 29634.0, 24834.911

this should reproduce.

@artlbv artlbv mentioned this issue Jan 29, 2025
Merged
@artlbv
Copy link
Contributor

artlbv commented Jan 29, 2025

The issue appeared when the number of eta regions (bounds) did not match e.g. the number of pt region thresholds.

Fixed in #47204

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
l1-pending pending-signatures upgrade-pending
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@iarspider @Dr15Jones @cmsbuild @artlbv @mmusich