Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Race condition affecting (at least) TEnum::GetEnum #7093

Closed
pcanal opened this issue Jan 25, 2021 · 1 comment · Fixed by #7094 or #7095
Closed

Race condition affecting (at least) TEnum::GetEnum #7093

pcanal opened this issue Jan 25, 2021 · 1 comment · Fixed by #7094 or #7095
Assignees
@pcanal
Labels
affects:master affects:6.22 bug in:Core Libraries priority:critical
Milestone
6.24/00

Comments

@pcanal
Copy link
Member

pcanal commented Jan 25, 2021

In cms-sw/cmssw#32734 report what looks like a race condition.
The crashing thread is at:

#8  0x00002b8fb50c5252 in __assert_fail () from /lib64/libc.so.6
#9  0x00002b8fced4b768 in TCling::AutoLoad (this=0x2b8fb708cf80, cls=0x2b90068c9c80 "size_t", knowDictNotLoaded=false) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/slc7_amd64_gcc900/lcg/root/6.23.01-8e7a9c4406e195d049d6d9526c8e2ae6/root-6.23.01/core/metacling/src/TCling.cxx:6147
#10 0x00002b8fb38b5e09 in TEnum::GetEnum (enumName=0x2b90068c9c80 "size_t", sa=TEnum::kAutoload) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/slc7_amd64_gcc900/lcg/root/6.23.01-8e7a9c4406e195d049d6d9526c8e2ae6/root-6.23.01/core/meta/src/TEnum.cxx:274
#11 0x00002b8fb20c4ca8 in edm::TypeWithDict::byName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long) () from /cvmfs/cms-ib.cern.ch/nweek-02665/slc7_amd64_gcc900/cms/cmssw/CMSSW_11_3_ROOT6_X_2021-01-24-0000/lib/slc7_amd64_gcc900/libFWCoreReflection.so
#12 0x00002b8fb20c519d in edm::TypeWithDict::TypeWithDict(TMethodArg*, long) () from /cvmfs/cms-ib.cern.ch/nweek-02665/slc7_amd64_gcc900/cms/cmssw/CMSSW_11_3_ROOT6_X_2021-01-24-0000/lib/slc7_amd64_gcc900/libFWCoreReflection.so
#13 0x00002b8ffa1de7dd in reco::checkMethod(edm::FunctionWithDict const&, edm::TypeWithDict const&, std::vector<std::variant<signed char, unsigned char, short, unsigned short, int, unsigned int, long, unsigned long, double, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::variant<signed char, unsigned char, short, unsigned short, int, unsigned int, long, unsigned long, double, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, std::vector<std::variant<signed char, unsigned char, short, unsigned short, int, unsigned int, long, unsigned long, double, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::variant<signed char, unsigned char, short, unsigned short, int, unsigned int, long, unsigned long, double, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >&) () from /cvmfs/cms-ib.cern.ch/nweek-02665/slc7_amd64_gcc900/cms/cmssw/CMSSW_11_3_ROOT6_X_2021-01-24-0000/lib/slc7_amd64_gcc900/libCommonToolsUtils.so
#14 0x00002b8ffa1deeab in reco::findMethod(edm::TypeWithDict const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::variant<signed char, unsigned char, short, unsigned short, int, unsigned int, long, unsigned long, double, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::variant<signed char, unsigned char, short, unsigned short, int, unsigned int, long, unsigned long, double, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, std::vector<std::variant<signed char, unsigned char, short, unsigned short, int, unsigned int, long, unsigned long, double, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::variant<signed char, unsigned char, short, unsigned short, int, unsigned int, long, unsigned long, double, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >&, char const*, int&) () from /cvmfs/cms-ib.cern.ch/nweek-02665/slc7_amd64_gcc900/cms/cmssw/CMSSW_11_3_ROOT6_X_2021-01-24-0000/lib/slc7_amd64_gcc900/libCommonToolsUtils.so

while another thread is at:

#3  <signal handler called>
#4  0x00002b8fcf63cbbb in clang::serialization::MultiOnDiskHashTable<clang::serialization::reader::ASTDeclContextNameLookupTrait>::find(clang::DeclarationName const&) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#5  0x00002b8fcf63cfae in clang::ASTReader::FindExternalVisibleDeclsByName(clang::DeclContext const*, clang::DeclarationName) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#6  0x00002b8fcf758534 in clang::MultiplexExternalSemaSource::FindExternalVisibleDeclsByName(clang::DeclContext const*, clang::DeclarationName) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#7  0x00002b8fd105c4ee in clang::DeclContext::lookup(clang::DeclarationName) const () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#8  0x00002b8fcfb18f77 in LookupDirect(clang::Sema&, clang::LookupResult&, clang::DeclContext const*) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#9  0x00002b8fcfb19a5a in CppNamespaceLookup(clang::Sema&, clang::LookupResult&, clang::ASTContext&, clang::DeclContext*, (anonymous namespace)::UnqualUsingDirectiveSet&) [clone .isra.0] () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#10 0x00002b8fcfb1c4f5 in clang::Sema::CppLookupName(clang::LookupResult&, clang::Scope*) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#11 0x00002b8fcfb1ca1d in clang::Sema::LookupName(clang::LookupResult&, clang::Scope*, bool) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#12 0x00002b8fcf87cf29 in clang::Sema::HandleDeclarator(clang::Scope*, clang::Declarator&, llvm::MutableArrayRef<clang::TemplateParameterList*>) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#13 0x00002b8fcf87fc2c in clang::Sema::ActOnStartOfFunctionDef(clang::Scope*, clang::Declarator&, llvm::MutableArrayRef<clang::TemplateParameterList*>, clang::Sema::SkipBodyInfo*) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#14 0x00002b8fcf5e98cb in clang::Parser::ParseFunctionDefinition(clang::ParsingDeclarator&, clang::Parser::ParsedTemplateInfo const&, clang::Parser::LateParsedAttrList*) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#15 0x00002b8fcf566f2e in clang::Parser::ParseDeclGroup(clang::ParsingDeclSpec&, unsigned int, clang::SourceLocation*, clang::Parser::ForRangeInit*) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#16 0x00002b8fcf5e5fa1 in clang::Parser::ParseDeclOrFunctionDefInternal(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec&, clang::AccessSpecifier) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#17 0x00002b8fcf5ec950 in clang::Parser::ParseExternalDeclaration(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec*) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#18 0x00002b8fcf57b004 in clang::Parser::ParseLinkage(clang::ParsingDeclSpec&, unsigned int) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#19 0x00002b8fcf5e611e in clang::Parser::ParseDeclOrFunctionDefInternal(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec&, clang::AccessSpecifier) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#20 0x00002b8fcf5e64b1 in clang::Parser::ParseDeclarationOrFunctionDefinition(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec*, clang::AccessSpecifier) [clone .part.0] () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#21 0x00002b8fcf5ecf42 in clang::Parser::ParseExternalDeclaration(clang::Parser::ParsedAttributesWithRange&, clang::ParsingDeclSpec*) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#22 0x00002b8fcf5ed399 in clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#23 0x00002b8fcef7c0e6 in cling::IncrementalParser::ParseInternal(llvm::StringRef) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#24 0x00002b8fcef7dfed in cling::IncrementalParser::Compile(llvm::StringRef, cling::CompilationOptions const&) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#25 0x00002b8fceee6852 in cling::Interpreter::DeclareInternal(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cling::CompilationOptions const&, cling::Transaction**) const () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#26 0x00002b8fceee69fa in cling::Interpreter::declare(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, cling::Transaction**) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#27 0x00002b8fceee79e1 in cling::Interpreter::DeclareCFunction(llvm::StringRef, llvm::StringRef, bool, cling::Transaction*&) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#28 0x00002b8fceeeaa2d in cling::Interpreter::compileFunction(llvm::StringRef, llvm::StringRef, bool, bool) () from /cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_11_3_ROOT6_X_2021-01-24-2300/external/slc7_amd64_gcc900/lib/libCling.so
#29 0x00002b8fcee7556b in TClingCallFunc::compile_wrapper (this=0x2b913aff4c00, wrapper_name=..., wrapper=..., withAccessControl=true) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/slc7_amd64_gcc900/lcg/root/6.23.01-8e7a9c4406e195d049d6d9526c8e2ae6/root-6.23.01/core/metacling/src/TClingCallFunc.cxx:267
#30 0x00002b8fcee78390 in TClingCallFunc::make_wrapper (this=0x2b913aff4c00) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/slc7_amd64_gcc900/lcg/root/6.23.01-8e7a9c4406e195d049d6d9526c8e2ae6/root-6.23.01/core/metacling/src/TClingCallFunc.cxx:1117
#31 0x00002b8fcee7d5a8 in TClingCallFunc::IFacePtr (this=0x2b913aff4c00) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/slc7_amd64_gcc900/lcg/root/6.23.01-8e7a9c4406e195d049d6d9526c8e2ae6/root-6.23.01/core/metacling/src/TClingCallFunc.cxx:2301
#32 0x00002b8fced52d75 in TCling::CallFunc_IFacePtr (this=0x2b8fb708cf80, func=0x2b913aff4c00) at /data/cmsbld/jenkins/workspace/build-any-ib/w/BUILD/slc7_amd64_gcc900/lcg/root/6.23.01-8e7a9c4406e195d049d6d9526c8e2ae6/root-6.23.01/core/metacling/src/TCling.cxx:7882
#33 0x00002b8fb20bc427 in edm::FunctionWithDict::FunctionWithDict(TMethod*) () from /cvmfs/cms-ib.cern.ch/nweek-02665/slc7_amd64_gcc900/cms/cmssw/CMSSW_11_3_ROOT6_X_2021-01-24-0000/lib/slc7_amd64_gcc900/libFWCoreReflection.so
#34 0x00002b8fb20c619a in edm::TypeWithDict::functionMemberByName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool) const () from /cvmfs/cms-ib.cern.ch/nweek-02665/slc7_amd64_gcc900/cms/cmssw/CMSSW_11_3_ROOT6_X_2021-01-24-0000/lib/slc7_amd64_gcc900/libFWCoreReflection.so
#35 0x00002b8ffa1dee7e in reco::findMethod(edm::TypeWithDict const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::variant<signed char, unsigned char, short, unsigned short, int, unsigned int, long, unsigned long, double, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::variant<signed char, unsigned char, short, unsigned short, int, unsigned int, long, unsigned long, double, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, std::vector<std::variant<signed char, unsigned char, short, unsigned short, int, unsigned int, long, unsigned long, double, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::variant<signed char, unsigned char, short, unsigned short, int, unsigned int, long, unsigned long, double, float, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >&, char const*, int&) () from /cvmfs/cms-ib.cern.ch/nweek-02665/slc7_amd64_gcc900/cms/cmssw/CMSSW_11_3_ROOT6_X_2021-01-24-0000/lib/slc7_amd64_gcc900/libCommonToolsUtils.so
@pcanal pcanal added the bug label Jan 25, 2021
@pcanal pcanal self-assigned this Jan 25, 2021
@pcanal pcanal added this to the 6.24/00 milestone Jan 25, 2021
@pcanal
Copy link
Member Author

pcanal commented Jan 25, 2021

The assert is in TCling::AutoLoad:

   if (!IsClassAutoLoadingEnabled()) {
...
   }

   assert(IsClassAutoLoadingEnabled() && "Calling when AutoLoading is off!");

which does indicate that the state changed between the if statement and the assert

pcanal added a commit to pcanal/root that referenced this issue Jan 25, 2021
@pcanal
Fix root-project#7093: race condition tripping TEnum::GetEnum
a20590d
pcanal added a commit to pcanal/root that referenced this issue Jan 25, 2021
@pcanal
Fix root-project#7093: race condition tripping TEnum::GetEnum
83603da
@pcanal pcanal mentioned this issue Jan 25, 2021
Merged
@pcanal pcanal linked a pull request Jan 25, 2021 that will close this issue
Fix #7093: race condition tripping TEnum::GetEnum #7094
Merged
pcanal added a commit to pcanal/root that referenced this issue Jan 25, 2021
@pcanal
Fix root-project#7093: race condition tripping TEnum::GetEnum
0f69f91
@pcanal pcanal mentioned this issue Jan 25, 2021
Merged
pcanal added a commit that referenced this issue Feb 24, 2021
@pcanal
Fix #7093: race condition tripping TEnum::GetEnum
cbfd8e4
@pcanal pcanal linked a pull request Feb 24, 2021 that will close this issue
Fix #7093: race condition tripping TEnum::GetEnum #7095
Merged
pcanal added a commit that referenced this issue Feb 25, 2021
@pcanal
Fix #7093: race condition tripping TEnum::GetEnum
6cf1d15
@mrodozov mrodozov mentioned this issue Feb 25, 2021
Merged
nicknagi pushed a commit to nicknagi/root that referenced this issue Mar 30, 2021
@pcanal @nicknagi
Fix root-project#7093: race condition tripping TEnum::GetEnum
e4198b9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pcanal pcanal

Labels
affects:master affects:6.22 bug in:Core Libraries priority:critical
Projects
None yet
Milestone
6.24/00
1 participant
@pcanal