Getting a crash while reading a Root Tree

[bsunanda]

===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
#0  0x00007f36ce4ec60c in waitpid () from /lib64/libc.so.6
#1  0x00007f36ce469f62 in do_system () from /lib64/libc.so.6
#2  0x00007f36d089c3bc in TUnixSystem::StackTrace() () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libCore.so
#3  0x00007f36d0899ab5 in TUnixSystem::DispatchSignals(ESignals) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libCore.so
#4  <signal handler called>
#5  0x00007f36d03eeb1f in TStreamerInfoActions::VectorLooper::ReadCollectionBool(TBuffer&, void*, TStreamerInfoActions::TConfiguration const*) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libRIO.so
#6  0x00007f36d02af085 in TBufferFile::ApplySequence(TStreamerInfoActions::TActionSequence const&, void*) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libRIO.so
#7  0x00007f36cf9d57bc in TBranchElement::ReadLeavesMakeClass(TBuffer&) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libTree.so
#8  0x00007f36cf9ba029 in TBranch::GetEntry(long long, int) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libTree.so
#9  0x00007f36cf9d4025 in TBranchElement::GetEntry(long long, int) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libTree.so
#10 0x00007f36cfa2f4c0 in TTree::GetEntry(long long, int) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libTree.so
#11 0x000000000040ffbd in HBHEMuonOfflineAnalyzer::Loop() ()
#12 0x000000000041960f in main ()
===========================================================

The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum https://root.cern/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at https://root.cern/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5  0x00007f36d03eeb1f in TStreamerInfoActions::VectorLooper::ReadCollectionBool(TBuffer&, void*, TStreamerInfoActions::TConfiguration const*) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libRIO.so
#6  0x00007f36d02af085 in TBufferFile::ApplySequence(TStreamerInfoActions::TActionSequence const&, void*) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libRIO.so
#7  0x00007f36cf9d57bc in TBranchElement::ReadLeavesMakeClass(TBuffer&) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libTree.so
#8  0x00007f36cf9ba029 in TBranch::GetEntry(long long, int) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libTree.so
#9  0x00007f36cf9d4025 in TBranchElement::GetEntry(long long, int) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libTree.so
#10 0x00007f36cfa2f4c0 in TTree::GetEntry(long long, int) () from /cvmfs/cms.cern.ch/slc7_amd64_gcc11/cms/cmssw/CMSSW_13_0_0_pre4/external/slc7_amd64_gcc11/bin/../../../../../../../slc7_amd64_gcc11/lcg/root/6.26.07-3c8fea8a0ce2aca35570ac22afed05d0/lib/libTree.so
#11 0x000000000040ffbd in HBHEMuonOfflineAnalyzer::Loop() ()
#12 0x000000000041960f in main ()
===========================================================


[1]  + Exit 139                      ./OfflineMain.exe /eos/cms/store/group/dpg_hcal/comm_hcal/Harjot/Jupyter_Notebooks/Validation_PU.root mc22PU.root 0 3 >& mc22PU.log

[vepadulano]

Hi @bsunanda ,
Can you provide a simple reproducer of your case? Together with some information about your system (how you installed ROOT etc., compiler version etc.)

[pcanal]

Alternatively you can run valgrind to get more information on this kind of failure:

valgrind --suppressions=$ROOTSYS/etc/valgrind-root.supp --log-file=val.01.log myexecutable myargs

[bsunanda]

Thanks Philippe - I shall try that

…

________________________________ From: Philippe Canal ***@***.*** Sent: 27 February 2023 18:50 To: root-project/root Cc: Sunanda Banerjee; Mention Subject: Re: [root-project/root] Getting a crash while reading a Root Tree (Issue #12370) Alternatively you can run valgrind to get more information on this kind of failure: valgrind --suppressions=$ROOTSYS/etc/valgrind-root.supp --log-file=val.01.log myexecutable myargs — Reply to this email directly, view it on GitHub<#12370 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABGMZOSBSY5NDNIO7K6JZEDWZTSPBANCNFSM6AAAAAAVFHT3OU>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[bsunanda]

Dear Philippe Sorry for late response. The crash depends on the input file. I landed with one file and here is the log file which came from running valgrind. I cannot find out the real reason from this. Could you point out where I should look into this?? Thanks and regards Sunanda

…

________________________________ From: Sunanda Banerjee ***@***.***> Sent: 28 February 2023 14:10 To: root-project/root ***@***.***>; root-project/root ***@***.***> Cc: Mention ***@***.***> Subject: RE: [root-project/root] Getting a crash while reading a Root Tree (Issue #12370) Thanks Philippe - I shall try that

________________________________ From: Philippe Canal ***@***.*** Sent: 27 February 2023 18:50 To: root-project/root Cc: Sunanda Banerjee; Mention Subject: Re: [root-project/root] Getting a crash while reading a Root Tree (Issue #12370) Alternatively you can run valgrind to get more information on this kind of failure: valgrind --suppressions=$ROOTSYS/etc/valgrind-root.supp --log-file=val.01.log myexecutable myargs — Reply to this email directly, view it on GitHub<#12370 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABGMZOSBSY5NDNIO7K6JZEDWZTSPBANCNFSM6AAAAAAVFHT3OU>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[Axel-Naumann]

here is the log file which came from running valgrind

I don't find this; could you attach it at #12370 please?

[bsunanda]

val.01.log

[pcanal]

The relevant part:

==9375== Conditional jump or move depends on uninitialised value(s)
==9375==    at 0x44AFDA: CalibMonitor::correctEnergy(double&, long long const&) (in /afs/cern.ch/work/s/sunanda/public/CMSSW_12_4_6/relval/d22gm/CalibMain.exe)
==9375==    by 0x4496BB: CalibMonitor::goodTrack(double&, double&, long long const&, bool) (in /afs/cern.ch/work/s/sunanda/public/CMSSW_12_4_6/relval/d22gm/CalibMain.exe)
==9375==    by 0x4467E8: CalibMonitor::Loop(long long) (in /afs/cern.ch/work/s/sunanda/public/CMSSW_12_4_6/relval/d22gm/CalibMain.exe)
==9375==    by 0x4664CA: main (in /afs/cern.ch/work/s/sunanda/public/CMSSW_12_4_6/relval/d22gm/CalibMain.exe)
==9375== 
==9375== Invalid read of size 4
==9375==    at 0x44B009: CalibMonitor::correctEnergy(double&, long long const&) (in /afs/cern.ch/work/s/sunanda/public/CMSSW_12_4_6/relval/d22gm/CalibMain.exe)
==9375==    by 0x4496BB: CalibMonitor::goodTrack(double&, double&, long long const&, bool) (in /afs/cern.ch/work/s/sunanda/public/CMSSW_12_4_6/relval/d22gm/CalibMain.exe)
==9375==    by 0x4467E8: CalibMonitor::Loop(long long) (in /afs/cern.ch/work/s/sunanda/public/CMSSW_12_4_6/relval/d22gm/CalibMain.exe)
==9375==    by 0x4664CA: main (in /afs/cern.ch/work/s/sunanda/public/CMSSW_12_4_6/relval/d22gm/CalibMain.exe)
==9375==  Address 0x0 is not stack'd, malloc'd or (recently) free'd

[pcanal]

The crash depends on the input file. I landed with one file and here is the log file which came from running valgrind.

Does the crash with valgrind have the same symptoms as the original problem (i.e. seg fault in TStreamerInfoActions::VectorLooper::ReadCollectionBool)?

[dpiparo]

Dear @bsunanda , I am sorry to read that problem hit you and to come back to this matter now. For me the issue cannot be reproduced, however, I invite you to re-open a similar ticket with the necessary context in order for us to fix any potential problem affecting CMS simulation.