[ROOT6 and ROOT628] Unit tests Miscellanea and PrimaryVertex failing
#42468
Comments
|
A new Issue was created by @aandvalenzuela Andrea Valenzuela. @Dr15Jones, @perrotta, @dpiparo, @rappoccio, @makortel, @smuzaffar can you please review it and eventually sign/assign? Thanks. cms-bot commands are listed here |
|
This issue seems related to the change to TCMalloc (#42440) since the failing cmsRun command runs fine when using |
|
assign alca |
|
New categories assigned: alca @perrotta,@consuegs,@francescobrivio,@saumyaphor4252,@tvami you have been requested to review this Pull request/Issue and eventually sign? Thanks |
|
@aandvalenzuela Would you be able to run the failing |
|
Sure, here it goes: |
|
Thanks! So the crash occurs inside TCMalloc, upon destruction of |
|
I ran the PrimaryVertex test in valgrind memcheck, and while there were a bunch of mismatched new/free errors (which I'll investigate), it didn't reproduce the crash or have a clear smoking gun. Also doesn't reproduce with 1 thread. Reproduces readily with 4 threads, stack trace below--as with the one from @aandvalenzuela it's in The only interesting part of the stack trace is thread 1: |
1 task
|
I built a debug version of tcmalloc, which turns on a bunch of internal consistency checks. It was about as slow as valgrind(!), didn't reproduce the problem and didn't find any errors. |
|
I believe the "mismatched new" warnings from valgrind are false positives, but I once I filtered those out I was left with 5 "Invalid free" errors similar to the ones in #40347: |
|
assign core |
|
New categories assigned: core @Dr15Jones,@smuzaffar,@makortel you have been requested to review this Pull request/Issue and eventually sign? Thanks |
|
@pcanal @vgvassilev Could you take a look of the valgrind report in #42468 (comment) ? |
This was referenced Aug 21, 2023
|
IIUC #42628 solved this particular issue.
All unit tests consistently succeeded. |
|
Given that the real fix has to happen in the Core ROOT s/w and the IBs failures are solved I am wondering if this issue could be closed. |
|
+core |
|
@cmsbuild, please close |
cmsbuild
added
core-approved
fully-signed
and removed
core-pending
pending-signatures
labels
|
This issue is fully signed and ready to be closed. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello,
Unit tests
MiscellaneaandPrimaryVertex(moduleAlignment/OfflineValidation) are failing in both ROOT6 and ROOT628 IBs with:at https://github.com/cms-sw/cmssw/blob/master/Alignment/OfflineValidation/test/testingScripts/test_unitMiscellanea.sh#L5
and
at https://github.com/cms-sw/cmssw/blob/master/Alignment/OfflineValidation/test/testingScripts/test_unitPrimaryVertex.sh#L6, respectively.
They started failing at the same time yesterday CMSSW_13_3_ROOT6_X_2023-08-02-2300. Today they failed again CMSSW_13_3_ROOT6_X_2023-08-03-2300.
I can reproduce the crashes locally.
The text was updated successfully, but these errors were encountered: