[G4VECGEOM] Update clhep to 2.4.6.0, vecgeom to 1.2.1

[iarspider]

Closes #8209, #8210.

[cmsbuild]

A new Pull Request was created by @iarspider for branch IB/CMSSW_12_6_X/g11_g4_vecgeom.

@cmsbuild, @smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.
cms-bot commands are listed here

[smuzaffar]

@iarspider , base branch for this PR is not correct

[civanch]

Is base IB OK?

[smuzaffar]

test parameters:

• full_cmssw = true

[smuzaffar]

G4 Vengeom IB is bit old , let me re-build a new one which we can use for testing

[smuzaffar]

@civanch , looks like tests are failing due to vecgeom update here

#5  0x00002adf1c01e5a5 in vecgeom::cxx::CommonUnplacedVolumeImplHelper<vecgeom::cxx::PolyhedronImplementation<(EInnerRadii)0, (EPhiCutout)0>, vecgeom::cxx::VUnplacedVolume>::SafetyToIn(vecgeom::cxx::Vector3D<double> const&) const () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8211/29458/CMSSW_13_0_G4VECGEOM_X_2022-12-04-2300/biglib/el8_amd64_gcc11/pluginSimulation.so
#6  0x00002adf1c021dc7 in G4UAdapter<vecgeom::cxx::UnplacedPolyhedron>::DistanceToIn(CLHEP::Hep3Vector const&) const () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8211/29458/CMSSW_13_0_G4VECGEOM_X_2022-12-04-2300/biglib/el8_amd64_gcc11/pluginSimulation.so
#7  0x00002adf1c0e6274 in G4VoxelNavigation::ComputeStep(CLHEP::Hep3Vector const&, CLHEP::Hep3Vector const&, double, double&, G4NavigationHistory&, bool&, CLHEP::Hep3Vector&, bool&, bool&, G4VPhysicalVolume**, int&) () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8211/29458/CMSSW_13_0_G4VECGEOM_X_2022-12-04-2300/biglib/el8_amd64_gcc11/pluginSimulation.so
#8  0x00002adf1bd00001 in G4Navigator::ComputeStep(CLHEP::Hep3Vector const&, CLHEP::Hep3Vector const&, double, double&) () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8211/29458/CMSSW_13_0_G4VECGEOM_X_2022-12-04-2300/biglib/el8_amd64_gcc11/pluginSimulation.so
#9  0x00002adf1bf1733a in G4Transportation::AlongStepGetPhysicalInteractionLength(G4Track const&, double, double, double&, G4GPILSelection*) () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8211/29458/CMSSW_13_0_G4VECGEOM_X_2022-12-04-2300/biglib/el8_amd64_gcc11/pluginSimulation.so
#10 0x00002adf1c554bf2 in G4SteppingManager::DefinePhysicalStepLength() () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8211/29458/CMSSW_13_0_G4VECGEOM_X_2022-12-04-2300/biglib/el8_amd64_gcc11/pluginSimulation.so
#11 0x00002adf1c555d00 in G4SteppingManager::Stepping() () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8211/29458/CMSSW_13_0_G4VECGEOM_X_2022-12-04-2300/biglib/el8_amd64_gcc11/pluginSimulation.so
#12 0x00002adf1bf0f27b in G4TrackingManager::ProcessOneTrack(G4Track*) () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8211/29458/CMSSW_13_0_G4VECGEOM_X_2022-12-04-2300/biglib/el8_amd64_gcc11/pluginSimulation.so
#13 0x00002adf1bba0f79 in G4EventManager::DoProcessing(G4Event*) () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8211/29458/CMSSW_13_0_G4VECGEOM_X_2022-12-04-2300/biglib/el8_amd64_gcc11/pluginSimulation.so
#14 0x00002adf1b80ae60 in RunManagerMTWorker::produce(edm::Event const&, edm::EventSetup const&, RunManagerMT&) () from /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8211/29458/CMSSW_13_0_G4VECGEOM_X_2022-12-04-2300/biglib/el8_amd64_gcc11/pluginSimulation.so

[civanch]

@smuzaffar , I make a ticket to VecGeom and Jonas just report that -O2 with gcc11 resolve the issue. So, it is a very specific VecGeom problem (code seems to be correct but during optimisation non-initialized vector is crashing). Until it s understood -O2 should be used for VecGeom.

[cmsbuild]

Pull request #8211 was updated.

[smuzaffar]

please test
ok, I have updated dd4hep, vecgeom and geant4 to use -O2 -DNDEBUG

[cmsbuild]

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-eac510/29481/summary.html
COMMIT: 2049e79
CMSSW: CMSSW_13_0_G4VECGEOM_X_2022-12-04-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8211/29481/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation error when building:

+ '[' -f /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc11/./etc/profile.d/init.sh ']'
+ export BOOST_ROOT
+ CMAKE_ARGS='-DCMAKE_INSTALL_PREFIX='\''/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/BUILDROOT/311365d3eaa0f99a8fdd92609ab78336/opt/cmssw/el8_amd64_gcc11/external/dd4hep/v01-23x-311365d3eaa0f99a8fdd92609ab78336'\''       -DBoost_NO_BOOST_CMAKE=ON       -DDD4HEP_USE_XERCESC=ON       -DXERCESC_ROOT_DIR=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc11/external/xerces-c/3.1.3-96261f23c7d6fbfb7d59be544bd882f3       -DDD4HEP_USE_PYROOT=ON       -DCMAKE_CXX_STANDARD=17       -DCMAKE_BUILD_TYPE=Release       -DCMAKE_CXX_FLAGS_RELEASE=-O2'
+ '-DNDEBUG       -DDD4HEP_USE_GEANT4_UNITS=ON       -DCMAKE_PREFIX_PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc11/external/clhep/2.4.6.0-80baf510802496a74335c6030dc0c72b;/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc11/external/xerces-c/3.1.3-96261f23c7d6fbfb7d59be544bd882f3'
/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.C4cLqv: line 40: -DNDEBUG       -DDD4HEP_USE_GEANT4_UNITS=ON       -DCMAKE_PREFIX_PATH=/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc11/external/clhep/2.4.6.0-80baf510802496a74335c6030dc0c72b;/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/el8_amd64_gcc11/external/xerces-c/3.1.3-96261f23c7d6fbfb7d59be544bd882f3: No such file or directory
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.C4cLqv (%build)


RPM build errors:
line 37: It's not recommended to have unversioned Obsoletes: Obsoletes: external+dd4hep+v01-23x-311365d3eaa0f99a8fdd92609ab78336
Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.C4cLqv (%build)


[smuzaffar]

please test

[cmsbuild]

Pull request #8211 was updated.

[aandvalenzuela]

It seems comparison is not going to report the results back since it is failing at:

09:08:17 + mv /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-eac510/1000.0_RunMinBias2011A /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-eac510/1000.0_RunMinBias2011A
09:08:17 mv: cannot move '/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-eac510/1000.0_RunMinBias2011A' to a subdirectory of itself, '/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-eac510/1000.0_RunMinBias2011A/1000.0_RunMinBias2011A'

I am writing a small fix for this

[smuzaffar]

thanks @aandvalenzuela for checking this. cms-sw/cms-bot@25334a8 change should fix the issue. I have restarted the comparison job now

[aandvalenzuela]

I first added this extra check https://github.com/cms-sw/cms-bot/blob/master/pr_testing/run-pr-comparisons#L126, should I keep it @smuzaffar?

[smuzaffar]

thanks for typo fix at cms-sw/cms-bot@06cf850 :-)
No need to have https://github.com/cms-sw/cms-bot/blob/master/pr_testing/run-pr-comparisons#L126 but it is harmless, so no need to remove it

[hahnjo]

FWIW I opened https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108008, let's see what the GCC people think

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-eac510/29485/summary.html
COMMIT: 7efc76b
CMSSW: CMSSW_13_0_G4VECGEOM_X_2022-12-04-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8211/29485/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @bsunanda Run-sim137 Correct the code in the analysis code of Simulation cmssw#40227
• @vlimant short name also in the final report cmssw#40235
• @nurfikri89 [JMENanoAOD] Setup DQM and increase precision of PileUp ID & QGL input variables cmssw#40232
• @smuzaffar [GEOMETRY-UPGRADE] [LLVM14] Apply code checks cmssw#40188
• @AdrianoDee Avoid CPE Fast ES Producers Renaming cmssw#40215
• @bsunanda Phase2-hgx333 Bug fix for invalid DetID declaration for SimHits n V16/V17 geometry versions of HGCal cmssw#40208
• @swertz Nano: fix and additions to collection matching, remove cross-cleaning cmssw#40162
• @bsunanda Phase2-hgx331 Transfer HGCal TB related stuff to Geometry/HGCalTestBeam and modify these accordingly cmssw#40160
• @bsunanda Run3-gex145 Change getByLabel to use tokens in Alignment/OfflineValidation cmssw#40051
• @Dr15Jones Revived being able to run modules inside a mixing module cmssw#40133
• @cms-tsg-storm Fix 2022v15 frozen HLT menu for PixelCPEFast changes cmssw#40243
• @gartung Update igprof to handle unrecognosed prologue for movq instruction with XMM registers #8215

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-eac510/29485/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-eac510/29485/git-merge-result

Comparison Summary

Summary:

• No significant changes to the logs found
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 62397 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3421214
• DQMHistoTests: Total failures: 116323
• DQMHistoTests: Total nulls: 247
• DQMHistoTests: Total successes: 3304622
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -11.074000000000002 KiB( 47 files compared)
• DQMHistoSizes: changed ( 10224.0 ): 0.469 KiB SiStrip/MechanicalView
• DQMHistoSizes: changed ( 11634.0,... ): -0.774 KiB Physics/NanoAODDQM
• DQMHistoSizes: changed ( 11834.0 ): 0.663 KiB SiStrip/MechanicalView
• DQMHistoSizes: changed ( 13234.0,... ): -0.454 KiB Physics/NanoAODDQM
• DQMHistoSizes: changed ( 250202.181 ): -0.053 KiB SiStrip/MechanicalView
• DQMHistoSizes: changed ( 25202.0 ): -0.416 KiB SiStrip/MechanicalView
• DQMHistoSizes: changed ( 7.3 ): -6.948 KiB SiStrip/MechanicalView
• DQMHistoSizes: changed ( 8.0 ): 1.537 KiB SiStrip/MechanicalView
• Checked 206 log files, 158 edm output root files, 48 DQM output files
• TriggerResults: found differences in 15 / 46 workflows

[smuzaffar]

@civanch , this looks good now. I am merging it for next G4 Vecgeom IB

[civanch]

@smuzaffar , thanks, it is confirmed by gcc that there is problem in -O3. When new IB will be available will try to check CPU.

[smuzaffar]

@civanch , CMSSW_13_0_G4VECGEOM_X_2022-12-07-2300 and CMSSW_13_0_GEANT4_X_2022-12-07-2300 IBs are available

[civanch]

I have check CPU. It turned out that this switch from -O3 to -O2 provide CPU penalty 2.2% for MinBias and 4.9% for ttbar. So, we need -O3 indeed. The solution may be work around for problematic class of VecGeom

[hahnjo]

Yes, I can also measure the performance penalty. We may try working around the problem (I can prepare a patch), or we can try compiling Geant4 with -O3 and only VecGeom with -O2 (as we did in the past).

[hahnjo]

FWIW the bug in GCC should be fixed, cf. https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107323 and https://gcc.gnu.org/bugzilla/show_bug.cgi?id=108008. Once there is an updated compiler with the fixes, we can consider going back to -O3 also for VecGeom.