Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update vecgeom to 1.1.20 #7735

Merged
merged 21 commits into from
May 4, 2022

Conversation

iarspider
Copy link
Contributor

@iarspider iarspider commented Mar 31, 2022

Closes #7734

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @iarspider for branch IB/CMSSW_12_4_X/geant4_vecgeom.

@cmsbuild, @smuzaffar, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @qliphy you are the release manager for this.
cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

Pull request #7735 was updated.

@iarspider
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fa11c2/23555/summary.html
COMMIT: 8adc016
CMSSW: CMSSW_12_4_G4VECGEOM_X_2022-03-30-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7735/23555/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation error when building:

been installed.


-- Configuring incomplete, errors occurred!
See also "/data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/slc7_amd64_gcc10/external/vecgeom/v1.1.20-f0f1f54e8f4aeb64f5135f9f30b3c429/build/CMakeFiles/CMakeOutput.log".
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.UqpPce (%build)


RPM build errors:
Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.UqpPce (%build)



@smuzaffar
Copy link
Contributor

@iarspider , looks like we need veccore?

@civanch
Copy link
Contributor

civanch commented Mar 31, 2022

geant4.spec should not be modified: -DGEANT4_BUILD_BUILTIN_BACKTRACE=OFF is useful, because it means all exceptions will be handled by CMSSW, no extra backtrace from Geant4.

-DGEANT4_USE_USOLIDS="CONS;POLYCONE;POLYHEDRA" better to remove in order do not mess up new version of VecGeom and this extra configuration. We may use it later but now we need to have a branch which can run big statistics smoothly and be minimally differ from vecgeom 1.1.18.

This reverts commit 8adc016.
@cmsbuild
Copy link
Contributor

Pull request #7735 was updated.

@iarspider
Copy link
Contributor Author

@iarspider , looks like we need veccore?

Yes, looks like so. I will look into it.

@civanch
Copy link
Contributor

civanch commented Mar 31, 2022

VecCore 0.8.0 is needed. Gabriele suggest following cmake options:

-DBACKEND=YOUR-CHOSEN-BACKEND [Vc|Scalar]
-DVECGEOM_VECTOR=YOUR-CHOSEN-VECTOR-INSTRUCTION-SET [native|sse2|avx2]
-DVecCore_DIR=[YOUR--VECCORE-INSTALL-PATH]/lib/cmake/VecCore
-DCMAKE_PREFIX_PATH=[YOUR-VC-INSTALL-PATH]
-DDATA_DOWNLOAD=OFF
-DGEANT4=OFF
-DCMAKE_BUILD_TYPE=Release
-DCMAKE_INSTALL_PREFIX=[YOUR-VECGEOM-INSTALL-PATH]

He said that the explicit path to VECCORE is needed.

@cmsbuild
Copy link
Contributor

Pull request #7735 was updated.

@iarspider
Copy link
Contributor Author

please test

@cmsbuild
Copy link
Contributor

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fa11c2/23566/summary.html
COMMIT: 4dd9086
CMSSW: CMSSW_12_4_G4VECGEOM_X_2022-03-30-1100/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7735/23566/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation warning when building: See details on the summary page.

@cmsbuild
Copy link
Contributor

cmsbuild commented May 2, 2022

Pull request #7735 was updated.

@cmsbuild
Copy link
Contributor

cmsbuild commented May 2, 2022

-1

Failed Tests: UnitTests RelVals AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fa11c2/24384/summary.html
COMMIT: 42f5258
CMSSW: CMSSW_12_4_G4VECGEOM_X_2022-04-28-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/7735/24384/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fa11c2/24384/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fa11c2/24384/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test trackerMaterialAnalysisPlots had ERRORS

RelVals

  • 9.09.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step1_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log
  • 25.025.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step1_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log
  • 1306.01306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step1_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log
Expand to see more relval errors ...

AddOn Tests

  • hlt_mc_FakecmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --fileout file:RelVal_Raw_Fake_MC.root : FAILED - time: date Mon May 2 18:04:18 2022-date Mon May 2 18:01:26 2022 s - exit: 35584
  • hlt_mc_Fake
----- Begin Fatal Exception 02-May-2022 18:03:01 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   Additional Info:
      [a] Input file file:RelVal_Raw_Fake_MC.root could not be opened.
      [b] Fatal Root Error: @SUB=TStorageFactoryFile::ReadBuffer
read from Storage::xread returned 256. Asked to read n bytes: 300 from offset: 0 with file size: 256

----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 02-May-2022 18:04:18 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   Additional Info:
      [a] Input file file:RelVal_Raw_Fake_MC.root could not be opened.
      [b] Fatal Root Error: @SUB=TStorageFactoryFile::ReadBuffer
read from Storage::xread returned 256. Asked to read n bytes: 300 from offset: 0 with file size: 256

----- End Fatal Exception -------------------------------------------------
Expand to see more addon errors ...

@smuzaffar
Copy link
Contributor

@civanch , this time I build vecgeom with buildin veccore but it still fails with the same error

@civanch
Copy link
Contributor

civanch commented May 3, 2022

@smuzaffar , may we have some bad interference with dd4hep build? The recent does not explicitly depend on vecgeom, only on geant4.

@cmsbuild
Copy link
Contributor

cmsbuild commented May 3, 2022

Pull request #7735 was updated.

@cmsbuild
Copy link
Contributor

cmsbuild commented May 3, 2022

Pull request #7735 was updated.

@smuzaffar
Copy link
Contributor

test parameters:

  • full_cmssw = true

@smuzaffar
Copy link
Contributor

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented May 4, 2022

-1

Failed Tests: UnitTests RelVals AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fa11c2/24427/summary.html
COMMIT: c43fa21
CMSSW: CMSSW_12_4_G4VECGEOM_X_2022-05-02-2300/slc7_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/7735/24427/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fa11c2/24427/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-fa11c2/24427/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test trackerMaterialAnalysisPlots had ERRORS
---> test testTauEmbeddingProducers had ERRORS

RelVals

  • 9.09.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step1_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log
  • 25.025.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step1_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log
  • 1306.01306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step1_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log
Expand to see more relval errors ...

AddOn Tests

  • hlt_mc_FakecmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi -s GEN,SIM,DIGI,L1,DIGI2RAW --mc --scenario=pp -n 10 --conditions auto:run1_mc_Fake --relval 9000,50 --datatier "GEN-SIM-RAW" --eventcontent RAWSIM --customise=HLTrigger/Configuration/CustomConfigs.L1T --fileout file:RelVal_Raw_Fake_MC.root : FAILED - time: date Wed May 4 01:57:29 2022-date Wed May 4 01:54:11 2022 s - exit: 35584
  • hlt_mc_Fake
----- Begin Fatal Exception 04-May-2022 01:56:41 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   Additional Info:
      [a] Input file file:RelVal_Raw_Fake_MC.root could not be opened.
      [b] Fatal Root Error: @SUB=TStorageFactoryFile::ReadBuffer
read from Storage::xread returned 256. Asked to read n bytes: 300 from offset: 0 with file size: 256

----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 04-May-2022 01:57:29 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootInputFileSequence::initTheFile()
   Additional Info:
      [a] Input file file:RelVal_Raw_Fake_MC.root could not be opened.
      [b] Fatal Root Error: @SUB=TStorageFactoryFile::ReadBuffer
read from Storage::xread returned 256. Asked to read n bytes: 300 from offset: 0 with file size: 256

----- End Fatal Exception -------------------------------------------------
Expand to see more addon errors ...

@smuzaffar
Copy link
Contributor

let merge this for G4VECGEOM Ib and build an IB to debug it. We can revert it back if needed

@smuzaffar smuzaffar merged commit 64525f3 into IB/CMSSW_12_4_X/geant4_vecgeom May 4, 2022
@smuzaffar
Copy link
Contributor

@civanch , if I build geant4 and vecgeom in Debug mode then workflow 1.0 does not fail. In Release mode it fails and valgrind shows

/afs/cern.ch/user/m/muzaffar/public/geant4-valgrind.log
==23410== Invalid read of size 8
==23410==    at 0x3FD779BA: vecgeom::cxx::CommonUnplacedVolumeImplHelper<vecgeom::cxx::PolyhedronImplementation<(EInnerRadii)0, (EPhiCutout)0>, vecgeom::cxx::VUnplacedVolume>::SafetyToIn(vecgeom::cxx::Vector3D<double> const&) const (in /cvmfs/cms-ib.cern.ch/nweek-02731/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_G4VECGEOM_X_2022-05-04-1100/biglib/slc7_amd64_gcc10/pluginSimulation.so)
==23410==    by 0x3FD7B2A6: G4UAdapter<vecgeom::cxx::UnplacedPolyhedron>::DistanceToIn(CLHEP::Hep3Vector const&) const (in /cvmfs/cms-ib.cern.ch/nweek-02731/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_G4VECGEOM_X_2022-05-04-1100/biglib/slc7_amd64_gcc10/pluginSimulation.so)
==23410==    by 0x3FE45C48: G4VoxelNavigation::ComputeStep(CLHEP::Hep3Vector const&, CLHEP::Hep3Vector const&, double, double&, G4NavigationHistory&, bool&, CLHEP::Hep3Vector&, bool&, bool&, G4VPhysicalVolume**, int&) (in /cvmfs/cms-ib.cern.ch/nweek-02731/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_G4VECGEOM_X_2022-05-04-1100/biglib/slc7_amd64_gcc10/pluginSimulation.so)
==23410==    by 0x3FA5A51D: G4Navigator::ComputeStep(CLHEP::Hep3Vector const&, CLHEP::Hep3Vector const&, double, double&) (in /cvmfs/cms-ib.cern.ch/nweek-02731/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_G4VECGEOM_X_2022-05-04-1100/biglib/slc7_amd64_gcc10/pluginSimulation.so)
==23410==    by 0x3FC6F953: G4Transportation::AlongStepGetPhysicalInteractionLength(G4Track const&, double, double, double&, G4GPILSelection*) (in /cvmfs/cms-ib.cern.ch/nweek-02731/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_G4VECGEOM_X_2022-05-04-1100/biglib/slc7_amd64_gcc10/pluginSimulation.so)
==23410==    by 0x401BD7E3: G4SteppingManager::DefinePhysicalStepLength() (in /cvmfs/cms-ib.cern.ch/nweek-02731/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_G4VECGEOM_X_2022-05-04-1100/biglib/slc7_amd64_gcc10/pluginSimulation.so)
==23410==    by 0x401BC10F: G4SteppingManager::Stepping() (in /cvmfs/cms-ib.cern.ch/nweek-02731/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_G4VECGEOM_X_2022-05-04-1100/biglib/slc7_amd64_gcc10/pluginSimulation.so)
==23410==    by 0x3FC66F87: G4TrackingManager::ProcessOneTrack(G4Track*) (in /cvmfs/cms-ib.cern.ch/nweek-02731/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_G4VECGEOM_X_2022-05-04-1100/biglib/slc7_amd64_gcc10/pluginSimulation.so)
==23410==    by 0x3F942408: G4EventManager::DoProcessing(G4Event*) (in /cvmfs/cms-ib.cern.ch/nweek-02731/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_G4VECGEOM_X_2022-05-04-1100/biglib/slc7_amd64_gcc10/pluginSimulation.so)
==23410==    by 0x3F5C946D: RunManagerMTWorker::produce(edm::Event const&, edm::EventSetup const&, RunManagerMT&) (in /cvmfs/cms-ib.cern.ch/nweek-02731/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_G4VECGEOM_X_2022-05-04-1100/biglib/slc7_amd64_gcc10/pluginSimulation.so)
==23410==    by 0x3F5DF634: omt::ThreadHandoff::Functor<OscarMTProducer::produce(edm::Event&, edm::EventSetup const&)::{lambda()#1}>::execute() (in /cvmfs/cms-ib.cern.ch/nweek-02731/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_G4VECGEOM_X_2022-05-04-1100/biglib/slc7_amd64_gcc10/pluginSimulation.so)
==23410==    by 0x3F5D52B9: omt::ThreadHandoff::threadLoop(void*) (in /cvmfs/cms-ib.cern.ch/nweek-02731/slc7_amd64_gcc10/cms/cmssw/CMSSW_12_4_G4VECGEOM_X_2022-05-04-1100/biglib/slc7_amd64_gcc10/pluginSimulation.so)
==23410==    by 0x7085EA4: start_thread (in /usr/lib64/libpthread-2.17.so)
==23410==    by 0x7398B0C: clone (in /usr/lib64/libc-2.17.so)
==23410==  Address 0x58 is not stack'd, malloc'd or (recently) free'd
==23410==

@civanch
Copy link
Contributor

civanch commented May 5, 2022

@smuzaffar , I am just debugging CMSSW_12_4_G4VECGEOM_X_2022-05-04-1100. The crash happens on a boundary of a complex volume with the trace exactly as you show. It is not first track but unlucky track. So, it is VecGeom bug and I just make a ticket to VecGeom:
https://sft.its.cern.ch/jira/browse/VECGEOM-600?jql=project%20%3D%20VECGEOM%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC

In discussion with Gabriele Cosmo he suggested to switch from 11.0.1 to 11.0ref04 - April reference version of G4 + new VecGeom 1.2.0. The first includes improvements of handling of complex volumes inside Geant4. The second means also some change in cmake options.

@smuzaffar
Copy link
Contributor

thanks @civanch . I guess G4 11.0ref04 is not publicly available, can you please provide the tar file for it?

@civanch
Copy link
Contributor

civanch commented May 5, 2022

@smuzaffar
Copy link
Contributor

thanks, I am testing these now. By the way do we still need cms-externals/geant4@271d2ff...cms/v11.0.1 changes for 11.0Ref04?

@civanch
Copy link
Contributor

civanch commented May 5, 2022

No, they are merged to the development version.

@smuzaffar smuzaffar deleted the update-vecgeom-1.1.20 branch May 12, 2022 05:18
@bsunanda
Copy link

bsunanda commented Oct 11, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants