Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new SimBeamSpotHLLHC CondFormat #43186

Merged

Conversation

francescobrivio
Copy link
Contributor

@francescobrivio francescobrivio commented Nov 3, 2023

PR description:

First step of resolving cms-AlCaDB/AlCaTools#95:

  • Added a new SimBeamSpotHLLHC CondFormat and relative record
    • Also added all the needed utilities to write/read/plot/inspect/test the new object
  • Updated HLLHCEvtVtxGenerator plugin adding the possibility to read the smearing parameters from the DB
  • Removed the CrabKissing parameters:

PR validation:

Code compiles and I ran successfully a few upgrade workflows.

Backport:

Not a backport, no backport needed.

@francescobrivio
Copy link
Contributor Author

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 3, 2023

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43186/37510

  • This PR adds an extra 80KB to repository

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 3, 2023

A new Pull Request was created by @francescobrivio for master.

It involves the following packages:

  • CondCore/BeamSpotPlugins (db)
  • CondCore/Utilities (db)
  • CondFormats/BeamSpotObjects (alca, db)
  • CondFormats/DataRecord (alca, db)
  • CondTools/BeamSpot (alca, db)
  • IOMC/EventVertexGenerators (simulation)

@perrotta, @civanch, @francescobrivio, @saumyaphor4252, @consuegs, @mdhildreth can you please review it and eventually sign? Thanks.
@seemasharmafnal, @grzanka, @missirol, @tocheng, @PonIlya, @rsreds, @mmusich, @fabiocos, @yuanchao this is something you requested to watch as well.
@antoniovilela, @sextonkennedy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 4, 2023

-1

Failed Tests: UnitTests RelVals RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/35591/summary.html
COMMIT: 9cff857
CMSSW: CMSSW_13_3_X_2023-11-03-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/35591/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/35591/git-merge-result

Unit Tests

I found 1 errors in the following unit tests:

---> test testTauEmbeddingProducers had ERRORS

RelVals

----- Begin Fatal Exception 04-Nov-2023 00:30:42 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_PAT_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoBTag/Combined/data/ParticleNetFromMiniAODAK4/CHS/Central/modelfile/model.onnx anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/136.7611_RunJetHT2016EreMINIAOD


At:
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Types.py(881): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(381): insertContentsInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(516): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Modules.py(161): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1216): _insertManyInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1490): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 04-Nov-2023 00:30:43 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_PAT_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoBTag/Combined/data/ParticleNetFromMiniAODAK4/CHS/Central/modelfile/model.onnx anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/136.8311_RunJetHT2017FreMINIAOD


At:
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Types.py(881): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(381): insertContentsInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(516): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Modules.py(161): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1216): _insertManyInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1490): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
----- Begin Fatal Exception 04-Nov-2023 00:30:44 CET-----------------------
An exception of category 'ConfigFileReadError' occurred while
   [0] Processing the python configuration file named step2_PAT_DQM.py
Exception Message:
 unknown python problem occurred.
RuntimeError: An exception of category 'FileInPathError' occurred.
Exception Message:
edm::FileInPath unable to find file RecoBTag/Combined/data/ParticleNetFromMiniAODAK4/CHS/Central/modelfile/model.onnx anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43186/35591/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/poison:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/external/el8_amd64_gcc12/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/136.88811_RunJetHT2018DreMINIAODUL


At:
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Types.py(881): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(381): insertContentsInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Mixins.py(516): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Modules.py(161): insertInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1216): _insertManyInto
  /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02809/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_13_3_X_2023-11-03-1100/src/FWCore/ParameterSet/python/Config.py(1490): fillProcessDesc
  <string>(2): <module>

----- End Fatal Exception -------------------------------------------------
Expand to see more relval errors ...

RelVals-INPUT

  • 4.64.6_MinimumBias2010A/step2_MinimumBias2010A.log
  • 136.72411136.72411_RunJetHT2016B_reminiaodUL/step2_RunJetHT2016B_reminiaodUL.log
  • 136.72412136.72412_RunJetHT2016B_reminiaodUL/step2_RunJetHT2016B_reminiaodUL.log
Expand to see more relval errors ...

@francescobrivio
Copy link
Contributor Author

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Nov 5, 2023

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/35613/summary.html
COMMIT: 9cff857
CMSSW: CMSSW_13_3_X_2023-11-05-0000/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43186/35613/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 361 lines from the logs
  • Reco comparison results: 134 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3363010
  • DQMHistoTests: Total failures: 1787
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3361201
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 214 log files, 167 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

@civanch
Copy link
Contributor

civanch commented Nov 6, 2023

+1

@smuzaffar
Copy link
Contributor

Milestone for this pull request has been moved to CMSSW_14_0_X.Please open a backport if it should also go in to CMSSW_13_3_X.

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 4, 2024

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43186/40812

  • This PR adds an extra 96KB to repository

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 4, 2024

Pull request #43186 was updated. @antoniovilela, @civanch, @cmsbuild, @davidlange6, @fabiocos, @mdhildreth, @rappoccio can you please check and sign again.

@francescobrivio
Copy link
Contributor Author

@cmsbuild please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 4, 2024

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-b23df1/40211/summary.html
COMMIT: 17721fb
CMSSW: CMSSW_14_1_X_2024-07-03-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43186/40211/install.sh to create a dev area with all the needed externals and cmssw changes.

  • DAS Queries: The DAS query tests failed, see the summary page for details.

Comparison Summary

There are some workflows for which there are errors in the baseline:
29634.911 step 2
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Summary:

  • You potentially removed 16 lines from the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 10879 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3353315
  • DQMHistoTests: Total failures: 52886
  • DQMHistoTests: Total nulls: 207
  • DQMHistoTests: Total successes: 3300202
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 203 log files, 168 edm output root files, 48 DQM output files
  • TriggerResults: found differences in 3 / 46 workflows

@civanch
Copy link
Contributor

civanch commented Jul 4, 2024

+1

comparison differences are expected because of minor change in formulas.

@antoniovilela
Copy link
Contributor

+1

@cmsbuild
Copy link
Contributor

cmsbuild commented Jul 4, 2024

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

@cmsbuild cmsbuild merged commit 55cba4e into cms-sw:master Jul 4, 2024
11 checks passed
@francescobrivio francescobrivio deleted the alca-add_SimBeamSpotHLLHCObjects branch July 29, 2024 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants