Add miniAODDQM FastSim Validation #38736

srimanob · 2022-07-14T11:16:38Z

PR description:

This PR follows the discussion in #37347 and #37351, especially on DQM part from @jfernan2 here
This PR is a first attempt to add @miniAODDQM to FastSim validation sequence.

Note:

Code style can be adjusted (i.e. completely separate between FullSim and FastSim, or share as this PR), following the suggestion from @cms-sw/dqm-l2
There are warning from JetMET analyzer as it looks for HLT. I assume this is a task of JetMET to modify the code to accept NoHLT from FastSim.
Next steps:
- Adding VALIDATION:@miniAODValidation is maybe the next step. My stupid question, what is difference between VALIDATION:@miniAODValidation and DQM:@miniAODDQM. How we choose which module should land on which?
- NanoAOD: The structure needs to change, by splitting GEN out from NANO.

FYI @sbein

PR validation:

Run the config files from the following runTheMatrix work,
runTheMatrix.py --what upgrade -l 13234.0 --command="-n 100" --wm init

Additional DQM histrograms are up as expected. One can see from https://tinyurl.com/25u6xovu (with this PR), compared with https://tinyurl.com/27xwxcqh (without this PR)

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

No need of backport.

cmsbuild · 2022-07-14T11:24:15Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38736/31050

This PR adds an extra 28KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Clean up Phase-2 geometry (Up to D85) #38702, Reorganize FastSim Run3 workflow #38660, Add Run3 wfs (NanoV10) #38733, Add run3 tracking low pu era #33532

cmsbuild · 2022-07-14T11:24:38Z

A new Pull Request was created by @srimanob (Phat Srimanobhas) for master.

It involves the following packages:

Configuration/PyReleaseValidation (pdmv, upgrade)
DQMOffline/Configuration (dqm)
DQMOffline/Muon (dqm)

@jordan-martins, @bbilin, @emanueleusai, @ahmad3213, @cmsbuild, @AdrianoDee, @srimanob, @jfernan2, @kskovpen, @pmandrik, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@sscruz, @barvic, @trtomei, @calderona, @makortel, @threus, @slomeo, @abbiendi, @Fedespring, @fabiocos, @Martin-Grunewald, @bellan, @cericeci, @HuguesBrun, @jhgoh, @ptcox, @missirol, @trocino, @beaucero, @kpedro88, @rociovilar this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

srimanob · 2022-07-14T11:39:16Z

@cmsbuild please test

cmsbuild · 2022-07-14T15:53:44Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0fbb44/26245/summary.html
COMMIT: cff59e2
CMSSW: CMSSW_12_5_X_2022-07-13-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38736/26245/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3653966
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3653942
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 6729.583 KiB( 49 files compared)
DQMHistoSizes: changed ( 11634.301 ): 2585.661 KiB Muons_miniAOD/MuonRecoAnalyzer
DQMHistoSizes: changed ( 11634.301 ): 1000.009 KiB ParticleFlow/Offset
DQMHistoSizes: changed ( 11634.301 ): 640.710 KiB JetMET/PFCandidates
DQMHistoSizes: changed ( 11634.301 ): 448.898 KiB ParticleFlow/slimmedJetValidation
DQMHistoSizes: changed ( 11634.301 ): 301.863 KiB ParticleFlow/JetResponse
DQMHistoSizes: changed ( 11634.301 ): 263.359 KiB ParticleFlow/slimmedElectronValidation
DQMHistoSizes: changed ( 11634.301 ): 261.354 KiB ParticleFlow/slimmedMuonValidation
DQMHistoSizes: changed ( 11634.301 ): 239.348 KiB ParticleFlow/slimmedMETValidation
DQMHistoSizes: changed ( 11634.301 ): 239.010 KiB Physics/Top
DQMHistoSizes: changed ( 11634.301 ): 222.585 KiB JetMET/Jet
DQMHistoSizes: changed ( 11634.301 ): ...
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

emanueleusai · 2022-07-15T06:54:34Z

+dqm

extra memory added caused by running miniAODDQM sequence on fastsim

srimanob · 2022-07-15T13:08:42Z

@emanueleusai
Could you please give a hint on what to be landed on Validation and DQM steps of MiniAOD? Thanks.

Note that, I add this PR (MiniAOD validation) together with the discussion of NanoAOD validation for FastSim in today SIM meeting (Talk of Run-3 FastSim workflow),
https://indico.cern.ch/event/1182398/

emanueleusai · 2022-07-18T05:32:06Z

@emanueleusai Could you please give a hint on what to be landed on Validation and DQM steps of MiniAOD? Thanks.

Note that, I add this PR (MiniAOD validation) together with the discussion of NanoAOD validation for FastSim in today SIM meeting (Talk of Run-3 FastSim workflow), https://indico.cern.ch/event/1182398/

@srimanob can you clarify your question? What does landing on Validation and DQM steps of MiniAOD mean?

srimanob · 2022-07-18T16:47:24Z

Hi @emanueleusai

Currently if I check on what VALIDATION and DQM are done for miniaod, I see:

validationMiniAOD = cms.Sequence(type0PFMEtCorrectionPFCandToVertexAssociationForValidationMiniAOD *
JetValidationMiniAOD * METValidationMiniAOD * tauValidationSequenceMiniAOD * bTagMiniValidationSource)

and

DQMOfflineMiniAOD = cms.Sequence(jetMETDQMOfflineRedoProductsMiniAOD*bTagMiniDQMSource*muonMonitors_miniAOD*MuonMiniAOD*DQMOfflinePF)

My curiosity is how do we know what should be in VALIDATION, or DQM? For example, the differences between bTagMiniValidationSource and bTagMiniDQMSource.

Thanks.

cmsbuild · 2022-07-22T22:09:19Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38736/31220

This PR adds an extra 56KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Update BeamSpot in Run3 MC GTs + swap Run3RoundOptics25ns13TeVLowSigmaZ with Realistic25ns13p6TeVEarly2022Collision vtx smearing #38760, Add wfs to test Run3 skims #38830, Add NANO step and Mini+Nano validations to FastSim workflows #38834, Don't recluster jets in HI miniAOD for Run 3 #38711
- File DQMOffline/Configuration/python/DQMOfflineFS_cff.py modified in PR(s): Add NANO step and Mini+Nano validations to FastSim workflows #38834
- File DQMOffline/Muon/python/muonAnalyzer_cff.py modified in PR(s): Add NANO step and Mini+Nano validations to FastSim workflows #38834

cmsbuild · 2022-07-22T22:09:29Z

Pull request #38736 was updated. @perrotta, @rappoccio, @jordan-martins, @bbilin, @emanueleusai, @ahmad3213, @cmsbuild, @AdrianoDee, @srimanob, @jfernan2, @pmandrik, @kskovpen, @qliphy, @rvenditti, @micsucmed, @fabiocos, @davidlange6 can you please check and sign again.

srimanob · 2022-07-22T22:10:48Z

code-checks

cmsbuild · 2022-07-22T22:20:36Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38736/31221

This PR adds an extra 64KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Update BeamSpot in Run3 MC GTs + swap Run3RoundOptics25ns13TeVLowSigmaZ with Realistic25ns13p6TeVEarly2022Collision vtx smearing #38760, Add wfs to test Run3 skims #38830, Add NANO step and Mini+Nano validations to FastSim workflows #38834, Don't recluster jets in HI miniAOD for Run 3 #38711
- File DQMOffline/Configuration/python/DQMOfflineFS_cff.py modified in PR(s): Add NANO step and Mini+Nano validations to FastSim workflows #38834
- File DQMOffline/Muon/python/muonAnalyzer_cff.py modified in PR(s): Add NANO step and Mini+Nano validations to FastSim workflows #38834

srimanob · 2022-07-22T22:22:06Z

@cmsbuild please test

srimanob · 2022-07-22T22:27:21Z

This PR seems to go far beyond expectation, to complete the MiniAODValidation and also harvesting for FastSim. Should we discuss it maybe at the DQM meeting, so that POGs (specially Tau and JetMET) aware of the change? @cms-sw/dqm-l2

cmsbuild · 2022-07-23T07:00:04Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0fbb44/26408/summary.html
COMMIT: 98cf532
CMSSW: CMSSW_12_5_X_2022-07-22-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38736/26408/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3667670
DQMHistoTests: Total failures: 19
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3667628
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 14762.932 KiB( 50 files compared)
DQMHistoSizes: changed ( 13234.0,... ): 2585.661 KiB Muons_miniAOD/MuonRecoAnalyzer
DQMHistoSizes: changed ( 13234.0,... ): 1000.009 KiB ParticleFlow/Offset
DQMHistoSizes: changed ( 13234.0,... ): 640.710 KiB JetMET/PFCandidates
DQMHistoSizes: changed ( 13234.0,... ): 448.898 KiB ParticleFlow/slimmedJetValidation
DQMHistoSizes: changed ( 13234.0,... ): 301.863 KiB ParticleFlow/JetResponse
DQMHistoSizes: changed ( 13234.0,... ): 263.359 KiB ParticleFlow/packedGenParticlesValidation
DQMHistoSizes: changed ( 13234.0,... ): 263.359 KiB ParticleFlow/slimmedElectronValidation
DQMHistoSizes: changed ( 13234.0,... ): 261.354 KiB ParticleFlow/slimmedMuonValidation
DQMHistoSizes: changed ( 13234.0,... ): 239.348 KiB ParticleFlow/slimmedMETValidation
DQMHistoSizes: changed ( 13234.0,... ): 239.010 KiB Physics/Top
DQMHistoSizes: changed ( 13234.0 ): ...
Checked 210 log files, 47 edm output root files, 51 DQM output files
TriggerResults: found differences in 5 / 50 workflows

srimanob · 2022-07-23T13:11:09Z

This PR can be closed as it is taken over by #38834 for complete task of FastSim validation.

cmsbuild added this to the CMSSW_12_5_X milestone Jul 14, 2022

cmsbuild added code-checks-pending dqm-pending orp-pending pdmv-pending pending-signatures tests-pending upgrade-pending labels Jul 14, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 14, 2022

cmsbuild added tests-started and removed tests-pending labels Jul 14, 2022

cmsbuild added tests-approved and removed tests-started labels Jul 14, 2022

cmsbuild added dqm-approved and removed dqm-pending labels Jul 15, 2022

cmsbuild mentioned this pull request Jul 15, 2022

Update BeamSpot in Run3 MC GTs + swap Run3RoundOptics25ns13TeVLowSigmaZ with Realistic25ns13p6TeVEarly2022Collision vtx smearing #38760

Merged

srimanob mentioned this pull request Jul 17, 2022

Reorganize FastSim Run3 workflow #38660

Merged

cmsbuild mentioned this pull request Jul 18, 2022

Add Run3 wfs (NanoV10) #38733

Merged

Norraphat added 2 commits July 21, 2022 19:49

rebase: add miniaod dqm to FastSim

6a58649

rebase

136b5b0

srimanob force-pushed the 125_MiniAODDQMForFastSim branch from cff59e2 to 136b5b0 Compare July 21, 2022 18:51

cmsbuild added code-checks-pending dqm-pending operations-pending pdmv-pending pending-signatures tests-pending upgrade-pending and removed pdmv-approved upgrade-approved code-checks-approved labels Jul 22, 2022

clean up unnecessary harvesting

98cf532

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 22, 2022

cmsbuild added code-checks-pending and removed code-checks-approved labels Jul 22, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 22, 2022

cmsbuild added tests-started and removed tests-pending labels Jul 22, 2022

cmsbuild added tests-approved and removed tests-started labels Jul 23, 2022

srimanob closed this Jul 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add miniAODDQM FastSim Validation #38736

Add miniAODDQM FastSim Validation #38736

srimanob commented Jul 14, 2022 •

edited

Loading

cmsbuild commented Jul 14, 2022

cmsbuild commented Jul 14, 2022

srimanob commented Jul 14, 2022

cmsbuild commented Jul 14, 2022

emanueleusai commented Jul 15, 2022

srimanob commented Jul 15, 2022

emanueleusai commented Jul 18, 2022

srimanob commented Jul 18, 2022

cmsbuild commented Jul 22, 2022

cmsbuild commented Jul 22, 2022

srimanob commented Jul 22, 2022

cmsbuild commented Jul 22, 2022

srimanob commented Jul 22, 2022

srimanob commented Jul 22, 2022

cmsbuild commented Jul 23, 2022

srimanob commented Jul 23, 2022

Add miniAODDQM FastSim Validation #38736

Add miniAODDQM FastSim Validation #38736

Conversation

srimanob commented Jul 14, 2022 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Jul 14, 2022

cmsbuild commented Jul 14, 2022

srimanob commented Jul 14, 2022

cmsbuild commented Jul 14, 2022

Comparison Summary

emanueleusai commented Jul 15, 2022

srimanob commented Jul 15, 2022

emanueleusai commented Jul 18, 2022

srimanob commented Jul 18, 2022

cmsbuild commented Jul 22, 2022

cmsbuild commented Jul 22, 2022

srimanob commented Jul 22, 2022

cmsbuild commented Jul 22, 2022

srimanob commented Jul 22, 2022

srimanob commented Jul 22, 2022

cmsbuild commented Jul 23, 2022

Comparison Summary

srimanob commented Jul 23, 2022

srimanob commented Jul 14, 2022 •

edited

Loading