[140X] implementation Z/Gamma plus Jet monitoring

[esiam]

PR description:

Adding DQM code for Jets monitoring with Z/Gamma plus Jets analysis module. These changes were discussed with JME trigger group. ( @theochatzis , @slehti ) And they were also presented in the TSG group. ( @missirol , Silvia Goy Lopez )

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

This is a backport of this PR. The backport is needed because it will be used for monitoring 2024 pp data taking.

[cmsbuild]

A new Pull Request was created by @esiam for CMSSW_14_0_X.

It involves the following packages:

• DQMOffline/Trigger (dqm)

@cmsbuild, @antoniovagnerini, @rvenditti, @tjavaid, @nothingface0, @syuvivida can you please review it and eventually sign? Thanks.
@jhgoh, @rociovilar, @Fedespring, @cericeci, @mtosi, @missirol, @trocino, @HuguesBrun this is something you requested to watch as well.
@antoniovilela, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

• Backported from JME DQM implementation of Z/Gamma plus Jet monitoring for Run3 (14_1_X) #44411

[cmsbuild]

cms-bot internal usage

[mmusich]

backport of #44411

[mmusich]

@cmsbuild, please test

[cmsbuild]

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cb959c/38599/summary.html
COMMIT: 49bdce5
CMSSW: CMSSW_14_0_X_2024-04-04-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44612/38599/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

• 1004.01004.0_RunHI2011/step2_RunHI2011.log

Comparison Summary

Summary:

• You potentially added 104 lines to the logs
• Reco comparison results: 36 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3346180
• DQMHistoTests: Total failures: 0
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3346158
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 22855.365 KiB( 48 files compared)
• DQMHistoSizes: changed ( 1000.0,... ): 2539.485 KiB HLT/JME
• Checked 205 log files, 166 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[theochatzis]

Hello, the test that doesn't pass 1004.0 is an old test from 2011 HION and furthermore it was not an issue in the PR to master branch. The reason is that this wf seems to not exist anymore in master.
Maybe this should be removed also in 14_0_X? (do a backport of runTheMatrix.py from 14_1_X change? )
Best,
Sami and Theo

[mmusich]

Maybe this should be removed also in 14_0_X? (do a backport of runTheMatrix.py from 14_1_X change? )

attn: @mandrenguyen

[mandrenguyen]

Hello, the test that doesn't pass 1004.0 is an old test from 2011 HION and furthermore it was not an issue in the PR to master branch. The reason is that this wf seems to not exist anymore in master. Maybe this should be removed also in 14_0_X? (do a backport of runTheMatrix.py from 14_1_X change? ) Best, Sami and Theo

Hi, indeed these old --scenario HeavyIons workflows are not going to work anymore. Would you like to remove it in this PR such that your PR can go forward? Otherwise, I can take care of it in a separate PR, but that seems less practical.

[theochatzis]

Thanks @mandrenguyen , okay this makes sense. @esiam can you do an extra commit for this?

[cmsbuild]

Pull request #44612 was updated. @rvenditti, @AdrianoDee, @sunilUIET, @syuvivida, @srimanob, @antoniovagnerini, @subirsarkar, @cmsbuild, @miquork, @tjavaid, @nothingface0 can you please check and sign again.

[esiam]

Hi! thank you all for your comments. I just pushed a new commit removing it.

[mmusich]

@esiam I am not sure to follow.
You removed wf. 140.53 (# HIRun2011 HIMinBiasUPC) while the failing wf is 1004.0. See also the original commit in master from Matt that removed it: 47ee613.
Can you please explain?

[esiam]

Hi! Marco, I thought they were in runTheMatrix and did a false comparison of 140X and master. Apologies, and thank you for pointing the correct one. I will fix this.

[mmusich]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cb959c/38636/summary.html
COMMIT: 97856e3
CMSSW: CMSSW_14_0_X_2024-04-05-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44612/38636/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 2 lines from the logs
• Reco comparison results: 59 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3346180
• DQMHistoTests: Total failures: 1355
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3344803
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 22855.365 KiB( 48 files compared)
• DQMHistoSizes: changed ( 1000.0,... ): 2539.485 KiB HLT/JME
• Checked 205 log files, 166 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[tjavaid]

+1

• spurious differences seen from unstable WF 12834.7 (open issue spurious differences in outputs of Run-3 wfs *.7 #39803 )

[srimanob]

+Upgrade

[saumyaphor4252]

urgent

• Needed for data taking
• To be included in the next 140X release

Saumya (ORM)

[AdrianoDee]

+pdmv

• wf-wise: "accidentally" back ports 1004 removal to 14_0_X.

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_14_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_14_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @sextonkennedy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[rappoccio]

+1

[perrotta]

This PR is likely generating errors in the 14_0_X IBs for wfs 140.0, 145.0, 150.0 (see e.g. https://cmssdt.cern.ch/SDT/html/cmssdt-ib/#/relVal/CMSSW_14_0/2024-04-08-2300?selectedArchs=el8_amd64_gcc12&selectedFlavors=X&selectedStatus=failed):

----- Begin Fatal Exception 09-Apr-2024 02:21:47 CEST-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Prefetching for module ZGammaplusJetsMonitor/'Photon110Jet_monitoring'
   [3] Prefetching for module ChainedJetCorrectorProducer/'ak4PFPuppiL1FastL2L3Corrector'
   [4] Calling method for module L1FastjetCorrectorProducer/'ak4PFPuppiL1FastjetCorrector'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: double
Looking for module label: fixedGridRhoFastjetAll
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "TryToContinue = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

@esiam @cms-sw/dqm-l2 @cms-sw/pdmv-l2 please have a look

[mmusich]

This PR is likely generating errors in the 14_0_X IBs for wfs 140.0, 145.0, 150.0

I guess all these are all "old" HI workflows that are not present in master (see e.g. this commit 01c75db).
@mandrenguyen would you agree in backporting that to 14_0_X?

[mandrenguyen]

This PR is likely generating errors in the 14_0_X IBs for wfs 140.0, 145.0, 150.0

I guess all these are all "old" HI workflows that are not present in master (see e.g. this commit 01c75db). @mandrenguyen would you agree in backporting that to 14_0_X?

Yes. Just to confirm, you would like to me to make a PR removing the offending relval workflows from CMSSW_14_0_X, right?

[mmusich]

Just to confirm, you would like to me to make a PR removing the offending relval workflows from CMSSW_14_0_X, right?

perhaps it's more straightforward, if you agree and have time to do it. Thanks!

[mandrenguyen]

Just to confirm, you would like to me to make a PR removing the offending relval workflows from CMSSW_14_0_X, right?

perhaps it's more straightforward, if you agree and have time to do it. Thanks!

Sure, I can do it this afternoon.