use @standardDQM for Run-3 re-HLT workflows [13_0_X]

[missirol]

backport of #40804

PR description:

This backport corrects the choice of DQM/Harvesting sequences used in RelVal wfs for Run-3 Data with an HLT step ("reHLT"), choosing sequences which properly monitor HLT (as opposed to using DQM for "fakeHLT"). Credits to @mmusich.

PR validation:

None.

If this PR is a backport, please specify the original PR and why you need to backport that PR. If this PR will be backported, please specify to which release cycle the backport is meant for:

#40804

Fix to Run-3 data RelVal wfs in release cycle for 2023 pp data-taking.

[cmsbuild]

A new Pull Request was created by @missirol (Marino Missiroli) for CMSSW_13_0_X.

It involves the following packages:

• Configuration/PyReleaseValidation (pdmv, upgrade)

@bbilin, @cmsbuild, @AdrianoDee, @srimanob, @kskovpen, @sunilUIET can you please review it and eventually sign? Thanks.
@makortel, @Martin-Grunewald, @fabiocos, @slomeo, @kpedro88 this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

• Backported from use @standardDQM for Run-3 re-HLT workflows #40804

[missirol]

type bugfix

[missirol]

test parameters:

• workflows = 139.004, 140.003, 140.023, 140.043, 140.063

[missirol]

please test

[cmsbuild]

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-973b38/30876/summary.html
COMMIT: 201178f
CMSSW: CMSSW_13_0_X_2023-02-23-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40872/30876/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

There are some workflows for which there are errors in the baseline:
22034.911 step 1
23234.911 step 1
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Summary:

• You potentially added 1723 lines to the logs
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 21 differences found in the comparisons
• DQMHistoTests: Total files compared: 54
• DQMHistoTests: Total histograms compared: 3813418
• DQMHistoTests: Total failures: 9
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3813387
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 565348.98 KiB( 53 files compared)
• DQMHistoSizes: changed ( 139.004,... ): 40481.807 KiB HLT/JME
• DQMHistoSizes: changed ( 139.004,... ): 12323.616 KiB HLT/Tracking
• DQMHistoSizes: changed ( 139.004,... ): 12284.354 KiB HLT/EXO
• DQMHistoSizes: changed ( 139.004,... ): 11211.193 KiB HLT/BTV
• DQMHistoSizes: changed ( 139.004,... ): 10388.740 KiB HLT/SiStrip
• DQMHistoSizes: changed ( 139.004,... ): 5333.370 KiB HLT/Pixel
• DQMHistoSizes: changed ( 139.004,... ): 4813.427 KiB HLT/TOP
• DQMHistoSizes: changed ( 139.004,... ): 4466.368 KiB HLT/HIG
• DQMHistoSizes: changed ( 139.004,... ): 3494.898 KiB HLT/EGM
• DQMHistoSizes: changed ( 139.004,... ): 3211.306 KiB HLT/SUSY
• DQMHistoSizes: changed ( 139.004 ): ...
• Checked 233 log files, 179 edm output root files, 54 DQM output files
• TriggerResults: no differences found

[srimanob]

@cmsbuild please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-973b38/30891/summary.html
COMMIT: 201178f
CMSSW: CMSSW_13_0_X_2023-02-24-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40872/30891/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 1729 lines to the logs
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 26 differences found in the comparisons
• DQMHistoTests: Total files compared: 54
• DQMHistoTests: Total histograms compared: 3813418
• DQMHistoTests: Total failures: 15
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3813381
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 565348.98 KiB( 53 files compared)
• DQMHistoSizes: changed ( 139.004,... ): 40481.807 KiB HLT/JME
• DQMHistoSizes: changed ( 139.004,... ): 12323.616 KiB HLT/Tracking
• DQMHistoSizes: changed ( 139.004,... ): 12284.354 KiB HLT/EXO
• DQMHistoSizes: changed ( 139.004,... ): 11211.193 KiB HLT/BTV
• DQMHistoSizes: changed ( 139.004,... ): 10388.740 KiB HLT/SiStrip
• DQMHistoSizes: changed ( 139.004,... ): 5333.370 KiB HLT/Pixel
• DQMHistoSizes: changed ( 139.004,... ): 4813.427 KiB HLT/TOP
• DQMHistoSizes: changed ( 139.004,... ): 4466.368 KiB HLT/HIG
• DQMHistoSizes: changed ( 139.004,... ): 3494.898 KiB HLT/EGM
• DQMHistoSizes: changed ( 139.004,... ): 3211.306 KiB HLT/SUSY
• DQMHistoSizes: changed ( 139.004 ): ...
• Checked 233 log files, 179 edm output root files, 54 DQM output files
• TriggerResults: no differences found

[srimanob]

+Upgrade

Backport PR.

[srimanob]

Kindly ping @cms-sw/pdmv-l2
It should be great to have this PR in 13_0_0.

[kskovpen]

+pdmv

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1