customize HLT offline DQM for Run3 PbPb and introduce HI 2023A relval workflow (with reHLT)

[mmusich]

PR description:

In the context of CMSALCA-240 it has become clear that in order to run alca validation workflows that re-run the HLT HIon menu, the collections saved in the step2 (with data-tier FEVTDEBUGHLT) are not sufficient for the scope of monitoring objects produced at the HLT (namely tracks and vertices, but also strip and pixel clusters).
This PRs, in presence of the pp_on_PbPb_run3 modifier:

• adds the collections needed for the validation in the FEVTDEBUGHLTEventContent;
• introduces a bunch of modifications of input collections of various monitoring sequences in the package DQMOffline/Trigger

In order to test the whole procedure I introduce a new workflow 142.0 (starting from RunHI2023A RawPrime data) that mimics the procedure in the ticket CMSALCA-240.

PR validation:

Run successfully runTheMatrix.py -l 142.0 -t 4 -j 8.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Not a backport, perhaps to be backported to 13.2.X if there is still a necessity to run the alca validation.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43141/37420

• This PR adds an extra 84KB to repository

• There are other open Pull requests which might conflict with changes you have proposed:

  • File Configuration/PyReleaseValidation/python/relval_steps.py modified in PR(s): Custom_Muon_POG_NanoAOD #42039

[cmsbuild]

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

• Configuration/EventContent (operations)
• Configuration/PyReleaseValidation (pdmv, upgrade)
• DQMOffline/Trigger (dqm)

@AdrianoDee, @miquork, @nothingface0, @antoniovagnerini, @srimanob, @fabiocos, @sunilUIET, @tjavaid, @syuvivida, @davidlange6, @rvenditti, @rappoccio, @cmsbuild, @antoniovilela can you please review it and eventually sign? Thanks.
@mtosi, @cericeci, @Fedespring, @makortel, @fabiocos, @jhgoh, @rociovilar, @slomeo, @HuguesBrun, @missirol, @Martin-Grunewald, @trocino this is something you requested to watch as well.
@rappoccio, @antoniovilela, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[mmusich]

test parameters:

• workflow = 142.0

[mmusich]

@cmsbuild, please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2f2a03/35482/summary.html
COMMIT: 74d4bd1
CMSSW: CMSSW_13_3_X_2023-10-28-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43141/35482/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 12 lines from the logs
• Reco comparison results: 7 differences found in the comparisons
• DQMHistoTests: Total files compared: 50
• DQMHistoTests: Total histograms compared: 3362691
• DQMHistoTests: Total failures: 6
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3362663
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
• Checked 214 log files, 167 edm output root files, 50 DQM output files
• TriggerResults: no differences found

[makortel]

I'm wondering if it would make sense to customize

cmssw/HLTrigger/Configuration/python/HLTrigger_EventContent_cff.py

Line 219 in 994bf20

HLTDebugFEVT = cms.PSet(

instead?

[mmusich]

No, because that implies a charge in the confdb menu and I don't think we want this at this point (perhaps later)

[mmusich]

I'm wondering if it would make sense to customize
cmssw/HLTrigger/Configuration/python/HLTrigger_EventContent_cff.py

For the record (while a bit inelegant) this was taken care in the development combined table at https://its.cern.ch/jira/browse/CMSHLT-2964. This should not stop integration of these changes though, as the ad-hoc customization can be removed once the HLT menu is updated in the release.

[mmusich]

this has been taken care of by #43300 (and corresponding backports)

[mmusich]

@cms-sw/dqm-l2 @cms-sw/pdmv-l2 @cms-sw/upgrade-l2 please review

[perrotta]

Some cleaning, just in case...

[perrotta]

@cms-sw/dqm-l2 @cms-sw/pdmv-l2 @cms-sw/upgrade-l2 could you please review and/or sign this?

[smuzaffar]

Milestone for this pull request has been moved to CMSSW_14_0_X.Please open a backport if it should also go in to CMSSW_13_3_X.

[mmusich]

@cms-sw/dqm-l2 your signature is required here. Thank you.

[tjavaid]

+1

• understanding the differences caused by unstable workflow 12434.7 as reported in spurious differences in outputs of Run-3 wfs *.7 #39803 .

[mmusich]

@cms-sw/orp-l2 please consider merging this PR (and its backports). They are needed in order to pursue the latest alignment @ HLT validation at CMSALCA-240

[antoniovilela]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.