Add pixelgpu online DQM client and augment DQM/Integration unit tests

[mmusich]

PR description:

This PR is a follow-up to my previous PR #41939 and to the TSG ticket https://its.cern.ch/jira/browse/CMSHLT-2846.
The main goal is to add DQM/Integration/python/clients/pixelgpu_dqm_sourceclient-live_cfg.py: a new DQM online client designed to monitor the GPU and CPU Pixel collections from within the streamDQMGPUvsCPU stream.
At the moment only the SiPixelRawDataError collections are monitored by SiPixelPhase1RawDataErrorComparator, but in future, once more event products could be persisted after the ongoing alpaka migration, also other clients that are currently run within the HLT menu itself could be moved here.
I profit of this PR to include the new GPUVsCPU client (and the two existing ECAL and HCAL ones) in the battery of unit tests.
For data a corresponding PR to cms-data has been created (cms-data/DQM-Integration#4) in order to supply the necessary input files (albeit - alas - these files lack the proper pixel collections (SiPixelRawDataErroredmDetSetVector_hltSiPixelDigisFromSoA_*_* and SiPixelRawDataErroredmDetSetVector_hltSiPixelDigisLegacy_*_*) that became available after CMSHLT-2846 was closed).

PR validation:

Run successfully:

scram b runtests_TestDQMOnlineClient-ecalgpu_dqm_sourceclient
scram b runtests_TestDQMOnlineClient-hcalgpu_dqm_sourceclient
scram b runtests_TestDQMOnlineClient-pixelgpu_dqm_sourceclient

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Not a backport, but to be backported for deployment in 2023 data-taking.

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42542/36555

• This PR adds an extra 24KB to repository

[mmusich]

test parameters:

• pull_request = Update DQM-Integration streamer files for streamDQMGPUvsCPU cms-data/DQM-Integration#4

[cmsbuild]

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

• DQM/Integration (dqm)
• DQM/SiPixelHeterogeneous (dqm)

@nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@batinkov, @rchatter, @wang0jin, @argiro, @fioriNTU, @thomreis, @jandrea, @idebruyn, @threus, @francescobrivio this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[mmusich]

@cmsbuild, please test

[cmsbuild]

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-98ba67/34226/summary.html
COMMIT: 56c6da4
CMSSW: CMSSW_13_3_X_2023-08-10-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42542/34226/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @fabiocos MTD Validation: add histograms for detailed material budget vs eta study #42517
• @fmanteca Update showerParams.py in L1Trigger/CSCTriggerPrimitives #42485

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-98ba67/34226/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-98ba67/34226/git-merge-result

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

Summary:

• You potentially removed 14 lines from the logs
• Reco comparison results: 6 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3150947
• DQMHistoTests: Total failures: 9
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3150916
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 207 log files, 159 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[mmusich]

please test

• even though unit tests passed OK

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-98ba67/34248/summary.html
COMMIT: 56c6da4
CMSSW: CMSSW_13_3_X_2023-08-11-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42542/34248/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @mmusich restrict PCA z-component axis in the tracks matched to PV (dz<0.1) #42520
• @cms-tsg-storm HLT menu development for 13_0_X (12/N) [13_3_X] #42537
• @fwyzard Revert the time category for re-joining threads #42533
• @missirol add HLTFilter template HLTL1TMatchedJetsVBFFilter #42543

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-98ba67/34248/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-98ba67/34248/git-merge-result

Comparison Summary

Summary:

• You potentially removed 14 lines from the logs
• ROOTFileChecks: Some differences in event products or their sizes found
• Reco comparison results: 38 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3150945
• DQMHistoTests: Total failures: 2611
• DQMHistoTests: Total nulls: 134
• DQMHistoTests: Total successes: 3148178
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 548.184 KiB( 47 files compared)
• DQMHistoSizes: changed ( 12434.0,... ): 196.482 KiB HLT/TAU
• DQMHistoSizes: changed ( 12434.0,... ): 51.651 KiB HLT/HLTEgammaValidation
• DQMHistoSizes: changed ( 12434.0,... ): 25.958 KiB HLT/Filters
• Checked 207 log files, 159 edm output root files, 48 DQM output files
• TriggerResults: found differences in 2 / 46 workflows

[mmusich]

@cms-sw/dqm-l2 could you please have a look to this one and corresponding cms-data/DQM-Integration#4 ?

[emanueleusai]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[rappoccio]

+1