[13_1_X] Online DQM: replace the hard-coded output directory name of event display clients with an input argument, backport of 41986 and 42128

[syuvivida]

PR description:

This is a backport of PR 41986 and PR 42128.

We are working on the upgrade of online DQM machines [0][1]. There will be a few months that we share the same CMSSW code between the new and the current machines.
In the current (old) DQM machines, the disks of bu-c2f11-09-01 and bu-c2f11-13-01 are mounted on our fu machines as /fff/BU0. Event display clients visualization-live and visualization-live-secondInstance produce output root files at /fff/BU0/output.

However, the mount point (path) has changed in the new online DQM machines [1]. In order to use the same event display client codes for both old and new machines and also to make the path name more flexible, we replace the output path with an input argument (with a default value of /fff/BU0/output). The old machines will use an old hltd version and take the default value of the argument outputBaseDir, while the input values for the new machines will be determined by hltd and startDqmRun.sh.

[0] twiki about the upgrade of DQM machines
[1] JIRA ticket that includes the communication with DAQ
[2] JIRA ticket of the tests during TS1

PR validation:

• This PR has been tested at lxplus by running the hlt, hcal, and ecal clients standalone with CMSSW_13_0_X_2023-06-27-1100 , CMSSW_13_1_X_2023-06-27-1100, and CMSSW_13_2_X_2023-06-26-2300 with the streamers at /eos/cms/store/group/comm_dqm/Collisions23_tempStreamers/.
• This PR has been tested at the current (old) online DQM playback machines and ran all clients without problem when using the default value of the input argument.
• This PR has been deployed/tested when we tested the data transfer during TS1 [2].

[cmsbuild]

A new Pull Request was created by @syuvivida for CMSSW_13_1_X.

It involves the following packages:

• DQM/Integration (dqm)

@nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@threus, @batinkov, @francescobrivio this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

• Backported from Online DQM: add an input argument outputBaseDir in pbsource_cfi.py (to fit with the new hltd version) #42128

[emanueleusai]

please test

[cmsbuild]

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-adeeb1/33509/summary.html
COMMIT: 7b1faf8
CMSSW: CMSSW_13_1_X_2023-07-02-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42133/33509/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

• 134.806134.806_RunMuonEG2015C/step2_RunMuonEG2015C.log
• 134.808134.808_RunSingleMuPrpt2015C/step2_RunSingleMuPrpt2015C.log
• 4.294.29_RunMinBias2011B/step2_RunMinBias2011B.log

Expand to see more relval errors ...

• 136.754
• 23.0
• 134.907
• 136.749
• 136.75
• 136.795
• 136.741
• 136.756
• 136.752
• 136.803
• 136.832
• 136.839
• 136.869
• 1347.0
• 13992502.0
• 134.99601
• 1325.517
• 10004.0
• 134.0
• 10007.0
• 10803.0
• 250206.18

Comparison Summary

Summary:

• You potentially removed 7 lines from the logs
• Reco comparison results: 10 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3222811
• DQMHistoTests: Total failures: 9
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3222780
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 207 log files, 159 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[perrotta]

backport of #42128
(also #41986)

[perrotta]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-adeeb1/33551/summary.html
COMMIT: 7b1faf8
CMSSW: CMSSW_13_1_X_2023-07-04-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42133/33551/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 15 lines to the logs
• Reco comparison results: 14 differences found in the comparisons
• DQMHistoTests: Total files compared: 48
• DQMHistoTests: Total histograms compared: 3222811
• DQMHistoTests: Total failures: 9
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3222780
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
• Checked 207 log files, 159 edm output root files, 48 DQM output files
• TriggerResults: no differences found

[emanueleusai]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_1_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1