[13.0.X] Modernize TrackSplittingMonitor and fix bug with filling ME-s

[mmusich]

backport of #41690

PR description:

The main goal of this PR is to fix the filling of the residuals in the TrackSplittingMonitor DQM code. I noticed that in recent cosmic data these plots are almost all empty (see e.g.https://tinyurl.com/2q9tgomk). This was caused by a bug in the filling of the MonitorElements which is fixed in this PR.
I profit of it to:

• modernize the code of the module
• introduce a fillDescriptions method and use it in the related configurations
• remove the TrackSplitMonitor from the Tracking DQM collisions setup as anyway there are no right input tracks to be fed to the module.

PR validation:

Run successfully runTheMatrix.py -l 138.1 -t 4 -j 8 --custom_command='-n 5000' --ibeos and checked that the cosmic plots are all filled.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Verbatim backport of #41690 needed for 2023 data-taking.

[cmsbuild]

A new Pull Request was created by @mmusich (Marco Musich) for CMSSW_13_0_X.

It involves the following packages:

• DQM/TrackingMonitor (dqm)
• DQM/TrackingMonitorSource (dqm)

@nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@arossi83, @VourMa, @hdelanno, @sroychow, @JanFSchulte, @VinInn, @fioriNTU, @jandrea, @mtosi, @idebruyn, @mmusich, @threus this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

• Backported from Modernize TrackSplittingMonitor and fix bug with filling ME-s #41690

[mmusich]

type bug-fix

[mmusich]

@cmsbuild, please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e4a7e5/32728/summary.html
COMMIT: 4c9a20f
CMSSW: CMSSW_13_0_X_2023-05-19-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41740/32728/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 19 lines to the logs
• Reco comparison results: 293 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3315979
• DQMHistoTests: Total failures: 4162
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3311795
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -367.125 KiB( 48 files compared)
• DQMHistoSizes: changed ( 1000.0,... ): -11.125 KiB Tracking/TrackParameters
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[mmusich]

ping @cms-sw/dqm-l2

[emanueleusai]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

@mmusich when comparing the differences wrt to the master version of this PR I realize that there is a bug (not from this fix) in line

cmssw/DQM/TrackingMonitor/src/TrackSplittingMonitor.cc

Line 235 in 4c9a20f

if ((nRechitinBPIX1 >= pixelHitsPerLeg_) && (nRechitinBPIX1 >= pixelHitsPerLeg_) &&

where nRechitinBPIX1 is called twice, the second time instead of nRechitinBPIX2.

This was "fixed" in master by removing the lines which define nRechitinBPIX2, see #41528

I think this should get fixed both in master and also here: could you please take care iof it?

[mmusich]

I think this should get fixed both in master and also here: could you please take care iof it?

let's do it in another PR.

[mmusich]

-1
Failed Tests: RelVals-INPUT

this cannot possibly come from this PR.

[mmusich]

@cmsbuild, please test

[cmsbuild]

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e4a7e5/32787/summary.html
COMMIT: fffea8d
CMSSW: CMSSW_13_0_X_2023-05-24-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41740/32787/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

• 14.014.0_WpM/step2_WpM.log
• 18.018.0_SingleGammaPt10/step2_SingleGammaPt10.log
• 12.012.0_ZpMM/step2_ZpMM.log

Expand to see more relval errors ...

• 11.0
• ERROR retrieving info from thread: object of type 'int' has no len()17.0

Comparison Summary

Summary:

• You potentially removed 10 lines from the logs
• Reco comparison results: 10 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3315916
• DQMHistoTests: Total failures: 118
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3315776
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -367.125 KiB( 48 files compared)
• DQMHistoSizes: changed ( 1000.0,... ): -11.125 KiB Tracking/TrackParameters
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[mmusich]

@cmsbuild, please test

[perrotta]

@mmusich those RelValInput errors are (annoyingly) rather frequent, and cannot depend on this PR. (They are not even present in the sibling PRs in master and 13_1_X).
Ok, let retry the tests now: however, I'd plan to merge this PR before 1100 regardless of them, so that it can be included in the CMSSW_13_0_7 release that will be started building right after.

[perrotta]

+1

• Tests already succeeded, besides the know and independent issues with RelVal Input ones
• This PR was already signed by @cms-sw/dqm-l2 before the last, quite logic and rather trivial, fix
• Let try to merge it for 13_0_X then, so that it can enter the forthcoming 13_0_7 build

[perrotta]

merge

• Backport of Modernize TrackSplittingMonitor and fix bug with filling ME-s #41690 and TrackSplittingMonitor: fix bug in the BPix hits counting #41769

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e4a7e5/32792/summary.html
COMMIT: fffea8d
CMSSW: CMSSW_13_0_X_2023-05-24-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41740/32792/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 2 lines from the logs
• Reco comparison results: 291 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3315916
• DQMHistoTests: Total failures: 4153
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3311741
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -367.125 KiB( 48 files compared)
• DQMHistoSizes: changed ( 1000.0,... ): -11.125 KiB Tracking/TrackParameters
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[emanueleusai]

Why?

because getting a DQM signature on simple bug-fixes takes several days (if not weeks).

Please refrain from making condescending and inaccurate statements. This is a professional working environment, so please act accordingly.
Please remember that (1) people live in different time zones than you (2) people might have overlapping commitments on certain days (incl. teaching, finals, traveling, conferences, grant deadlines, etc) (3) people might be on vacation or, worse, on medical leave (4) people might be unable to work on weekends. Please respect that.
At the very least, if you have a problem with me kindly reach out to me directly, instead of making passive-aggressive statements. Lastly, I would like to remind the conversations happening on GitHub are still bound by the CMS Code of Conduct [1].

As a recommendation for the future, for truly urgent matters, please use the urgent label.
Additionally, release managers can bypass these signatures for PRs that I have previously approved and have only minor changes.

[1] http://cds.cern.ch/record/2690506/files/Code%20of%20Conduct,%20English.pdf

[emanueleusai]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_2_X is complete. This pull request will be automatically merged.

[mmusich]

Answering on gitHub for the public record:

Please refrain from making condescending and inaccurate statements. This is a professional working environment, so please act accordingly.

The statement was not condescending nor inaccurate. I was merely expressing the preference to introduce the further bug-fix in a second PR in order to avoid loosing the deadline for inclusion of the main fix in the data-taking release due to the subsystem (and cms-bot) signature has arrived after the deadline. This has unfortunately happened a number of times in my experience.

Please remember that (1) people live in different time zones than you (2) people might have overlapping commitments on certain days (incl. teaching, finals, traveling, conferences, grant deadlines, etc) (3) people might be on vacation or, worse, on medical leave (4) people might be unable to work on weekends. Please respect that.

I appreciate that. This is true for most people in the collaboration. Although, precisely for this reason, we generally have more than one manager per software area. I count 7 CMSSW_L2 for DQM here. I am genuinely wondering why the load is not shared among the people that formally hold responsibility.

At the very least, if you have a problem with me kindly reach out to me directly, instead of making passive-aggressive statements. Lastly, I would like to remind the conversations happening on GitHub are still bound by the CMS Code of Conduct [1].

Please point me to which item of the code of conduct I have infringed.
Also, I don't see why this has to become personal. If anything I find your reply rather condescending and possibly violating the CMS code of conduct.

As a recommendation for the future, for truly urgent matters, please use the urgent label.

I would have preferred to resort to that for case of real urgency. In this case it would have been a nice to have feature, that could easily enter the release given it's very limited consequences. But if that's what's required I'll do that in future.