a first unit test for backward compatibility of Scouting data formats [13_0_X]

[missirol]

backport of #41093

PR description:

This PR adds a unit test to catch non-backward-compatible changes to the Scouting data formats.

From the description of #41093:

This PR adds a refined version of the unit test described in #41040 (comment) [*]. The aim of the test is to fail whenever non-backward-compatible changes are introduced in the Scouting data formats.

The implementation of the test could certainly be improved (one suggestion is in #41040 (comment)), but I decided to open a draft PR because I think the test in this PR is better than nothing. (I plan to address #41040 (comment), but I don't have an ETA for that.)

[*]

More precisely, there are two unit tests, one for the Run-2 Scouting data formats ("Scouting*"), and one for the Run-3 ones ("Run3Scouting*"). The two tests have the same structure:

• a 1st step reads O(100) events from an existing EDM file, and writes the Scouting collections to disk (this first step should fail if non-backward-compatible changes are made to the data formats);

• a 2nd step uses FWLite to print to stdout one entry/object for each of the Scouting data formats being tested. To check that the correct values are read, the test compares the output of step-2 to a reference in the form of a text file. This is currently done for 1 object per data format. It might be necessary to check more than 1 object (and/or more than 1 event), because Add additional track variables to the Run 3 scouting electron collection for low pT electrons. #41025 (comment) showed that some issues can only be spotted when checking multiple objects/events.

PR validation:

The new unit tests pass locally.

If this PR is a backport, please specify the original PR and why you need to backport that PR. If this PR will be backported, please specify to which release cycle the backport is meant for:

#41093

Unit test for Scouting data formats in the release cycle currently used online.

[cmsbuild]

A new Pull Request was created by @missirol (Marino Missiroli) for CMSSW_13_0_X.

It involves the following packages:

• DataFormats/Scouting (core)

@cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please review it and eventually sign? Thanks.
@rovere this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

• Backported from a first unit test for backward compatibility of Scouting data formats #41093

[missirol]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-75a142/31576/summary.html
COMMIT: a6cafc9
CMSSW: CMSSW_13_0_X_2023-03-23-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41168/31576/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 5 lines from the logs
• Reco comparison results: 8 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3552993
• DQMHistoTests: Total failures: 9
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3552962
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[makortel]

+core

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[missirol]

hold

Unit test fails on PowerPC, see #41189.

[cmsbuild]

Pull request has been put on hold by @missirol
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

[cmsbuild]

Pull request #41168 was updated. @cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please check and sign again.

[missirol]

The latest push backports the fix in #41219 for PowerPC builds.

I don't know if the "hold" should stay: the pyroot issue described in #41189 and #41222 may, or may not, require further changes to the unit test. Maybe it's better to wait a bit.

[missirol]

unhold

The pyroot issue described in #41189 and #41222 hasn't been solved yet. Given that ROOT6_X IBs do not run for 13_0_X, I dare say this backport could be merged. I let Core-Sw decide.

[makortel]

+core

Let's get them in. In the longer term we'll working on an alternative testing mechanism (in reaction to #41348, along #41395)

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs after it passes the integration tests and once validation in the development release cycle CMSSW_13_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-75a142/32142/summary.html
COMMIT: a777727
CMSSW: CMSSW_13_0_X_2023-04-25-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41168/32142/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 14 lines from the logs
• Reco comparison results: 3 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3554298
• DQMHistoTests: Total failures: 3
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3554273
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[perrotta]

+1

• Verbatim backport of a first unit test for backward compatibility of Scouting data formats #41093 and Fix DataFormats/Scouting tests on PowerPC #41219