(13_0_X) Add additional track variables to the Run 3 scouting electron collection for low pT electrons.

[arsahasransu]

PR description:

• The PR adds four variables to the Run3 scouting electron collection - pt, eta, phi and chi2/ndof.
• This is necessary for low-energy electrons in the scouting collection.
• The dataset format of the scouting output files should be compatible with previous releases.

PR validation:

• Code validated with privately produced J/PsiToEE MC. Full presentation validated at the scouting meeting Scouting Meeting.
• Basic code-check and code-formatting done.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

• Backport from CMSSW_13_1_X to CMSSW_13_0_X release cycle.
• Master PR is at 41025.

[cmsbuild]

A new Pull Request was created by @arsahasransu (A.R.Sahasransu) for CMSSW_13_0_X.

It involves the following packages:

• DataFormats/Scouting (core)
• HLTrigger/Egamma (hlt)

@Martin-Grunewald, @Dr15Jones, @smuzaffar, @makortel, @cmsbuild, @missirol can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @silviodonato, @rovere this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

• Backported from Add additional track variables to the Run 3 scouting electron collection for low pT electrons. #41025

[missirol]

please test

@arsahasransu , please replace "Backport to" with "(13_0_X)" in the title of the PR.

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd5e9c/31218/summary.html
COMMIT: 14afe4d
CMSSW: CMSSW_13_0_X_2023-03-10-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41035/31218/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 1 lines to the logs
• Reco comparison results: 21 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3557934
• DQMHistoTests: Total failures: 112
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3557800
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[missirol]

backport of #41025

[cmsbuild]

Pull request #41035 was updated. @Martin-Grunewald, @Dr15Jones, @smuzaffar, @makortel, @cmsbuild, @missirol can you please check and sign again.

[missirol]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd5e9c/31231/summary.html
COMMIT: 75a9b77
CMSSW: CMSSW_13_0_X_2023-03-12-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41035/31231/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially removed 14 lines from the logs
• Reco comparison results: 5 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3557934
• DQMHistoTests: Total failures: 6
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3557906
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[makortel]

+core

[cmsbuild]

Pull request #41035 was updated. @Martin-Grunewald, @Dr15Jones, @smuzaffar, @makortel, @cmsbuild, @missirol can you please check and sign again.

[arsahasransu]

Updated the backport with the commit from the main PR.

[missirol]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd5e9c/31355/summary.html
COMMIT: 2342f30
CMSSW: CMSSW_13_0_X_2023-03-16-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41035/31355/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• You potentially added 17 lines to the logs
• Reco comparison results: 5 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3551048
• DQMHistoTests: Total failures: 3
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3551023
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[missirol]

+hlt

• verbatim backport of #41025
• backward-compatible extension of the Run3ScoutingElectron data format for 2023 data-taking (addition of GSF-Tracks-related variables), see #41025 (comment)
• will require the deployment of a new 13_0_X release at HLT before the pp menu is used online in 2023

[makortel]

+core

[makortel]

@cms-sw/orp-l2 I believe this PR could now be unhold

[perrotta]

unhold

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[perrotta]

+1

• Verbati backport of Add additional track variables to the Run 3 scouting electron collection for low pT electrons. #41025
• This changes the DataFormat for scouting, which is something rather unpleasant to do in closed cycles
• However, we understand that this can be needed for the 2023 data taking, which is not yet started (and therefore no data with different format will be taken, provided a new 13_0_1 release is build and implemented at T0 before the actual pp physics will start)
• We rely on the manual validation of backward compatibility verified on existing RAW files by @missirol with the test described in Add tests or workflows that read existing scouting data #41040 (comment)
  • A more formal test for it is now in preparation, see a first unit test for backward compatibility of Scouting data formats #41093