Replace buggy UL2016 EGM energy correction files (backport of #46046 to 10_6_X)

[SanghyunKo]

Backport of #46046. As same as the main PR, this should be tested & merged together with cms-data/EgammaAnalysis-ElectronTools#12. After that, the tag pointing to the data repo should be updated as well.

[cmsbuild]

A new Pull Request was created by @SanghyunKo for CMSSW_10_6_X.

It involves the following packages:

• PhysicsTools/NanoAOD (xpog)
• RecoEgamma/EgammaTools (reconstruction)

@cmsbuild, @ftorrresd, @hqucms, @jfernan2, @mandrenguyen can you please review it and eventually sign? Thanks.
@AnnikaStein, @Prasant1993, @Sam-Harper, @a-kapoor, @afiqaize, @gpetruc, @jainshilpi, @lgray, @missirol, @ram1123, @sameasy, @sobhatta, @valsdav, @varuns23 this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

• Backported from Replace buggy UL2016 EGM energy correction files #46046

[cmsbuild]

cms-bot internal usage

[jfernan2]

please test

[cmsbuild]

-1

Failed Tests: RelVals
Size: This PR adds an extra 32KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-75b190/41792/summary.html
COMMIT: 18a5411
CMSSW: CMSSW_10_6_X_2024-09-22-0000/slc7_amd64_gcc700
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46138/41792/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

• 135.4

----- Begin Fatal Exception 27-Sep-2024 22:27:35 CEST-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=CalibratedPatElectronProducer label='calibratedPatElectronsNano'
Exception Message:
edm::FileInPath unable to find file EgammaAnalysis/ElectronTools/data/ScalesSmearings/Run2016_UltraLegacy_postVFP_RunFineEtaR9Gain_v1_scales.dat anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46138/41792/CMSSW_10_6_X_2024-09-22-0000/poison:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46138/41792/CMSSW_10_6_X_2024-09-22-0000/src:/cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46138/41792/CMSSW_10_6_X_2024-09-22-0000/external/slc7_amd64_gcc700/data:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02856/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2024-09-22-0000/src:/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02856/slc7_amd64_gcc700/cms/cmssw/CMSSW_10_6_X_2024-09-22-0000/external/slc7_amd64_gcc700/data
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/matrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS+NANOAOD2016FS
----- End Fatal Exception -------------------------------------------------

[SanghyunKo]

this should be tested & merged together with cms-data/EgammaAnalysis-ElectronTools#12.

The CI will fail without the external :(

[jfernan2]

please test with cms-data/EgammaAnalysis-ElectronTools#12

[antoniovilela]

You will need to backport the tag update in cmsdist as in cms-sw/cmsdist#9448

[SanghyunKo]

Thanks, the backport PRs in cmsdist are created cms-sw/cmsdist#9449 & cms-sw/cmsdist#9450.
Btw, I guess the test is stuck by some reason... (maybe temporary glitch?)

[cmsbuild]

REMINDER @rappoccio, @antoniovilela, @mandrenguyen, @sextonkennedy: This PR was tested with cms-sw/cmsdist#9449, please check if they should be merged together

[hqucms]

test parameters:

• pull_request = [10.6.X] Backport the electron & photon energy correction file updates for 10_6_X/gcc700 cmsdist#9449
• enable = nano

[hqucms]

please abort

[hqucms]

please test

[cmsbuild]

+1

Size: This PR adds an extra 12KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-75b190/41965/summary.html
COMMIT: 18a5411
CMSSW: CMSSW_10_6_X_2024-09-29-0000/slc7_amd64_gcc700
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/46138/41965/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 35
• DQMHistoTests: Total histograms compared: 3215686
• DQMHistoTests: Total failures: 1
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3215351
• DQMHistoTests: Total skipped: 334
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
• Checked 144 log files, 103 edm output root files, 35 DQM output files
• TriggerResults: no differences found

NANO Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 0 differences found in the comparisons
• DQMHistoTests: Total files compared: 1
• DQMHistoTests: Total histograms compared: 0
• DQMHistoTests: Total failures: 0
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 0
• DQMHistoTests: Total skipped: 0
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0 KiB( 0 files compared)
• Checked 0 log files, 0 edm output root files, 1 DQM output files

[jfernan2]

+1

[hqucms]

+1

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_14_2_X is complete. This pull request will now be reviewed by the release team before it's merged. @mandrenguyen, @antoniovilela, @rappoccio, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)
Notice This PR was tested with additional Pull Request(s), please also merge them if necessary: cms-sw/cmsdist#9449

[hqucms]

backport of #46046

Note that this is a bug fix for UL2016 EGM energy correction, and it will change the NanoAOD content.

[vlimant]

from https://indico.cern.ch/event/1470300/ : PPD would be happy to have this in a new 10_6 release. with a view to targeted / on demand data re-nano

[mandrenguyen]

+1

[antoniovilela]

@SanghyunKo @cms-sw/xpog-l2 @cms-sw/ppd-l2
Hi, it seems the backport of the external related to this PR removed something that made tests fail?
Could you have a look?
Thanks.

[SanghyunKo]

Hi @antoniovilela, would you mind specifying which test is failing? I don't see from neither this one nor cms-sw/cmsdist#9449...

[hqucms]

@SanghyunKo The failures are in the 10_6_X IBs:

• https://cmssdt.cern.ch/SDT/html/cmssdt-ib/#/relVal/CMSSW_10_6/2024-11-03-0000?selectedArchs=slc7_amd64_gcc700&selectedFlavors=X&selectedStatus=failed
• https://cmssdt.cern.ch/SDT/cgi-bin/showBuildLogs.py/slc7_amd64_gcc700/www/sun/10.6-sun-00/CMSSW_10_6_X_2024-11-03-0000?utests
  I think it's due to the removal of some old data files -- they are not really used in production workflows, but still needed by some tests. Adding them back in the data PR should fix it.

[smuzaffar]

see cms-data/EgammaAnalysis-ElectronTools#12 (comment)

[SanghyunKo]

@hqucms @smuzaffar @antoniovilela just confirmed it; there is no modifier for these specific IB tests, and therefore they're still running the prelegacy (9_4_X) workflow.

[SanghyunKo]

#46595 should fix the test failures