[14_0_X] Custom nano for e/gamma #44255

swagata87 · 2024-02-28T09:01:15Z

Backport of #42607, as requested in #42607 (comment). Seems urgent from this comment by @RSalvatico in original PR:

There is also a chance to test the custom nano production this week if we manage to integrate it,

cmsbuild · 2024-02-28T09:01:42Z

A new Pull Request was created by @swagata87 for CMSSW_14_0_X.

It involves the following packages:

CommonTools/Utils (reconstruction)
Configuration/PyReleaseValidation (pdmv, upgrade)
PhysicsTools/NanoAOD (xpog)

@srimanob, @miquork, @mandrenguyen, @sunilUIET, @vlimant, @cmsbuild, @hqucms, @AdrianoDee, @subirsarkar, @jfernan2 can you please review it and eventually sign? Thanks.
@missirol, @Martin-Grunewald, @fabiocos, @slomeo, @AnnikaStein, @gpetruc, @makortel this is something you requested to watch as well.
@rappoccio, @antoniovilela, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

Backported from EGamma custom nano #42607

cmsbuild · 2024-02-28T09:01:43Z

cms-bot internal usage

swagata87 · 2024-02-28T09:02:49Z

type egamma

swagata87 · 2024-02-28T09:04:41Z

enable nano

swagata87 · 2024-02-28T09:05:16Z

please test

cmsbuild · 2024-02-28T14:34:36Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8714a4/37786/summary.html
COMMIT: ed30b71
CMSSW: CMSSW_14_0_X_2024-02-27-2300/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/44255/37786/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 91 lines from the logs
Reco comparison results: 43 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3245401
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3245373
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 200 log files, 161 edm output root files, 48 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially added 2 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 15
DQMHistoTests: Total histograms compared: 16430
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 16430
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 14 files compared)
Checked 40 log files, 20 edm output root files, 15 DQM output files

Nano size comparison Summary:

Nano ERROR: Missing ref/2500.315-size.json
Nano ERROR: Missing ref/2500.315-size.json
Nano ERROR: Missing ref/2500.316-size.json
Nano ERROR: Missing ref/2500.316-size.json
Nano ERROR: Missing ref/2500.402-size.json
Nano ERROR: Missing ref/2500.402-size.json
| Sample | kb/ev | ref kb/ev | diff kb/ev | ev/s/thd | ref ev/s/thd | diff rate | mem/thd | ref mem/thd |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 2500.0 | 2.551 | 2.551 | 0.000 ( +0.0% ) | 5.23 | 5.25 | -0.3% | 2.235 | 2.235 |
| 2500.001 | 2.700 | 2.700 | 0.000 ( +0.0% ) | 4.70 | 4.73 | -0.6% | 2.665 | 2.667 |
| 2500.002 | 2.641 | 2.641 | 0.000 ( +0.0% ) | 4.74 | 4.87 | -2.9% | 2.652 | 2.661 |
| 2500.01 | 1.322 | 1.322 | 0.000 ( +0.0% ) | 9.73 | 9.82 | -0.8% | 2.355 | 2.354 |
| 2500.011 | 1.745 | 1.745 | 0.000 ( +0.0% ) | 5.28 | 5.29 | -0.2% | 2.524 | 2.537 |
| 2500.012 | 1.586 | 1.586 | 0.000 ( +0.0% ) | 7.51 | 7.52 | -0.2% | 2.446 | 2.398 |
| 2500.1 | 2.194 | 2.194 | 0.000 ( +0.0% ) | 5.28 | 5.30 | -0.3% | 2.073 | 2.074 |
| 2500.2 | 2.310 | 2.310 | 0.000 ( +0.0% ) | 6.08 | 6.07 | +0.1% | 1.984 | 1.987 |
| 2500.21 | 1.185 | 1.185 | 0.000 ( +0.0% ) | 4.33 | 4.35 | -0.3% | 2.288 | 2.286 |
| 2500.211 | 1.549 | 1.549 | 0.000 ( +0.0% ) | 3.71 | 3.71 | +0.1% | 2.367 | 2.352 |
| 2500.3 | 2.060 | 2.060 | 0.000 ( +0.0% ) | 12.40 | 12.54 | -1.1% | 1.972 | 1.975 |
| 2500.31 | 1.258 | 1.258 | 0.000 ( +0.0% ) | 20.07 | 19.88 | +0.9% | 2.363 | 2.375 |
| 2500.311 | 1.644 | 1.644 | 0.000 ( +0.0% ) | 13.47 | 13.43 | +0.3% | 2.447 | 2.447 |
| 2500.312 | 7.159 | 7.159 | 0.000 ( +0.0% ) | 1.43 | 1.38 | +3.7% | 1.698 | 1.695 |
| 2500.313 | 1.564 | 1.564 | 0.000 ( +0.0% ) | 6.72 | 6.61 | +1.8% | 1.055 | 1.056 |
| 2500.314 | 1.157 | 1.157 | 0.000 ( +0.0% ) | 14.08 | 14.13 | -0.4% | 2.209 | 2.191 |
| 2500.4 | 2.060 | 2.060 | 0.000 ( +0.0% ) | 12.63 | 12.60 | +0.3% | 1.974 | 1.971 |
| 2500.401 | 1.817 | 1.817 | 0.000 ( +0.0% ) | 10.22 | 10.31 | -0.9% | 1.829 | 1.826 |
| 2500.5 | 19.575 | 19.575 | 0.000 ( +0.0% ) | 1.17 | 1.10 | +6.0% | 1.364 | 1.361 |

vlimant · 2024-02-28T15:29:30Z

Configuration/PyReleaseValidation/python/relval_nano.py

@@ -239,12 +248,15 @@ def subnext(self):
 workflows[_wfn()] = ['muDPGNANO130Xrun3', ['ZMuSkim2023DRAWRECO13.0', 'muDPGNANO_data13.0']]
 workflows[_wfn()] = ['muDPGNANOBkg130Xrun3', ['ZeroBias2023DRAW13.0', 'muDPGNANOBkg_data13.0']]
 workflows[_wfn()] = ['muPOGNANO_data13.0', ['MuonEG2023MINIAOD13.0', 'muPOGNANO_data13.0']]
+workflows[_wfn()] = ['EGMNANOdata130Xrun3', ['MuonEG2023MINIAOD13.0', 'EGMNano_data13.0']]
+workflows[_wfn()] = ['EGMNANOmc130X', ['TTBarMINIAOD13.0', 'EGMNano_mc13.0']]


please move this up to line 245 below NANOmc130X

cmsbuild · 2024-02-29T08:19:19Z

Pull request #44255 was updated. @miquork, @jfernan2, @subirsarkar, @vlimant, @mandrenguyen, @hqucms, @cmsbuild, @srimanob, @AdrianoDee, @sunilUIET can you please check and sign again.

vlimant · 2024-02-29T22:32:48Z

please test

cmsbuild · 2024-03-01T01:19:43Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8714a4/37838/summary.html
COMMIT: cc72de8
CMSSW: CMSSW_14_0_X_2024-02-29-1100/el8_amd64_gcc12
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/44255/37838/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 95 lines from the logs
Reco comparison results: 59 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3241450
DQMHistoTests: Total failures: 1189
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3240239
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 200 log files, 161 edm output root files, 48 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially added 2 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 15
DQMHistoTests: Total histograms compared: 16430
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 16430
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 14 files compared)
Checked 40 log files, 20 edm output root files, 15 DQM output files

Nano size comparison Summary:

Nano ERROR: Missing ref/2500.301-size.json
Nano ERROR: Missing ref/2500.301-size.json
Nano ERROR: Missing ref/2500.315-size.json
Nano ERROR: Missing ref/2500.315-size.json
Nano ERROR: Missing ref/2500.402-size.json
Nano ERROR: Missing ref/2500.402-size.json
| Sample | kb/ev | ref kb/ev | diff kb/ev | ev/s/thd | ref ev/s/thd | diff rate | mem/thd | ref mem/thd |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 2500.0 | 2.551 | 2.551 | 0.000 ( +0.0% ) | 5.14 | 5.26 | -2.2% | 1.907 | 2.171 |
| 2500.001 | 2.700 | 2.700 | 0.000 ( +0.0% ) | 4.59 | 4.72 | -2.9% | 2.318 | 2.563 |
| 2500.002 | 2.641 | 2.641 | 0.000 ( +0.0% ) | 4.77 | 4.91 | -2.8% | 2.319 | 2.559 |
| 2500.01 | 1.322 | 1.322 | 0.000 ( +0.0% ) | 9.40 | 9.70 | -3.1% | 2.107 | 2.271 |
| 2500.011 | 1.745 | 1.745 | 0.000 ( +0.0% ) | 5.18 | 5.25 | -1.3% | 2.273 | 2.429 |
| 2500.012 | 1.586 | 1.586 | 0.000 ( +0.0% ) | 7.26 | 7.46 | -2.7% | 2.276 | 2.338 |
| 2500.1 | 2.194 | 2.194 | 0.000 ( +0.0% ) | 5.17 | 5.29 | -2.2% | 1.697 | 2.071 |
| 2500.2 | 2.310 | 2.310 | 0.000 ( +0.0% ) | 5.84 | 6.06 | -3.8% | 1.615 | 1.983 |
| 2500.21 | 1.185 | 1.185 | 0.000 ( +0.0% ) | 4.25 | 4.35 | -2.2% | 1.873 | 2.285 |
| 2500.211 | 1.549 | 1.549 | 0.000 ( +0.0% ) | 3.42 | 3.79 | -9.8% | 1.990 | 2.366 |
| 2500.3 | 2.060 | 2.060 | 0.000 ( +0.0% ) | 11.73 | 12.72 | -7.8% | 1.603 | 1.969 |
| 2500.31 | 1.258 | 1.258 | 0.000 ( +0.0% ) | 18.52 | 20.25 | -8.5% | 1.975 | 2.342 |
| 2500.311 | 1.644 | 1.644 | 0.000 ( +0.0% ) | 12.11 | 13.66 | -11.4% | 2.066 | 2.423 |
| 2500.312 | 7.159 | 7.159 | 0.000 ( +0.0% ) | 1.31 | 1.43 | -8.4% | 1.699 | 1.696 |
| 2500.313 | 1.564 | 1.564 | 0.000 ( +0.0% ) | 6.18 | 6.55 | -5.6% | 1.052 | 1.044 |
| 2500.314 | 1.157 | 1.157 | 0.000 ( +0.0% ) | 13.12 | 14.32 | -8.3% | 2.135 | 2.176 |
| 2500.4 | 2.060 | 2.060 | 0.000 ( +0.0% ) | 11.56 | 12.43 | -6.9% | 1.972 | 1.900 |
| 2500.401 | 1.817 | 1.817 | 0.000 ( +0.0% ) | 9.47 | 10.80 | -12.4% | 1.825 | 1.821 |
| 2500.5 | 19.575 | 19.575 | 0.000 ( +0.0% ) | 1.11 | 1.26 | -11.9% | 1.183 | 1.367 |

vlimant · 2024-03-01T09:41:13Z

+1

mandrenguyen · 2024-03-01T10:03:27Z

backport of #42607

mandrenguyen · 2024-03-01T10:04:08Z

+1

subirsarkar · 2024-03-01T11:55:35Z

+Upgrade

AdrianoDee · 2024-03-01T13:06:14Z

+pdmv

cmsbuild · 2024-03-01T13:06:35Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_14_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_14_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @rappoccio, @sextonkennedy, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

antoniovilela · 2024-03-03T23:12:08Z

+1

custom nano for e/gamma

ed30b71

cmsbuild added this to the CMSSW_14_0_X milestone Feb 28, 2024

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending pdmv-pending upgrade-pending xpog-pending labels Feb 28, 2024

swagata87 mentioned this pull request Feb 28, 2024

EGamma custom nano #42607

Merged

cmsbuild added the egamma label Feb 28, 2024

cmsbuild added tests-started and removed tests-pending labels Feb 28, 2024

cmsbuild added tests-approved and removed tests-started labels Feb 28, 2024

vlimant reviewed Feb 28, 2024

View reviewed changes

addressed JR's comment

cc72de8

cmsbuild added tests-pending and removed tests-approved labels Feb 29, 2024

cmsbuild added tests-started and removed tests-pending labels Feb 29, 2024

cmsbuild added tests-approved and removed tests-started labels Mar 1, 2024

cmsbuild added xpog-approved and removed xpog-pending labels Mar 1, 2024

cmsbuild added the backport label Mar 1, 2024

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Mar 1, 2024

cmsbuild added upgrade-approved and removed upgrade-pending labels Mar 1, 2024

cmsbuild added fully-signed pdmv-approved and removed pending-signatures pdmv-pending labels Mar 1, 2024

cmsbuild added backport-ok and removed backport labels Mar 1, 2024

cmsbuild added orp-approved and removed orp-pending labels Mar 3, 2024

cmsbuild merged commit f2ddc9b into cms-sw:CMSSW_14_0_X Mar 3, 2024
12 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[14_0_X] Custom nano for e/gamma #44255

[14_0_X] Custom nano for e/gamma #44255

swagata87 commented Feb 28, 2024

cmsbuild commented Feb 28, 2024 •

edited

Loading

cmsbuild commented Feb 28, 2024 •

edited

Loading

swagata87 commented Feb 28, 2024

swagata87 commented Feb 28, 2024

swagata87 commented Feb 28, 2024

cmsbuild commented Feb 28, 2024

vlimant Feb 28, 2024

cmsbuild commented Feb 29, 2024

vlimant commented Feb 29, 2024

cmsbuild commented Mar 1, 2024

vlimant commented Mar 1, 2024

mandrenguyen commented Mar 1, 2024

mandrenguyen commented Mar 1, 2024

subirsarkar commented Mar 1, 2024

AdrianoDee commented Mar 1, 2024

cmsbuild commented Mar 1, 2024

antoniovilela commented Mar 3, 2024

[14_0_X] Custom nano for e/gamma #44255

[14_0_X] Custom nano for e/gamma #44255

Conversation

swagata87 commented Feb 28, 2024

cmsbuild commented Feb 28, 2024 • edited Loading

cmsbuild commented Feb 28, 2024 • edited Loading

swagata87 commented Feb 28, 2024

swagata87 commented Feb 28, 2024

swagata87 commented Feb 28, 2024

cmsbuild commented Feb 28, 2024

Comparison Summary

NANO Comparison Summary

vlimant Feb 28, 2024

Choose a reason for hiding this comment

cmsbuild commented Feb 29, 2024

vlimant commented Feb 29, 2024

cmsbuild commented Mar 1, 2024

Comparison Summary

NANO Comparison Summary

vlimant commented Mar 1, 2024

mandrenguyen commented Mar 1, 2024

mandrenguyen commented Mar 1, 2024

subirsarkar commented Mar 1, 2024

AdrianoDee commented Mar 1, 2024

cmsbuild commented Mar 1, 2024

antoniovilela commented Mar 3, 2024

cmsbuild commented Feb 28, 2024 •

edited

Loading

cmsbuild commented Feb 28, 2024 •

edited

Loading