[13_0_X backport ] Fall17V2 photon IDs and Isolations saved only in Run2 Nanos #42717

Prasant1993 · 2023-09-04T20:24:54Z

PR description:

Backport of PR #42255 (Add 4 Isolation variables for Photon to NanoAOD) and #42247 (Add hasConversionTracks to nano). See description there.

PR validation:

Same as #42255

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport required for processing of 2022 and 2023 data/MC which will happen in 13_0_X.

cmsbuild · 2023-09-04T20:25:22Z

A new Pull Request was created by @Prasant1993 (Prasant Kumar Rout) for CMSSW_13_0_X.

It involves the following packages:

PhysicsTools/NanoAOD (xpog)

@cmsbuild, @simonepigazzini, @vlimant can you please review it and eventually sign? Thanks.
@gpetruc, @AnnikaStein this is something you requested to watch as well.
@rappoccio, @antoniovilela, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

simonepigazzini · 2023-09-05T06:26:15Z

enable nano

simonepigazzini · 2023-09-05T06:26:23Z

please test

cmsbuild · 2023-09-05T09:20:23Z

-1

Failed Tests: UnitTests RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-593f40/34602/summary.html
COMMIT: 6bd0f8d
CMSSW: CMSSW_13_0_X_2023-09-04-2300/el8_amd64_gcc11
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42717/34602/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found 1 errors in the following unit tests:

---> test test-das-selected-lumis had ERRORS

RelVals-INPUT

144.6144.6_MinimumBias2010A/step2_MinimumBias2010A.log
138.1DAS Error
138.2DAS Error

Expand to see more relval errors ...

140.51
140.52
140.54
158.01
158.1
158.2
158.3
159.01
1306.0
1307.0
1308.0
1309.0
1310.0
1311.0
1312.0
1313.0
1314.0
1315.0
1316.0
1317.0
1318.0
1319.0
1320.0
1321.0
1322.0
1323.0
1324.0
1325.0
1325.1
1325.2
1325.3
1325.4
1325.5
1325.51
1325.516
1325.5161
1325.517
1325.518
1325.9
1325.91
1326.0
1327.0
1328.0
1329.0
1330.0
1331.0
1332.0
1333.0
1334.0
1335.0
1336.0
1337.0
1338.0
1339.0
1340.0
1341.0
1343.0
1344.0
1345.0
1347.0
1348.0
1349.0
1350.0
1351.0
1352.0
1353.0
1354.0
1355.0
1364.0
1365.0
1366.0
134.0
139901.0
139902.0
13992501.0
13992502.0
200.0
202.0
203.0
205.0
11024.2
25200.0
25202.0
25202.1
25202.15
25202.2
25203.0
25204.0
25205.0
25206.0
25207.0
25208.0
25209.0
25214.0
50200.0
50202.0
50203.0
50204.0
50205.0
50206.0
50207.0
50208.0
1000.0
1001.0
1002.0
1003.0
1010.0
1020.0

Comparison Summary

Summary:

You potentially added 2 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 128 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3180587
DQMHistoTests: Total failures: 1032
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3179533
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 29.413000000000004 KiB( 47 files compared)
DQMHistoSizes: changed ( 11634.0,... ): 3.681 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 13234.0,... ): 1.823 KiB Physics/NanoAODDQM
Checked 210 log files, 160 edm output root files, 48 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially removed 4 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 35 differences found in the comparisons
DQMHistoTests: Total files compared: 12
DQMHistoTests: Total histograms compared: 12351
DQMHistoTests: Total failures: 32
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 12319
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 31.201000000000004 KiB( 11 files compared)
DQMHistoSizes: changed ( 2500.311,... ): 3.681 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 2500.331,... ): 1.823 KiB Physics/NanoAODDQM
Checked 25 log files, 11 edm output root files, 12 DQM output files

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.31	2.459	2.445	0.013 ( +0.5% )	5.34	5.42	-1.4%	2.090	2.077
2500.311	2.594	2.552	0.042 ( +1.6% )	4.80	4.85	-1.0%	2.522	2.509
2500.312	2.510	2.497	0.013 ( +0.5% )	5.00	4.99	+0.1%	2.497	2.498
2500.33	1.252	1.240	0.011 ( +0.9% )	10.19	10.20	-0.1%	2.208	2.185
2500.331	1.618	1.603	0.014 ( +0.9% )	5.44	5.47	-0.4%	2.398	2.358
2500.332	1.500	1.485	0.015 ( +1.0% )	7.85	7.80	+0.6%	2.305	2.225
2500.401	2.153	2.158	-0.005 ( -0.3% )	5.42	5.50	-1.5%	1.951	1.946
2500.501	1.756	1.762	-0.006 ( -0.3% )	9.12	9.32	-2.2%	1.832	1.828
2500.511	1.140	1.146	-0.006 ( -0.5% )	4.50	4.51	-0.2%	2.151	2.129
2500.5111	1.498	1.504	-0.006 ( -0.4% )	3.88	3.99	-2.7%	2.224	2.207
2500.601	2.019	2.025	-0.006 ( -0.3% )	12.69	13.10	-3.1%	1.708	1.710

simonepigazzini · 2023-09-05T09:48:20Z

please test

simonepigazzini · 2023-09-05T13:07:28Z

@Prasant1993 what is your time scale for the electron PR? we would like to launch a NANO production very soon and would like to finalize the release for that this week.

Thank you

simonepigazzini · 2023-09-05T13:09:23Z

btw, http://tinyurl.com/2xkm4m6g some of the added variables are empty. Chiefly "hasConvertionTracks"

Prasant1993 · 2023-09-05T13:33:57Z

@Prasant1993 what is your time scale for the electron PR? we would like to launch a NANO production very soon and would like to finalize the release for that this week.

Thank you

Hi @simonepigazzini, yes I will do the electron PR by doing the necessary changes as photons by tomorrow.

Prasant1993 · 2023-09-05T13:39:53Z

btw, http://tinyurl.com/2xkm4m6g some of the added variables are empty. Chiefly "hasConvertionTracks"

Thanks for pointing this out. The name of this DQM histogram is not correctly saved. It is currently saved as "hasConversionTrackTracks".
However, it should be saved as "hasConversionTracks" as the right variable.
So it is also wrong in this PR #42247 by @JaLuka98 .
I will correct this in this PR.

The Fall17V2 DQM histograms are not being filled up, as the variable name is not there now as we have same name for both Run2 and Run3. Can you suggest what should we do for Run2 ID and isolation variables in DQM histograms ?

cmsbuild · 2023-09-05T13:53:54Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-593f40/34607/summary.html
COMMIT: 6bd0f8d
CMSSW: CMSSW_13_0_X_2023-09-04-2300/el8_amd64_gcc11
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42717/34607/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 15 lines to the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 126 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3180587
DQMHistoTests: Total failures: 54
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3180511
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 29.413000000000004 KiB( 47 files compared)
DQMHistoSizes: changed ( 11634.0,... ): 3.681 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 13234.0,... ): 1.823 KiB Physics/NanoAODDQM
Checked 210 log files, 160 edm output root files, 48 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially removed 2 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 35 differences found in the comparisons
DQMHistoTests: Total files compared: 12
DQMHistoTests: Total histograms compared: 12351
DQMHistoTests: Total failures: 32
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 12319
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 31.201000000000004 KiB( 11 files compared)
DQMHistoSizes: changed ( 2500.311,... ): 3.681 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 2500.331,... ): 1.823 KiB Physics/NanoAODDQM
Checked 25 log files, 11 edm output root files, 12 DQM output files

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.31	2.459	2.445	0.013 ( +0.5% )	5.41	5.42	-0.2%	2.087	2.077
2500.311	2.594	2.552	0.042 ( +1.6% )	4.81	4.85	-0.9%	2.524	2.509
2500.312	2.510	2.497	0.013 ( +0.5% )	5.05	4.99	+1.0%	2.500	2.498
2500.33	1.252	1.240	0.011 ( +0.9% )	10.25	10.20	+0.5%	2.205	2.185
2500.331	1.618	1.603	0.014 ( +0.9% )	5.46	5.47	-0.1%	2.399	2.358
2500.332	1.500	1.485	0.015 ( +1.0% )	7.91	7.80	+1.4%	2.308	2.225
2500.401	2.153	2.158	-0.005 ( -0.3% )	5.45	5.50	-0.9%	1.947	1.946
2500.501	1.756	1.762	-0.006 ( -0.3% )	9.38	9.32	+0.6%	1.825	1.828
2500.511	1.140	1.146	-0.006 ( -0.5% )	4.52	4.51	+0.3%	2.154	2.129
2500.5111	1.498	1.504	-0.006 ( -0.4% )	4.00	3.99	+0.4%	2.230	2.207
2500.601	2.019	2.025	-0.006 ( -0.3% )	13.09	13.10	-0.0%	1.712	1.710

cmsbuild · 2023-09-05T15:01:09Z

Pull request #42717 was updated. @cmsbuild, @simonepigazzini, @vlimant can you please check and sign again.

simonepigazzini · 2023-09-05T15:19:55Z

please test

JaLuka98 · 2023-09-05T15:38:01Z

btw, http://tinyurl.com/2xkm4m6g some of the added variables are empty. Chiefly "hasConvertionTracks"

Thanks for pointing this out. The name of this DQM histogram is not correctly saved. It is currently saved as "hasConversionTrackTracks". However, it should be saved as "hasConversionTracks" as the right variable. So it is also wrong in this PR #42247 by @JaLuka98 . I will correct this in this PR.

Hi @Prasant1993 thanks for fixing this issue. My bad, it was an oversight. Sorry!

cmsbuild · 2023-09-05T17:38:57Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-593f40/34613/summary.html
COMMIT: b07ab5d
CMSSW: CMSSW_13_0_X_2023-09-05-1100/el8_amd64_gcc11
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42717/34613/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 237 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 128 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3283193
DQMHistoTests: Total failures: 49
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3283122
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 29.333000000000002 KiB( 48 files compared)
DQMHistoSizes: changed ( 11634.0,... ): 3.671 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 13234.0,... ): 1.818 KiB Physics/NanoAODDQM
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

NANO Comparison Summary

Summary:

You potentially removed 2 lines from the logs
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 35 differences found in the comparisons
DQMHistoTests: Total files compared: 12
DQMHistoTests: Total histograms compared: 12351
DQMHistoTests: Total failures: 32
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 12319
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 31.116 KiB( 11 files compared)
DQMHistoSizes: changed ( 2500.311,... ): 3.671 KiB Physics/NanoAODDQM
DQMHistoSizes: changed ( 2500.331,... ): 1.818 KiB Physics/NanoAODDQM
Checked 25 log files, 11 edm output root files, 12 DQM output files

Nano size comparison Summary:

Sample	kb/ev	ref kb/ev	diff kb/ev	ev/s/thd	ref ev/s/thd	diff rate	mem/thd	ref mem/thd
2500.31	2.459	2.445	0.013 ( +0.5% )	5.37	5.41	-0.8%	1.969	2.090
2500.311	2.593	2.552	0.041 ( +1.6% )	4.89	4.88	+0.2%	2.391	2.514
2500.312	2.510	2.496	0.013 ( +0.5% )	5.06	4.98	+1.5%	2.382	2.505
2500.33	1.252	1.240	0.011 ( +0.9% )	10.35	10.34	+0.1%	2.205	2.213
2500.331	1.618	1.604	0.014 ( +0.9% )	5.50	5.48	+0.4%	2.391	2.397
2500.332	1.500	1.485	0.015 ( +1.0% )	7.93	7.97	-0.5%	2.312	2.313
2500.401	2.153	2.158	-0.005 ( -0.3% )	5.50	5.48	+0.3%	1.842	1.951
2500.501	1.756	1.762	-0.006 ( -0.3% )	9.38	9.38	-0.0%	1.700	1.833
2500.511	1.140	1.146	-0.006 ( -0.5% )	4.52	4.54	-0.4%	2.050	2.160
2500.5111	1.498	1.504	-0.006 ( -0.4% )	4.07	4.06	+0.2%	2.123	2.230
2500.601	2.019	2.025	-0.006 ( -0.3% )	13.19	13.20	-0.1%	1.574	1.712

simonepigazzini · 2023-09-06T08:23:25Z

+1

cmsbuild · 2023-09-06T08:23:49Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_3_X is complete. This pull request will now be reviewed by the release team before it's merged. @rappoccio, @antoniovilela, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

antoniovilela · 2023-09-06T17:09:03Z

+1

Fall17V2 photon IDs and Isolations saved only in Run2 Nanos

6bd0f8d

cmsbuild added this to the CMSSW_13_0_X milestone Sep 4, 2023

cmsbuild added pending-signatures tests-pending orp-pending xpog-pending labels Sep 4, 2023

cmsbuild added tests-started and removed tests-pending labels Sep 5, 2023

cmsbuild added tests-rejected and removed tests-started labels Sep 5, 2023

cmsbuild added tests-started and removed tests-rejected labels Sep 5, 2023

cmsbuild added tests-approved and removed tests-started labels Sep 5, 2023

Correct hasConversionTracks variable for photon in NANODQM

b07ab5d

cmsbuild added tests-pending and removed tests-approved labels Sep 5, 2023

cmsbuild removed the tests-pending label Sep 5, 2023

cmsbuild added the tests-started label Sep 5, 2023

cmsbuild added tests-approved and removed tests-started labels Sep 5, 2023

cmsbuild added fully-signed xpog-approved and removed pending-signatures xpog-pending labels Sep 6, 2023

cmsbuild added orp-approved and removed orp-pending labels Sep 6, 2023

cmsbuild merged commit 35bbeb5 into cms-sw:CMSSW_13_0_X Sep 6, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[13_0_X backport ] Fall17V2 photon IDs and Isolations saved only in Run2 Nanos #42717

[13_0_X backport ] Fall17V2 photon IDs and Isolations saved only in Run2 Nanos #42717

Prasant1993 commented Sep 4, 2023

cmsbuild commented Sep 4, 2023

simonepigazzini commented Sep 5, 2023

simonepigazzini commented Sep 5, 2023

cmsbuild commented Sep 5, 2023

simonepigazzini commented Sep 5, 2023

simonepigazzini commented Sep 5, 2023

simonepigazzini commented Sep 5, 2023

Prasant1993 commented Sep 5, 2023

Prasant1993 commented Sep 5, 2023 •

edited

Loading

cmsbuild commented Sep 5, 2023

cmsbuild commented Sep 5, 2023

simonepigazzini commented Sep 5, 2023

JaLuka98 commented Sep 5, 2023

cmsbuild commented Sep 5, 2023

simonepigazzini commented Sep 6, 2023

cmsbuild commented Sep 6, 2023

antoniovilela commented Sep 6, 2023

[13_0_X backport ] Fall17V2 photon IDs and Isolations saved only in Run2 Nanos #42717

[13_0_X backport ] Fall17V2 photon IDs and Isolations saved only in Run2 Nanos #42717

Conversation

Prasant1993 commented Sep 4, 2023

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Sep 4, 2023

simonepigazzini commented Sep 5, 2023

simonepigazzini commented Sep 5, 2023

cmsbuild commented Sep 5, 2023

Unit Tests

RelVals-INPUT

Comparison Summary

NANO Comparison Summary

simonepigazzini commented Sep 5, 2023

simonepigazzini commented Sep 5, 2023

simonepigazzini commented Sep 5, 2023

Prasant1993 commented Sep 5, 2023

Prasant1993 commented Sep 5, 2023 • edited Loading

cmsbuild commented Sep 5, 2023

Comparison Summary

NANO Comparison Summary

cmsbuild commented Sep 5, 2023

simonepigazzini commented Sep 5, 2023

JaLuka98 commented Sep 5, 2023

cmsbuild commented Sep 5, 2023

Comparison Summary

NANO Comparison Summary

simonepigazzini commented Sep 6, 2023

cmsbuild commented Sep 6, 2023

antoniovilela commented Sep 6, 2023

Prasant1993 commented Sep 5, 2023 •

edited

Loading