Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add run3_miniAOD_12X era modifier #42740

Merged
merged 3 commits into from
Sep 15, 2023

Conversation

simonepigazzini
Copy link
Contributor

PR description:

Add a modifier to reprocess 2022 data/MC from AOD to produce MINIAOD in 13X.

The new modifier is used to activate the PUPPI jets and MET reclustering in the PAT sequence

PR validation:

Run manually on 1000 events with and w/o the modifier. Differences (at the NANOAOD level, produced on top of the MINIAODs) are spotted for all concerned variables (Jets/MET) while other objects are unchanged. The differences are at the level expect. Plots to be added to the PR.

Backport expected for 13_0_X.

A RelVal workflow should be added for data and MC.

@simonepigazzini
Copy link
Contributor Author

@cms-sw/jetmet-pog-l2

@cms-sw/tau-pog-l2 is there anything needed from your side to make a consistent update of taus from 12_4/12_6 AOD to 13_0 MINIAOD?

@simonepigazzini
Copy link
Contributor Author

enable nano

@simonepigazzini
Copy link
Contributor Author

please test

@simonepigazzini
Copy link
Contributor Author

Enabling nano just to have a convenient way to check that the modifier is "harmless" when not active

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 8, 2023

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-42740/36833

  • This PR adds an extra 32KB to repository

  • There are other open Pull requests which might conflict with changes you have proposed:

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 8, 2023

A new Pull Request was created by @simonepigazzini for master.

It involves the following packages:

  • Configuration/Eras (operations)
  • Configuration/StandardSequences (operations)
  • PhysicsTools/PatAlgos (xpog, reconstruction)

@rappoccio, @vlimant, @jfernan2, @simonepigazzini, @antoniovilela, @mandrenguyen, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @Ming-Yan, @makortel, @felicepantaleo, @hatakeyamak, @emilbols, @Martin-Grunewald, @mbluj, @ahinzmann, @demuller, @seemasharmafnal, @VourMa, @mmarionncern, @missirol, @Senphy, @JanFSchulte, @dgulhan, @jdolen, @azotz, @slomeo, @GiacomoSguazzoni, @rovere, @VinInn, @jdamgov, @nhanvtran, @gkasieczka, @schoef, @mmusich, @mtosi, @fabiocos, @AlexDeMoor, @AnnikaStein, @JyothsnaKomaragiri, @gpetruc, @mariadalfonso, @sameasy, @andrzejnovak this is something you requested to watch as well.
@rappoccio, @antoniovilela, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 8, 2023

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12984e/34670/summary.html
COMMIT: 1db74eb
CMSSW: CMSSW_13_3_X_2023-09-08-1100/el8_amd64_gcc11
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/42740/34670/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially removed 4 lines from the logs
  • Reco comparison results: 342 differences found in the comparisons
  • DQMHistoTests: Total files compared: 48
  • DQMHistoTests: Total histograms compared: 3153414
  • DQMHistoTests: Total failures: 152
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3153240
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
  • Checked 207 log files, 159 edm output root files, 48 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

  • You potentially removed 3 lines from the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 15
  • DQMHistoTests: Total histograms compared: 15610
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 15610
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 14 files compared)
  • Checked 31 log files, 14 edm output root files, 15 DQM output files

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.0 2.460 2.460 0.000 ( +0.0% ) 5.31 5.26 +1.0% 2.162 2.052
2500.001 2.596 2.596 0.000 ( +0.0% ) 4.81 4.75 +1.4% 2.597 2.432
2500.002 2.512 2.512 0.000 ( +0.0% ) 4.98 4.92 +1.1% 2.587 2.427
2500.01 1.253 1.253 0.000 ( +0.0% ) 9.98 9.74 +2.5% 2.296 2.182
2500.011 1.620 1.620 0.000 ( +0.0% ) 5.32 5.26 +1.0% 2.483 2.325
2500.012 1.502 1.502 0.000 ( +0.0% ) 7.69 7.42 +3.5% 2.391 2.204
2500.1 2.118 2.118 0.000 ( +0.0% ) 5.39 5.35 +0.7% 2.028 1.901
2500.2 2.229 2.229 0.000 ( +0.0% ) 6.03 6.18 -2.4% 1.941 1.804
2500.21 1.110 1.110 0.000 ( +0.0% ) 4.44 4.39 +1.0% 2.234 2.071
2500.211 1.464 1.464 0.000 ( +0.0% ) 3.91 3.79 +3.4% 2.317 2.127
2500.3 1.967 1.967 0.000 ( +0.0% ) 12.93 12.96 -0.2% 1.746 1.636
2500.31 1.175 1.175 0.000 ( +0.0% ) 21.10 20.69 +2.0% 2.149 1.981
2500.311 1.550 1.550 0.000 ( +0.0% ) 15.23 14.30 +6.5% 2.226 2.027
2500.4 1.967 1.967 0.000 ( +0.0% ) 13.01 13.03 -0.2% 1.748 1.644

@simonepigazzini
Copy link
Contributor Author

Few plots comparing jets from 126X AOD (estd.Jet_* variables) vs those reclustered using v17 in the PAT sequence activating the modifier introduced in this PR (Jet_* variables). The plots are made from NANOAODs produced with the same cmsDriver on top. All other objects present in the NANOAOD show no difference as expected. The plots are made form a 1000 events from the JetHT dataset (2022D).

On average the reclustered jets have larger pt and a larger EM fraction.

@nurfikri89 thank you again for pointing out the need for a modifier, can you have a look at the PR and the plots?

image
image

@nurfikri89
Copy link
Contributor

@nurfikri89 thank you again for pointing out the need for a modifier, can you have a look at the PR and the plots?

Thanks for making the PR @simonepigazzini. The plots look as expected and everything is in place to recompute puppi weights and recluster puppi jets and MET at PAT level.

@simonepigazzini
Copy link
Contributor Author

please test

@@ -508,6 +508,10 @@
workflows[140.112] = ['',['RunCommissioning2022D','HLTDR3_2022','SKIMCOMMISSIONINGRUN3_reHLT_2022','HARVESTRUN3_2022']]
workflows[140.113] = ['',['RunCosmics2022D','HLTDR3_2022','SKIMCOSMICSRUN3_reHLT_2022','HARVESTRUN3_COS_2022']]

### run3 (2022) reMINIAOD+NANO ###
workflows[140.201] = ['',['RunJetMET2022D_reMINI', 'REMINIAOD_data2022']]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need only MINI workflow, and MINI+Nano+DQM?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MINI + Validation + DQM:@miniAOD fails with an error of missing HGCal collections. I'm really puzzled by the error and do not have time to debug the DQM side at this stage. The MINI and NANO content are anyway as expected, therefore I think the issue with DQM can be dealt with later

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I have that question too on DQM. Since you start from RECO, I am not sure DQM will work as you may miss some transient product which are produced during reconstruction. May I propose to drop the broken workflow for now, for example, you comment it. This will avoid broken in the long matrix test in IB, as workflow in the standard will run. Thx.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as they are right now both tests run ok, or do you mean that the IB expects a DQM output?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, OK. I read it quick and confuse on DQM module.
The DQM of Mini should fail as they need transient output from RECO, but DQM of Nano does not need.

What I don't see the point is why we need both workflows as both produce the same MINI.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There was some concern (I just unjustified) that running MINI or MINI+NANO in the same job could lead to different results / issues. I agree with you that the MINI only workflow is not strictly needed. I will revise that with a follow up PR (we, xpog, want to revise some validation workflows anyway).

@srimanob
Copy link
Contributor

@cmsbuild please test workflow 140.201

@cmsbuild
Copy link
Contributor

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12984e/34750/summary.html
COMMIT: 674e1ab
CMSSW: CMSSW_13_3_X_2023-09-14-1100/el8_amd64_gcc11
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42740/34750/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

  • 13434.013434.0_TTbar_14TeV+2021FSPU/step2_TTbar_14TeV+2021FSPU.log
  • 14234.014234.0_TTbar_14TeV+2023FSPU/step2_TTbar_14TeV+2023FSPU.log

Comparison Summary

Summary:

  • You potentially removed 2 lines from the logs
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3348648
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3348626
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 214 log files, 167 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 15
  • DQMHistoTests: Total histograms compared: 15715
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 15715
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 14 files compared)
  • Checked 31 log files, 14 edm output root files, 15 DQM output files

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.0 2.469 2.469 0.000 ( +0.0% ) 5.18 5.33 -2.8% 2.055 2.113
2500.001 2.611 2.611 0.000 ( +0.0% ) 4.70 4.79 -1.8% 2.444 2.543
2500.002 2.522 2.522 0.000 ( +0.0% ) 4.84 4.95 -2.3% 2.432 2.544
2500.01 1.264 1.264 0.000 ( +0.0% ) 9.52 9.76 -2.5% 2.140 2.217
2500.011 1.634 1.634 0.000 ( +0.0% ) 5.18 5.29 -2.2% 2.274 2.357
2500.012 1.517 1.517 0.000 ( +0.0% ) 7.42 7.51 -1.3% 2.201 2.279
2500.1 2.126 2.126 0.000 ( +0.0% ) 5.28 5.37 -1.7% 1.908 2.034
2500.2 2.237 2.237 0.000 ( +0.0% ) 6.08 6.13 -0.8% 1.810 1.941
2500.21 1.125 1.125 0.000 ( +0.0% ) 4.32 4.40 -1.9% 2.079 2.215
2500.211 1.479 1.479 0.000 ( +0.0% ) 3.82 3.90 -2.2% 2.129 2.194
2500.3 1.977 1.977 0.000 ( +0.0% ) 12.76 13.04 -2.1% 1.797 1.914
2500.31 1.190 1.190 0.000 ( +0.0% ) 19.87 20.77 -4.4% 2.151 2.291
2500.311 1.565 1.565 0.000 ( +0.0% ) 14.03 14.70 -4.5% 2.192 2.257
2500.4 1.977 1.977 0.000 ( +0.0% ) 12.78 13.10 -2.4% 1.799 1.914

@AdrianoDee
Copy link
Contributor

AdrianoDee commented Sep 14, 2023

please test

let's give it another try, seems unrelated to me

@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-12984e/34762/summary.html
COMMIT: 674e1ab
CMSSW: CMSSW_13_3_X_2023-09-14-1100/el8_amd64_gcc11
Additional Tests: NANO
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/42740/34762/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 5 lines to the logs
  • Reco comparison results: 14 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3348648
  • DQMHistoTests: Total failures: 9
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3348617
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 214 log files, 167 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

NANO Comparison Summary

Summary:

  • You potentially added 8 lines to the logs
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 15
  • DQMHistoTests: Total histograms compared: 15715
  • DQMHistoTests: Total failures: 0
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 15715
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 14 files compared)
  • Checked 31 log files, 14 edm output root files, 15 DQM output files

Nano size comparison Summary:

Sample kb/ev ref kb/ev diff kb/ev ev/s/thd ref ev/s/thd diff rate mem/thd ref mem/thd
2500.0 2.469 2.469 0.000 ( +0.0% ) 5.25 5.33 -1.5% 2.009 2.113
2500.001 2.611 2.611 0.000 ( +0.0% ) 4.73 4.79 -1.2% 2.012 2.543
2500.002 2.522 2.522 0.000 ( +0.0% ) 4.93 4.95 -0.5% 2.026 2.544
2500.01 1.264 1.264 0.000 ( +0.0% ) 9.66 9.76 -1.0% 2.129 2.217
2500.011 1.634 1.634 0.000 ( +0.0% ) 5.23 5.29 -1.1% 1.916 2.357
2500.012 1.517 1.517 0.000 ( +0.0% ) 7.47 7.51 -0.5% 1.912 2.279
2500.1 2.126 2.126 0.000 ( +0.0% ) 5.34 5.37 -0.5% 1.877 2.034
2500.2 2.237 2.237 0.000 ( +0.0% ) 6.11 6.13 -0.3% 1.756 1.941
2500.21 1.125 1.125 0.000 ( +0.0% ) 4.37 4.40 -0.6% 1.732 2.215
2500.211 1.479 1.479 0.000 ( +0.0% ) 3.85 3.90 -1.3% 1.829 2.194
2500.3 1.977 1.977 0.000 ( +0.0% ) 12.85 13.04 -1.4% 1.809 1.914
2500.31 1.190 1.190 0.000 ( +0.0% ) 20.41 20.77 -1.7% 2.148 2.291
2500.311 1.565 1.565 0.000 ( +0.0% ) 14.13 14.70 -3.9% 2.202 2.257
2500.4 1.977 1.977 0.000 ( +0.0% ) 12.85 13.10 -1.9% 1.807 1.914

@antoniovilela
Copy link
Contributor

+operations

@AdrianoDee
Copy link
Contributor

+pdmv

@srimanob
Copy link
Contributor

srimanob commented Sep 15, 2023

+Upgrade

The PR adds 2 workflows. One with MINI only, one with MINI+Nano+nano DQM. MiniAOD DQM is not possible as it needs transient products during RECO step.

One unclear point to me is why do need both 140.201 and 140.202 as they produce the same mini. But I assume there is a reason behind this decision, which I may skip when reviewed.

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @rappoccio, @antoniovilela, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

@antoniovilela
Copy link
Contributor

+1

@cmsbuild cmsbuild merged commit 95265a9 into cms-sw:master Sep 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants