Enable pt-dependent JER-SF and phi-dependent JEC application in PAT [106X backport] #28097

ahinzmann · 2019-10-01T10:11:36Z

PR description:

Enable pt-dependent jet resolution scale factors in PAT. This is needed to apply Autumn18_V7 JER-SF, released last week.
Enable phi-dependent jet energy scale in PAT. This will be needed for 2018 JEC to deal with the HEM issue.
This PR has no impact with the current global tags in any release. However, it will enable using a new global tag (currently in the makings) with pt-dependent JER-SFs and later with phi-dependent JECs.

A test config to apply resolution smearing from a db file has been added under /test.
Here is an example db-file containing pt-dependent JER-SFs:
https://github.com/cms-jet/JRDatabase/blob/master/SQLiteFiles/Autumn18_V7_MC.db

PR validation:

Test config runs after inclusion of this PR. It was checked that this does not change the memory consumption, compared to using the old code with Autumn18_V1_MC.db (no pt-dependence).

if this PR is a backport please specify the original PR:

Backport of #28096

cmsbuild · 2019-10-01T10:12:06Z

A new Pull Request was created by @ahinzmann for CMSSW_10_6_X.

It involves the following packages:

JetMETCorrections/Modules
PhysicsTools/PatAlgos
PhysicsTools/PatUtils

@perrotta, @cmsbuild, @santocch, @slava77 can you please review it and eventually sign? Thanks.
@emilbols, @gouskos, @rappoccio, @HeinerTholen, @peruzzim, @seemasharmafnal, @mmarionncern, @smoortga, @acaudron, @jdolen, @ferencek, @jdamgov, @nhanvtran, @gkasieczka, @schoef, @andrzejnovak, @clelange, @riga, @JyothsnaKomaragiri, @mverzett, @gpetruc, @mariadalfonso, @pvmulder this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

Backported from Enable pt-dependent JER-SF and phi-dependent JEC application in PAT #28096

slava77 · 2019-10-03T09:53:55Z

@cmsbuild please test

cmsbuild · 2019-10-03T09:54:20Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2770/console Started: 2019/10/03 11:54

cmsbuild · 2019-10-03T11:58:19Z

+1
Tested at: 845c13b
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23e80c/2770/summary.html

cmsbuild · 2019-10-03T11:58:23Z

Comparison job queued.

cmsbuild · 2019-10-03T13:56:11Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23e80c/2770/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 3212305
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3211970
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
Checked 137 log files, 14 edm output root files, 33 DQM output files

cmsbuild · 2019-10-04T07:46:26Z

Pull request #28097 was updated. @perrotta, @cmsbuild, @santocch, @slava77 can you please check and sign again.

fabiocos · 2019-10-06T14:13:45Z

please test

cmsbuild · 2019-10-06T14:14:10Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2805/console Started: 2019/10/06 16:14

cmsbuild · 2019-10-06T16:06:36Z

+1
Tested at: ce23314
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23e80c/2805/summary.html

cmsbuild · 2019-10-06T16:06:40Z

Comparison job queued.

cmsbuild · 2019-10-06T18:21:38Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-23e80c/2805/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 3212305
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3211970
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
Checked 137 log files, 14 edm output root files, 33 DQM output files

slava77

no need to follow up on these; I'm leaving the notes here for the record of a tiny inconsistency in the backport

slava77 · 2019-10-08T13:04:28Z

PhysicsTools/PatUtils/interface/SmearedJetProducerT.h

+          {{JME::Binning::JetPt, jet.pt()}, {JME::Binning::JetEta, jet.eta()}, {JME::Binning::Rho, *rho}});
+      double jer_sf = resolution_sf.getScaleFactor({{JME::Binning::JetPt, jet.pt()}, {JME::Binning::JetEta, jet.eta()}},
+                                                   m_systematic_variation);
+      if (m_debug) {


minor: the master version has an empty line above if (m_debug) {

slava77 · 2019-10-08T13:07:28Z

JetMETCorrections/Modules/plugins/JetResolutionDemo.cc

+        res_sf.getScaleFactor({{JME::Binning::JetPt, jet.pt()}, {JME::Binning::JetEta, jet.eta()}}, Variation::UP);
+    float sf_down =
+        res_sf.getScaleFactor({{JME::Binning::JetPt, jet.pt()}, {JME::Binning::JetEta, jet.eta()}}, Variation::DOWN);
+    if (m_debug) {


minor: the master version has an empty line above if (m_debug) {

slava77 · 2019-10-08T13:13:53Z

backport of #28096
this is more of a copy-paste from the master version though; it includes clang-format changes on top of #28096

slava77 · 2019-10-08T13:15:33Z

+1

for #28097 ce23314

code changes are in line with the PR description; added to support additional parameterization in the JEC/JER
the backport is essentially correct (Enable pt-dependent JER-SF and phi-dependent JEC application in PAT [106X backport] #28097 (review))
jenkins tests pass and comparisons with the baseline show no differences

santocch · 2019-10-09T07:53:40Z

+1

cmsbuild · 2019-10-09T07:54:07Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_0_X is complete. This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2019-10-29T08:47:29Z

+1

pt-JER and phi-JEC

845c13b

cmsbuild added this to the CMSSW_10_6_X milestone Oct 1, 2019

cmsbuild added analysis-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Oct 1, 2019

ahinzmann changed the title ~~Enable pt-dependent JER and phi-dependent JEC application in PAT [106X backport]~~ Enable pt-dependent JER-SF and phi-dependent JEC application in PAT [106X backport] Oct 1, 2019

cmsbuild added tests-started and removed tests-pending labels Oct 3, 2019

cmsbuild added tests-approved and removed tests-started labels Oct 3, 2019

cmsbuild added comparison-available and removed comparison-pending labels Oct 3, 2019

update example

ce23314

cmsbuild added comparison-pending tests-pending and removed comparison-available tests-approved labels Oct 4, 2019

cmsbuild removed the tests-pending label Oct 6, 2019

cmsbuild added the tests-started label Oct 6, 2019

cmsbuild added tests-approved and removed tests-started labels Oct 6, 2019

cmsbuild added comparison-available and removed comparison-pending labels Oct 6, 2019

slava77 reviewed Oct 8, 2019

View reviewed changes

cmsbuild added the backport label Oct 8, 2019

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Oct 8, 2019

slava77 mentioned this pull request Oct 8, 2019

Enable pt-dependent JER-SF and phi-dependent JEC application in PAT [102X backport] #28098

Merged

cmsbuild removed the analysis-pending label Oct 9, 2019

cmsbuild added analysis-approved fully-signed backport-ok and removed pending-signatures backport labels Oct 9, 2019

peruzzim mentioned this pull request Oct 23, 2019

NanoAODv6 release preparation #28119

Closed

5 tasks

cmsbuild added orp-approved and removed orp-pending labels Oct 29, 2019

cmsbuild merged commit 20bcf6a into cms-sw:CMSSW_10_6_X Oct 29, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable pt-dependent JER-SF and phi-dependent JEC application in PAT [106X backport] #28097

Enable pt-dependent JER-SF and phi-dependent JEC application in PAT [106X backport] #28097

ahinzmann commented Oct 1, 2019

cmsbuild commented Oct 1, 2019 •

edited

Loading

slava77 commented Oct 3, 2019

cmsbuild commented Oct 3, 2019 •

edited

Loading

cmsbuild commented Oct 3, 2019

cmsbuild commented Oct 3, 2019

cmsbuild commented Oct 3, 2019

cmsbuild commented Oct 4, 2019

fabiocos commented Oct 6, 2019

cmsbuild commented Oct 6, 2019 •

edited

Loading

cmsbuild commented Oct 6, 2019

cmsbuild commented Oct 6, 2019

cmsbuild commented Oct 6, 2019

slava77 left a comment

slava77 Oct 8, 2019

slava77 Oct 8, 2019

slava77 commented Oct 8, 2019

slava77 commented Oct 8, 2019

santocch commented Oct 9, 2019

cmsbuild commented Oct 9, 2019

fabiocos commented Oct 29, 2019

Enable pt-dependent JER-SF and phi-dependent JEC application in PAT [106X backport] #28097

Enable pt-dependent JER-SF and phi-dependent JEC application in PAT [106X backport] #28097

Conversation

ahinzmann commented Oct 1, 2019

PR description:

PR validation:

if this PR is a backport please specify the original PR:

cmsbuild commented Oct 1, 2019 • edited Loading

slava77 commented Oct 3, 2019

cmsbuild commented Oct 3, 2019 • edited Loading

cmsbuild commented Oct 3, 2019

cmsbuild commented Oct 3, 2019

cmsbuild commented Oct 3, 2019

cmsbuild commented Oct 4, 2019

fabiocos commented Oct 6, 2019

cmsbuild commented Oct 6, 2019 • edited Loading

cmsbuild commented Oct 6, 2019

cmsbuild commented Oct 6, 2019

cmsbuild commented Oct 6, 2019

slava77 left a comment

Choose a reason for hiding this comment

slava77 Oct 8, 2019

Choose a reason for hiding this comment

slava77 Oct 8, 2019

Choose a reason for hiding this comment

slava77 commented Oct 8, 2019

slava77 commented Oct 8, 2019

santocch commented Oct 9, 2019

cmsbuild commented Oct 9, 2019

fabiocos commented Oct 29, 2019

cmsbuild commented Oct 1, 2019 •

edited

Loading

cmsbuild commented Oct 3, 2019 •

edited

Loading

cmsbuild commented Oct 6, 2019 •

edited

Loading