Adding parameters needed for PUjetID 94X and 102X v2 #28984

alefisico · 2020-02-18T08:32:21Z

PR description:

Including the files and parameters needed for the latest PUjetID training for 94X and 102X samples.
Once merged, we need to backport this PR to the corresponding CMSSW release for the latest nanoAOD.
There is also a PR for the new weights for PUJetID Including weights for 94X and 102X PUjetID training cms-data/RecoJets-JetProducers#10 (comment)

This PR superseeds #28827.
Expected changes with this PR: https://indico.cern.ch/event/860457/#17-pujetid-training-and-valida

FYI: @ahinzmann @lathomas @camclean

cmsbuild · 2020-02-18T08:32:49Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-02-18T08:39:49Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28984/13812

This PR adds an extra 16KB to repository

cmsbuild · 2020-02-18T08:40:10Z

A new Pull Request was created by @alefisico (Alejandro Gomez Espinosa) for master.

It involves the following packages:

RecoJets/JetProducers

@perrotta, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@rappoccio, @jdolen, @ahinzmann, @jdamgov, @yslai, @nhanvtran, @gkasieczka, @clelange, @schoef, @mariadalfonso, @seemasharmafnal this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

slava77 · 2020-02-18T12:59:54Z

@cmsbuild please test

cmsbuild · 2020-02-18T13:00:15Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4730/console Started: 2020/02/18 14:00

alefisico · 2020-02-18T13:05:17Z

I think this test will fail because this PR cms-data/RecoJets-JetProducers#10 needs to be included.

cmsbuild · 2020-02-18T13:53:57Z

-1

Tested at: 76fe14d

CMSSW: CMSSW_11_1_X_2020-02-18-1100
SCRAM_ARCH: slc7_amd64_gcc820
You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-577f23/4730/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

RelVals:

When I ran the RelVals I found an error in the following workflows:
136.7611 step2

runTheMatrix-results/136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM/step2_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM.log

136.8311 step2

runTheMatrix-results/136.8311_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017/step2_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017.log

140.56 step2

runTheMatrix-results/140.56_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18/step2_RunHI2018+RunHI2018+RECOHID18+HARVESTDHI18.log

4.53 step3

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step3_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log

136.731 step3

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step3_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log

136.793 step3

runTheMatrix-results/136.793_RunDoubleEG2017C+RunDoubleEG2017C+HLTDR2_2017+RECODR2_2017reHLT_skimDoubleEG_Prompt+HARVEST2017/step3_RunDoubleEG2017C+RunDoubleEG2017C+HLTDR2_2017+RECODR2_2017reHLT_skimDoubleEG_Prompt+HARVEST2017.log

136.874 step3

runTheMatrix-results/136.874_RunEGamma2018C+RunEGamma2018C+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM/step3_RunEGamma2018C+RunEGamma2018C+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM.log

135.4 step3

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step3_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

1325.7 step2

runTheMatrix-results/1325.7_TTbar_13_94Xv2NanoAODINPUT+TTbar_13_94Xv2NanoAODINPUT+NANOEDMMC2017_94XMiniAODv2+HARVESTNANOAODMC2017_94XMiniAODv2/step2_TTbar_13_94Xv2NanoAODINPUT+TTbar_13_94Xv2NanoAODINPUT+NANOEDMMC2017_94XMiniAODv2+HARVESTNANOAODMC2017_94XMiniAODv2.log

1000.0 step2

runTheMatrix-results/1000.0_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT/step2_RunMinBias2011A+RunMinBias2011A+TIER0+SKIMD+HARVESTDfst2+ALCASPLIT.log

1001.0 step2

runTheMatrix-results/1001.0_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5/step2_RunMinBias2011A+RunMinBias2011A+TIER0EXP+ALCAEXP+ALCAHARVDSIPIXELCALRUN1+ALCAHARVD1+ALCAHARVD2+ALCAHARVD3+ALCAHARVD4+ALCAHARVD5.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM+NANOUP15/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM+NANOUP15.log

158.0 step3

runTheMatrix-results/158.0_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO/step3_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO.log

10042.0 step3

runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017/step3_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017.log

10024.0 step3

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017.log

10824.0 step3

runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018.log

11634.0 step3

runTheMatrix-results/11634.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021.log

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25+NANOUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25+NANOUP15_PU25.log

12434.0 step3

runTheMatrix-results/12434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023.log

10224.0 step3

runTheMatrix-results/10224.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU+NanoFull_2017PU/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU+NanoFull_2017PU.log

20034.0 step3

runTheMatrix-results/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D35+RecoFullGlobal_2026D35+HARVESTFullGlobal_2026D35/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D35+RecoFullGlobal_2026D35+HARVESTFullGlobal_2026D35.log

23234.0 step3

runTheMatrix-results/23234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D49_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D49+RecoFullGlobal_2026D49+HARVESTFullGlobal_2026D49/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D49_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D49+RecoFullGlobal_2026D49+HARVESTFullGlobal_2026D49.log

20434.0 step3

runTheMatrix-results/20434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D41+RecoFullGlobal_2026D41+HARVESTFullGlobal_2026D41/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D41+RecoFullGlobal_2026D41+HARVESTFullGlobal_2026D41.log

21234.0 step3

runTheMatrix-results/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D44_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D44+RecoFullGlobal_2026D44+HARVESTFullGlobal_2026D44/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D44_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D44+RecoFullGlobal_2026D44+HARVESTFullGlobal_2026D44.log

250202.181 step4

runTheMatrix-results/250202.181_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25/step4_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25.log

cmsbuild · 2020-02-18T13:53:59Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

slava77 · 2020-02-18T16:18:34Z

test parameters

pull_request = Including weights for 94X and 102X PUjetID training cms-data/RecoJets-JetProducers#10

slava77 · 2020-02-18T16:19:00Z

@cmsbuild please test

cmsbuild · 2020-02-20T17:26:40Z

Pull request #28984 was updated. @perrotta, @cmsbuild, @slava77 can you please check and sign again.

slava77 · 2020-02-20T18:43:24Z

@cmsbuild please test

cmsbuild · 2020-02-20T18:43:47Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-data/RecoJets-JetProducers#10
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4803/console Started: 2020/02/20 19:45

cmsbuild · 2020-02-20T20:26:33Z

+1
Tested at: 0e644b8
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0ba9ea/4803/summary.html
CMSSW: CMSSW_11_1_X_2020-02-19-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-02-20T20:26:39Z

Comparison job queued.

cmsbuild · 2020-02-20T21:50:29Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0ba9ea/4803/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 45 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2694086
DQMHistoTests: Total failures: 1032
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2692735
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

lathomas · 2020-02-21T05:44:26Z

Hello,
For the record the changes in the PU ID output now look as expected.
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_11_1_X_2020-02-19-2300+0ba9ea/35177/validateJR/all_OldVSNew_RunJetHT2017FreMINIAODwf136p8311/all_OldVSNew_RunJetHT2017FreMINIAODwf136p8311c_patJets_slimmedJets__PAT_obj___userFloats__3_.png

slava77 · 2020-02-24T19:55:25Z

@alefisico
please edit the PR description to add a link to some slides showing expected changes in performance.

https://twiki.cern.ch/twiki/bin/viewauth/CMS/PileupJetID points to https://indico.cern.ch/event/860457/#17-pujetid-training-and-valida . I'm guessing it's the latest on this subject.

slava77 · 2020-02-24T19:58:12Z

+1

for #28984 0e644b8

code changes are in line with the PR description and the follow up review [most of the review history was in Adding parameters needed for PUjetID 94X and 102X #28827]
jenkins tests pass and comparisons with the baseline show differences only in jet PU ID variables. The changes look roughly as expected from https://indico.cern.ch/event/860457/#17-pujetid-training-and-valida and were confirmed to be as expected earlier in Adding parameters needed for PUjetID 94X and 102X v2 #28984 (comment)

cmsbuild · 2020-02-24T19:58:35Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @silviodonato, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-02-25T09:35:44Z

+1

fix PUID order in variables

76fe14d

cmsbuild added this to the CMSSW_11_1_X milestone Feb 18, 2020

cmsbuild added code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Feb 18, 2020

alefisico mentioned this pull request Feb 18, 2020

Adding parameters needed for PUjetID 94X and 102X #28827

Closed

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 18, 2020

alefisico mentioned this pull request Feb 18, 2020

Add pileup jet ID discriminant + add 94X and 102X training cms-nanoAOD/cmssw#493

Merged

cmsbuild added tests-started and removed tests-pending labels Feb 18, 2020

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Feb 18, 2020

cmsbuild removed comparison-notrun tests-rejected labels Feb 18, 2020

cmsbuild added the code-checks-approved label Feb 20, 2020

cmsbuild added requires-external tests-started and removed tests-pending labels Feb 20, 2020

cmsbuild added tests-approved and removed tests-started labels Feb 20, 2020

cmsbuild added comparison-available and removed comparison-pending labels Feb 20, 2020

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Feb 24, 2020

cmsbuild added orp-approved and removed orp-pending labels Feb 25, 2020

cmsbuild merged commit 4faa3a2 into cms-sw:master Feb 25, 2020

This was referenced Feb 26, 2020

Adding parameters needed for PUjetID 94X and 102X (backport to 102X) #29044

Merged

Adding parameters needed for PUjetID 94X and 102X (backport to 106X) #29045

Merged

This was referenced Apr 2, 2020

fix miniaod_devel modifier use for pileupJetId.algos #29376

Merged

fix miniaod_devel modifier use for pileupJetId.algos #29377

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding parameters needed for PUjetID 94X and 102X v2 #28984

Adding parameters needed for PUjetID 94X and 102X v2 #28984

alefisico commented Feb 18, 2020 •

edited

Loading

cmsbuild commented Feb 18, 2020

cmsbuild commented Feb 18, 2020

cmsbuild commented Feb 18, 2020

slava77 commented Feb 18, 2020

cmsbuild commented Feb 18, 2020 •

edited

Loading

alefisico commented Feb 18, 2020

cmsbuild commented Feb 18, 2020

cmsbuild commented Feb 18, 2020

slava77 commented Feb 18, 2020

slava77 commented Feb 18, 2020

cmsbuild commented Feb 20, 2020

slava77 commented Feb 20, 2020

cmsbuild commented Feb 20, 2020 •

edited

Loading

cmsbuild commented Feb 20, 2020

cmsbuild commented Feb 20, 2020

cmsbuild commented Feb 20, 2020

lathomas commented Feb 21, 2020

slava77 commented Feb 24, 2020

slava77 commented Feb 24, 2020

cmsbuild commented Feb 24, 2020

silviodonato commented Feb 25, 2020

Adding parameters needed for PUjetID 94X and 102X v2 #28984

Adding parameters needed for PUjetID 94X and 102X v2 #28984

Conversation

alefisico commented Feb 18, 2020 • edited Loading

PR description:

cmsbuild commented Feb 18, 2020

cmsbuild commented Feb 18, 2020

cmsbuild commented Feb 18, 2020

slava77 commented Feb 18, 2020

cmsbuild commented Feb 18, 2020 • edited Loading

alefisico commented Feb 18, 2020

cmsbuild commented Feb 18, 2020

cmsbuild commented Feb 18, 2020

slava77 commented Feb 18, 2020

slava77 commented Feb 18, 2020

cmsbuild commented Feb 20, 2020

slava77 commented Feb 20, 2020

cmsbuild commented Feb 20, 2020 • edited Loading

cmsbuild commented Feb 20, 2020

cmsbuild commented Feb 20, 2020

cmsbuild commented Feb 20, 2020

lathomas commented Feb 21, 2020

slava77 commented Feb 24, 2020

slava77 commented Feb 24, 2020

cmsbuild commented Feb 24, 2020

silviodonato commented Feb 25, 2020

alefisico commented Feb 18, 2020 •

edited

Loading

cmsbuild commented Feb 18, 2020 •

edited

Loading

cmsbuild commented Feb 20, 2020 •

edited

Loading