Add ONNX models of DeepJet and DeepBoostedJet #25

hqucms · 2019-10-03T16:31:09Z

No description provided.

cmsbuild · 2019-10-03T16:31:31Z

A new Pull Request was created by @hqucms (Huilin Qu) for branch master.

@perrotta, @smuzaffar, @cmsbuild, @mrodozov, @slava77 can you please review it and eventually sign? Thanks.
@emilbols, @smoortga, @riga, @HeinerTholen, @JyothsnaKomaragiri, @mverzett, @ferencek, @andrzejnovak, @pvmulder, @acaudron this is something you requested to watch as well.
cms-bot commands are listed here

cmsbuild · 2019-10-04T13:19:57Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmssw#28112,cms-sw/cmsdist#5259
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2786/console Started: 2019/10/04 15:21

cmsbuild · 2019-10-04T15:18:45Z

-1

Tested at: 4b7d7ce

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd181f/2786/summary.html

I found follow errors while testing this PR

Failed tests: UnitTests RelVals

Unit Tests:

I found errors in the following unit tests:

---> test runtestPhysicsToolsNanoAOD had ERRORS

RelVals:

When I ran the RelVals I found an error in the following workflows:
136.8311 step2

runTheMatrix-results/136.8311_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017/step2_RunJetHT2017F_reminiaod+RunJetHT2017F_reminiaod+REMINIAOD_data2017+HARVEST2017_REMINIAOD_data2017.log

136.7611 step2

runTheMatrix-results/136.7611_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM/step2_RunJetHT2016E_reminiaod+RunJetHT2016E_reminiaod+REMINIAOD_data2016_HIPM+HARVESTDR2_REMINIAOD_data2016_HIPM.log

4.53 step3

runTheMatrix-results/4.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step3_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log

135.4 step3

runTheMatrix-results/135.4_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS/step3_ZEE_13+ZEEFS_13+HARVESTUP15FS+MINIAODMCUP15FS.log

136.731 step3

runTheMatrix-results/136.731_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2/step3_RunSinglePh2016B+RunSinglePh2016B+HLTDR2_2016+RECODR2_2016reHLT_skimSinglePh_HIPM+HARVESTDR2.log

136.788 step3

runTheMatrix-results/136.788_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017/step3_RunSinglePh2017B+RunSinglePh2017B+HLTDR2_2017+RECODR2_2017reHLT_skimSinglePh_Prompt+HARVEST2017.log

136.85 step3

runTheMatrix-results/136.85_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM/step3_RunEGamma2018A+RunEGamma2018A+HLTDR2_2018+RECODR2_2018reHLT_skimEGamma_Offline_L1TEgDQM+HARVEST2018_L1TEgDQM.log

1325.7 step2

runTheMatrix-results/1325.7_TTbar_13_94Xv2NanoAODINPUT+TTbar_13_94Xv2NanoAODINPUT+NANOEDMMC2017_94XMiniAODv2+HARVESTNANOAODMC2017_94XMiniAODv2/step2_TTbar_13_94Xv2NanoAODINPUT+TTbar_13_94Xv2NanoAODINPUT+NANOEDMMC2017_94XMiniAODv2+HARVESTNANOAODMC2017_94XMiniAODv2.log

9.0 step3

runTheMatrix-results/9.0_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST/step3_Higgs200ChargedTaus+Higgs200ChargedTaus+DIGI+RECO+HARVEST.log

1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

25.0 step3

runTheMatrix-results/25.0_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT/step3_TTbar+TTbar+DIGI+RECOAlCaCalo+HARVEST+ALCATT.log

1330.0 step3

runTheMatrix-results/1330.0_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM+NANOUP15/step3_ZMM_13+ZMM_13+DIGIUP15+RECOUP15_L1TMuDQM+HARVESTUP15_L1TMuDQM+NANOUP15.log

158.0 step3

runTheMatrix-results/158.0_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO/step3_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO.log

10042.0 step3

runTheMatrix-results/10042.0_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017/step3_ZMM_13+ZMM_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017.log

10024.0 step3

runTheMatrix-results/10024.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017_GenSimFull+DigiFull_2017+RecoFull_2017+HARVESTFull_2017+ALCAFull_2017+NanoFull_2017.log

10824.0 step3

runTheMatrix-results/10824.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2018_GenSimFull+DigiFull_2018+RecoFull_2018+HARVESTFull_2018+ALCAFull_2018+NanoFull_2018.log

25202.0 step3

runTheMatrix-results/25202.0_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25+NANOUP15_PU25/step3_TTbar_13+TTbar_13+DIGIUP15_PU25+RECOUP15_PU25+HARVESTUP15_PU25+NANOUP15_PU25.log

11634.0 step3

runTheMatrix-results/11634.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2021_GenSimFull+DigiFull_2021+RecoFull_2021+HARVESTFull_2021+ALCAFull_2021.log

10224.0 step3

runTheMatrix-results/10224.0_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU+NanoFull_2017PU/step3_TTbar_13+TTbar_13TeV_TuneCUETP8M1_2017PU_GenSimFull+DigiFullPU_2017PU+RecoFullPU_2017PU+HARVESTFullPU_2017PU+NanoFull_2017PU.log

12434.0 step3

runTheMatrix-results/12434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2023_GenSimFull+DigiFull_2023+RecoFull_2023+HARVESTFull_2023+ALCAFull_2023.log

20034.0 step3

runTheMatrix-results/20034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D35+RecoFullGlobal_2026D35+HARVESTFullGlobal_2026D35/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D35_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D35+RecoFullGlobal_2026D35+HARVESTFullGlobal_2026D35.log

20434.0 step3

runTheMatrix-results/20434.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D41+RecoFullGlobal_2026D41+HARVESTFullGlobal_2026D41/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D41_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D41+RecoFullGlobal_2026D41+HARVESTFullGlobal_2026D41.log

21234.0 step3

runTheMatrix-results/21234.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D44_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D44+RecoFullGlobal_2026D44+HARVESTFullGlobal_2026D44/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D44_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D44+RecoFullGlobal_2026D44+HARVESTFullGlobal_2026D44.log

22034.0 step3

runTheMatrix-results/22034.0_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D46_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D46+RecoFullGlobal_2026D46+HARVESTFullGlobal_2026D46/step3_TTbar_14TeV+TTbar_14TeV_TuneCUETP8M1_2026D46_GenSimHLBeamSpotFull14+DigiFullTrigger_2026D46+RecoFullGlobal_2026D46+HARVESTFullGlobal_2026D46.log

250202.181 step4

runTheMatrix-results/250202.181_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25/step4_TTbar_13UP18+TTbar_13UP18+PREMIXUP18_PU25+DIGIPRMXLOCALUP18_PU25+RECOPRMXUP18_PU25+HARVESTUP18_PU25.log

cmsbuild · 2019-10-04T15:18:52Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

cmsbuild · 2019-10-05T05:40:11Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmssw#28112,cms-sw/cmsdist#5259
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2795/console Started: 2019/10/05 09:53

cmsbuild · 2019-10-05T10:28:32Z

-1

Tested at: 4b7d7ce

You can see the results of the tests here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd181f/2795/summary.html

I found follow errors while testing this PR

Failed tests: RelVals

RelVals:

When I ran the RelVals I found an error in the following workflows:
1306.0 step3

runTheMatrix-results/1306.0_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15/step3_SingleMuPt1_UP15+SingleMuPt1_UP15+DIGIUP15+RECOUP15+HARVESTUP15.log

158.0 step3

runTheMatrix-results/158.0_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO/step3_HydjetQ_B12_5020GeV_2018_ppReco+HydjetQ_B12_5020GeV_2018_ppReco+DIGIHI2018PPRECO+RECOHI2018PPRECO+ALCARECOHI2018PPRECO+HARVESTHI2018PPRECO.log

cmsbuild · 2019-10-05T10:28:37Z

Comparison not run due to runTheMatrix errors (RelVals and Igprof tests were also skipped)

cmsbuild · 2019-10-07T06:53:44Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmssw#28112,cms-sw/cmsdist#5259
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/2809/console Started: 2019/10/07 08:55

cmsbuild · 2019-10-19T12:32:29Z

Pull request #25 was updated.

cmsbuild · 2019-10-19T12:59:40Z

The tests are being triggered in jenkins.
Tested with other pull request(s) cms-sw/cmssw#28112,cms-sw/cmsdist#5259
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/3037/console Started: 2019/10/19 15:03

cmsbuild · 2019-10-19T16:11:59Z

+1
Tested at: 2ae8705
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd181f/3037/summary.html

cmsbuild · 2019-10-19T16:12:07Z

Comparison job queued.

cmsbuild · 2019-10-19T18:38:08Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cd181f/3037/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 3 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2961064
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2960722
DQMHistoTests: Total skipped: 341
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

slava77 · 2019-10-29T21:08:27Z

+1

for #25 2ae8705

new model files are consistently added with the other variants of inference engines
runtime/performance signoff is based on runtime tests of ONNXRuntime-based implementation of DeepJet, DeepAK8 and DeepDoubleX cms-sw/cmssw#28112

smuzaffar · 2019-10-29T21:08:57Z

+externals

cmsbuild · 2019-10-29T21:09:25Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

Add ONNX models of DeepJet and DeepBoostedJet.

4b7d7ce

cmsbuild added comparison-pending externals-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Oct 3, 2019

hqucms mentioned this pull request Oct 3, 2019

ONNXRuntime-based implementation of DeepJet, DeepAK8 and DeepDoubleX cms-sw/cmssw#28112

Merged

cmsbuild mentioned this pull request Oct 4, 2019

Update ONNXRuntime to a custom version cms-sw/cmsdist#5259

Merged

cmsbuild added requires-external tests-started and removed tests-pending labels Oct 4, 2019

cmsbuild added comparison-notrun tests-rejected and removed comparison-pending tests-started labels Oct 4, 2019

cmsbuild added tests-started and removed tests-rejected labels Oct 5, 2019

cmsbuild added tests-rejected and removed tests-started labels Oct 5, 2019

cmsbuild added tests-started and removed tests-rejected labels Oct 7, 2019

Add ONNX models of DeepDoubleX.

2ae8705

cmsbuild added comparison-pending tests-pending and removed comparison-available requires-external tests-approved labels Oct 19, 2019

cmsbuild added requires-external tests-started and removed tests-pending labels Oct 19, 2019

cmsbuild added tests-approved and removed tests-started labels Oct 19, 2019

cmsbuild added comparison-available and removed comparison-pending labels Oct 19, 2019

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Oct 29, 2019

cmsbuild added externals-approved fully-signed and removed externals-pending pending-signatures labels Oct 29, 2019

smuzaffar merged commit d838b42 into cms-data:master Oct 29, 2019

cmsbuild mentioned this pull request Oct 29, 2019

Update tag for RecoBTag-Combined to V01-02-01 cms-sw/cmsdist#5314

Merged

hqucms mentioned this pull request Jun 5, 2020

[10_6_X] ONNXRuntime-based implementation of DeepJet, DeepAK8 and DeepDoubleX cms-sw/cmssw#30123

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add ONNX models of DeepJet and DeepBoostedJet #25

Add ONNX models of DeepJet and DeepBoostedJet #25

hqucms commented Oct 3, 2019

cmsbuild commented Oct 3, 2019

cmsbuild commented Oct 4, 2019 •

edited

Loading

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 5, 2019 •

edited

Loading

cmsbuild commented Oct 5, 2019

cmsbuild commented Oct 5, 2019

cmsbuild commented Oct 7, 2019 •

edited

Loading

cmsbuild commented Oct 19, 2019

cmsbuild commented Oct 19, 2019 •

edited

Loading

cmsbuild commented Oct 19, 2019

cmsbuild commented Oct 19, 2019

cmsbuild commented Oct 19, 2019

slava77 commented Oct 29, 2019 •

edited

Loading

smuzaffar commented Oct 29, 2019

cmsbuild commented Oct 29, 2019

Add ONNX models of DeepJet and DeepBoostedJet #25

Add ONNX models of DeepJet and DeepBoostedJet #25

Conversation

hqucms commented Oct 3, 2019

cmsbuild commented Oct 3, 2019

cmsbuild commented Oct 4, 2019 • edited Loading

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 4, 2019

cmsbuild commented Oct 5, 2019 • edited Loading

cmsbuild commented Oct 5, 2019

cmsbuild commented Oct 5, 2019

cmsbuild commented Oct 7, 2019 • edited Loading

cmsbuild commented Oct 19, 2019

cmsbuild commented Oct 19, 2019 • edited Loading

cmsbuild commented Oct 19, 2019

cmsbuild commented Oct 19, 2019

cmsbuild commented Oct 19, 2019

slava77 commented Oct 29, 2019 • edited Loading

smuzaffar commented Oct 29, 2019

cmsbuild commented Oct 29, 2019

cmsbuild commented Oct 4, 2019 •

edited

Loading

cmsbuild commented Oct 5, 2019 •

edited

Loading

cmsbuild commented Oct 7, 2019 •

edited

Loading

cmsbuild commented Oct 19, 2019 •

edited

Loading

slava77 commented Oct 29, 2019 •

edited

Loading