update working point for new DNN #36285

minxiyang · 2021-11-29T17:11:39Z

Update working point of the MkFit DNN for the newest update of the MkFit reconstruction: #36246.

The performance of the updated DNN: https://jschulte.web.cern.ch/jschulte/DNN_Nov29th/run3finalv4_TTbar/

The test is ran on the dataset /RelValTTbar_14TeV/CMSSW_12_1_0_pre5-PU_121X_mcRun3_2021_realistic_v15-v2/GEN-SIM-DIGI-RAW with the CMSSW _12_2_X. The performance of the MkFit DNN on the MkFit tracks is compared to the CKF DNN on the CKF dataset.

cmsbuild · 2021-11-29T17:18:23Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36285/26980

This PR adds an extra 12KB to repository

cmsbuild · 2021-11-29T17:18:49Z

A new Pull Request was created by @minxiyang (Minxi Yang) for master.

It involves the following packages:

RecoTracker/IterativeTracking (reconstruction)

@jpata, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @ebrondol, @gpetruc, @mmusich, @mtosi, @dgulhan this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

slava77 · 2021-11-29T18:35:15Z

test parameters:

pull_request = update MkFit DNN cms-data/RecoTracker-FinalTrackSelectors#11

slava77 · 2021-11-29T18:36:12Z

enable profiling

slava77 · 2021-11-29T18:36:22Z

@cmsbuild please test

cmsbuild · 2021-11-29T22:17:30Z

-1

Failed Tests: RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5790f3/20838/summary.html
COMMIT: e29fccd
CMSSW: CMSSW_12_2_X_2021-11-29-1100/slc7_amd64_gcc900
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

312.0

----- Begin Fatal Exception 29-Nov-2021 20:25:17 CET-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=GsfElectronProducer label='gedGsfElectronsTmp'
Exception Message:
edm::FileInPath unable to find file RecoEgamma/ElectronIdentification/data/Ele_PFID_dnn/lowpT/lowpT_modelDNN.pb anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_2_X_2021-11-29-1100/poison:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_2_X_2021-11-29-1100/src
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/312.0_Pyquen_ZeemumuJets_pt10_2760GeV_2021+Pyquen_ZeemumuJets_pt10_2760GeV_2021+DIGIHI2021MIX+RECOHI2021MIX+HARVESTHI2021PPRECO
----- End Fatal Exception -------------------------------------------------

11634.914

----- Begin Fatal Exception 29-Nov-2021 20:36:42 CET-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=GsfElectronProducer label='gedGsfElectronsTmp'
Exception Message:
edm::FileInPath unable to find file RecoEgamma/ElectronIdentification/data/Ele_PFID_dnn/lowpT/lowpT_modelDNN.pb anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_2_X_2021-11-29-1100/poison:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_2_X_2021-11-29-1100/src
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/11634.914_TTbar_14TeV+2021_DDDDB+TTbar_14TeV_TuneCP5_GenSim+Digi+Reco+HARVEST+ALCA
----- End Fatal Exception -------------------------------------------------

11634.0

----- Begin Fatal Exception 29-Nov-2021 20:37:24 CET-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=GsfElectronProducer label='gedGsfElectronsTmp'
Exception Message:
edm::FileInPath unable to find file RecoEgamma/ElectronIdentification/data/Ele_PFID_dnn/lowpT/lowpT_modelDNN.pb anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_2_X_2021-11-29-1100/poison:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_2_X_2021-11-29-1100/src
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-relvals/runTheMatrix-results/11634.0_TTbar_14TeV+2021+TTbar_14TeV_TuneCP5_GenSim+Digi+RecNan+HARVESTRecNan+ALCA
----- End Fatal Exception -------------------------------------------------

Expand to see more relval errors ...

RelVals-INPUT

138.4138.4_PromptCollisions+RunMinimumBias2021+ALCARECOPROMPTR3+HARVESTDPROMPTR3/step2_PromptCollisions+RunMinimumBias2021+ALCARECOPROMPTR3+HARVESTDPROMPTR3.log
138.5138.5_ExpressCollisions+RunMinimumBias2021+TIER0EXPRUN3+ALCARECOEXPR3+HARVESTDEXPR3/step2_ExpressCollisions+RunMinimumBias2021+TIER0EXPRUN3+ALCARECOEXPR3+HARVESTDEXPR3.log

AddOn Tests

hlt_mc_GRun

----- Begin Fatal Exception 29-Nov-2021 20:22:26 CET-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=GsfElectronProducer label='gedGsfElectronsTmp'
Exception Message:
edm::FileInPath unable to find file RecoEgamma/ElectronIdentification/data/Ele_PFID_dnn/lowpT/lowpT_modelDNN.pb anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_2_X_2021-11-29-1100/poison:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_2_X_2021-11-29-1100/src
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-addon/addOnTests/hlt_mc_GRun
----- End Fatal Exception -------------------------------------------------

hlt_mc_HIon

----- Begin Fatal Exception 29-Nov-2021 20:19:49 CET-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=GsfElectronProducer label='gedGsfElectronsTmp'
Exception Message:
edm::FileInPath unable to find file RecoEgamma/ElectronIdentification/data/Ele_PFID_dnn/lowpT/lowpT_modelDNN.pb anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_2_X_2021-11-29-1100/poison:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_2_X_2021-11-29-1100/src
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-addon/addOnTests/hlt_mc_HIon
----- End Fatal Exception -------------------------------------------------

hlt_mc_PIon

----- Begin Fatal Exception 29-Nov-2021 20:19:03 CET-----------------------
An exception of category 'FileInPathError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing module: class=GsfElectronProducer label='gedGsfElectronsTmp'
Exception Message:
edm::FileInPath unable to find file RecoEgamma/ElectronIdentification/data/Ele_PFID_dnn/lowpT/lowpT_modelDNN.pb anywhere in the search path.
The search path is defined by: CMSSW_SEARCH_PATH
${CMSSW_SEARCH_PATH} is: /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/poison:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/src:/cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20838/CMSSW_12_2_X_2021-11-29-1100/external/slc7_amd64_gcc900/data:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_2_X_2021-11-29-1100/poison:/cvmfs/cms-ib.cern.ch/week1/slc7_amd64_gcc900/cms/cmssw-patch/CMSSW_12_2_X_2021-11-29-1100/src
Current directory is: /data/cmsbld/jenkins/workspace/ib-run-pr-addon/addOnTests/hlt_mc_PIon
----- End Fatal Exception -------------------------------------------------

Expand to see more addon errors ...

mmasciov · 2021-11-29T22:37:22Z

@minxiyang, @mmusich, @slava77: please consider suggestion in cms-data/RecoTracker-FinalTrackSelectors#11 (comment)

slava77 · 2021-11-30T15:46:48Z

@cmsbuild please test

cmsbuild · 2021-11-30T16:24:18Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-36285/27026

This PR adds an extra 12KB to repository

cmsbuild · 2021-11-30T16:24:39Z

Pull request #36285 was updated. @jpata, @cmsbuild, @slava77 can you please check and sign again.

slava77 · 2021-11-30T16:29:03Z

@cmsbuild please test

cmsbuild · 2021-11-30T23:12:41Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5790f3/20874/summary.html
COMMIT: 83c6517
CMSSW: CMSSW_12_2_X_2021-11-29-2300/slc7_amd64_gcc900
Additional Tests: PROFILING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/36285/20874/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 16481 differences found in the comparisons
DQMHistoTests: Total files compared: 41
DQMHistoTests: Total histograms compared: 3042214
DQMHistoTests: Total failures: 23195
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3018997
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 40 files compared)
Checked 175 log files, 37 edm output root files, 41 DQM output files
TriggerResults: no differences found

slava77 · 2021-12-01T17:08:29Z

+reconstruction

for #36285 83c6517

code changes are in line with the PR description
jenkins tests pass and comparisons with the baseline show differences in tracks entering the generalTracks collection and downstream quantities.
- technical performance as seen in the profiler reports are roughly unchanged (the memory use and the time per event are similar/same)
- track validation plots seem rather similar in these tests, but there are just 10 events in each case
- somewhat higher stat wf with data EGamma2018C 136.874 shows roughly expected behavior (to have slightly lower fakes):
  - the number of generalTracks is down by about 1.7% (expected to be mainly from fakes reduction)
  - the number of charged hadrons is down by about 0.6%
CKF to mkFit comparisons are provided in the PR description for the standard relval ttbar with PU (IIUC, prior to the update in 83c6517)
incremental comparisons corresponding to the latest/83c6517 version of the PR are available including Update mkFit for CMSSW_12_2_0_pre3 #36246 in https://indico.cern.ch/event/1098365/#6-mkfit-status-report e.g. in

cmsbuild · 2021-12-01T17:08:50Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

slava77 · 2021-12-02T03:07:34Z

@perrotta @qliphy
this together with cms-data/RecoTracker-FinalTrackSelectors#11 are expected in 12_2_0_pre3

perrotta · 2021-12-02T07:41:42Z

+1

Updated working points with slightly improved physics performance

update working point for new DNN

e29fccd

cmsbuild added this to the CMSSW_12_2_X milestone Nov 29, 2021

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Nov 29, 2021

minxiyang mentioned this pull request Nov 29, 2021

update MkFit DNN cms-data/RecoTracker-FinalTrackSelectors#11

Merged

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 29, 2021

cmsbuild added the requires-external label Nov 29, 2021

cmsbuild added tests-started and removed tests-pending labels Nov 29, 2021

cmsbuild added tests-rejected and removed tests-started labels Nov 29, 2021

cmsbuild added tests-started and removed tests-rejected labels Nov 30, 2021

update pixelLess working point

83c6517

cmsbuild added code-checks-pending tests-pending and removed tests-started code-checks-approved labels Nov 30, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 30, 2021

cmsbuild added tests-started and removed tests-pending labels Nov 30, 2021

cmsbuild added tests-approved and removed tests-started labels Nov 30, 2021

JanFSchulte mentioned this pull request Dec 1, 2021

[12.1.X] Update of mkFit as in 12_2_0_pre3 (backport of PR #36246) #36315

Merged

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Dec 1, 2021

JanFSchulte mentioned this pull request Dec 1, 2021

backport DNN cuts for mkFit 12_2_0_pre3 #36327

Merged

cmsbuild added orp-approved and removed orp-pending labels Dec 2, 2021

cmsbuild merged commit 7851d0c into cms-sw:master Dec 2, 2021

smuzaffar mentioned this pull request Dec 2, 2021

Update tag for RecoTracker-FinalTrackSelectors to V01-03-00 cms-sw/cmsdist#7485

Merged

JanFSchulte mentioned this pull request Dec 2, 2021

Update RecoTracker-FinalTrackSelectors tag (backport of #36285) cms-sw/cmsdist#7486

Merged

This was referenced Dec 2, 2021

[DQM] nanoDQMIO change in V0.1 #36331

Merged

Move to root 6.24 cms-sw/cmsdist#7445

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update working point for new DNN #36285

update working point for new DNN #36285

minxiyang commented Nov 29, 2021

cmsbuild commented Nov 29, 2021

cmsbuild commented Nov 29, 2021

slava77 commented Nov 29, 2021

slava77 commented Nov 29, 2021

slava77 commented Nov 29, 2021

cmsbuild commented Nov 29, 2021

mmasciov commented Nov 29, 2021

slava77 commented Nov 30, 2021

cmsbuild commented Nov 30, 2021

cmsbuild commented Nov 30, 2021

slava77 commented Nov 30, 2021

cmsbuild commented Nov 30, 2021

slava77 commented Dec 1, 2021

cmsbuild commented Dec 1, 2021

slava77 commented Dec 2, 2021

perrotta commented Dec 2, 2021

update working point for new DNN #36285

update working point for new DNN #36285

Conversation

minxiyang commented Nov 29, 2021

cmsbuild commented Nov 29, 2021

cmsbuild commented Nov 29, 2021

slava77 commented Nov 29, 2021

slava77 commented Nov 29, 2021

slava77 commented Nov 29, 2021

cmsbuild commented Nov 29, 2021

RelVals

RelVals-INPUT

AddOn Tests

mmasciov commented Nov 29, 2021

slava77 commented Nov 30, 2021

cmsbuild commented Nov 30, 2021

cmsbuild commented Nov 30, 2021

slava77 commented Nov 30, 2021

cmsbuild commented Nov 30, 2021

Comparison Summary

slava77 commented Dec 1, 2021

cmsbuild commented Dec 1, 2021

slava77 commented Dec 2, 2021

perrotta commented Dec 2, 2021