track selection DNN update and relative working points

[leonardogiannini]

PR description:

• the track selection DNN is updated with a retraining under more recent Run3 conditions (CMSSW 12_5_0_pre5) and relative working points
• two trainings are performed: mkFit for 7 iterations + CKF for the rest and CKF for all the iterations
• for the current tracking, where mkFit is used in 4 iterations + InitialStepPreSplitting (5), the first DNN is used for the 4 mkFit iterations (initial , high pt triplet, detached quad and triplet) and the iterations where both trainings use CKF (pixel pair, mixed triplet, tobtec, jetCore), while the CKF training and working points is used as a fallback for pixelLess, low pT quad and triplet, trained with mkFit in the first training.
• the update was presented at the TRK POG -> link

PR validation:

the MTV plots for TTbar and QCD high pT (1800-2400) are linked below

• default tracking
  TTbar default
  QCD default

• pure CKF
  TTbar CKF
  QCD CKF

NB: the cand DNN cut for mkFit high pT triplet is kept the same, as it doesn't change the performance (same as before retraining)

for testing one needs to include this PR to cms-data
cms-data/RecoTracker-FinalTrackSelectors#12

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-39715/32547

• This PR adds an extra 24KB to repository

[cmsbuild]

A new Pull Request was created by @leonardogiannini for master.

It involves the following packages:

• RecoTracker/FinalTrackSelectors (reconstruction)
• RecoTracker/IterativeTracking (reconstruction)

@cmsbuild, @mandrenguyen, @clacaputo can you please review it and eventually sign? Thanks.
@VourMa, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @missirol, @ebrondol, @gpetruc, @mmusich, @mtosi, @dgulhan this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

[mmusich]

test parameters:

• pull_request = Add retraining in 12_5_0_pre5 cms-data/RecoTracker-FinalTrackSelectors#12

[mmusich]

type tracking

[mmusich]

please test

[cmsbuild]

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-06172f/28219/summary.html
COMMIT: bef89f3
CMSSW: CMSSW_12_6_X_2022-10-12-2300/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39715/28219/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @bsunanda Run3-alca231 Update some of the defaults for Muon analysis code in HCalAlCaRecoProducers and HcalCalibAlgos #39687

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-06172f/28219/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-06172f/28219/git-merge-result

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-06172f/20834.0_TTbar_14TeV+2026D88+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-06172f/20834.75_TTbar_14TeV+2026D88_HLT75e33+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HLT75e33+HARVESTGlobal
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-06172f/20896.0_CloseByPGun_CE_E_Front_120um+2026D88+CE_E_Front_120um_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-06172f/20900.0_CloseByPGun_CE_H_Coarse_Scint+2026D88+CE_H_Coarse_Scint_GenSimHLBeamSpotHGCALCloseBy+DigiTrigger+RecoGlobal+HARVESTGlobal
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-06172f/21034.999_TTbar_14TeV+2026D88PU_PMXS1S2PR+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+PREMIX_PremixHLBeamSpot14PU+DigiTriggerPU+RecoGlobalPU+HARVESTGlobalPU
• /data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-06172f/23234.0_TTbar_14TeV+2026D94+TTbar_14TeV_TuneCP5_GenSimHLBeamSpot14+DigiTrigger+RecoGlobal+HARVESTGlobal

Summary:

• No significant changes to the logs found
• Reco comparison results: 20456 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3392309
• DQMHistoTests: Total failures: 111888
• DQMHistoTests: Total nulls: 2
• DQMHistoTests: Total successes: 3280397
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: -0.016 KiB( 48 files compared)
• DQMHistoSizes: changed ( 138.5 ): -0.016 KiB JetMET/SUSYDQM
• Checked 204 log files, 49 edm output root files, 49 DQM output files
• TriggerResults: found differences in 2 / 48 workflows

[mmusich]

@cms-sw/reconstruction-l2 do you have any comment on this PR?

[clacaputo]

Hi, I'm checking the reco differences and they are minimal. I see a ~1% decrease in the number of general tracks, maybe it is expected, could you please confirm it?

[slava77]

Hi, I'm checking the reco differences and they are minimal. I see a ~1% decrease in the number of general tracks, maybe it is expected, could you please confirm it?

which workflow is this?
In events with PU the number of fake tracks is expected to be lower, as shown in the slides linked in the PR description.
A link to the internal validation (available in the slides), somewhat representative to the inclusive distribution you have picked up is in http://uaf-10.t2.ucsd.edu/~legianni/DNN-retraining-eval/MKWP/TTbar_epoch4-4/plots_ootb/distPtEtaPhi.pdf

[clacaputo]

which workflow is this?

Sorry, it's 11834p0

[slava77]

which workflow is this?

Sorry, it's 11834p0

OK, then my link is supposed to be equivalent (ttbar with PU in run3)

[clacaputo]

which workflow is this?

Sorry, it's 11834p0

OK, then my link is supposed to be equivalent (ttbar with PU in run3)

Indeed, I can see the decrease in number of tracks, thanks

[clacaputo]

+reconstruction

• track selection DNN update
• new model in Add retraining in 12_5_0_pre5 cms-data/RecoTracker-FinalTrackSelectors#12
• reco changes expected

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[rappoccio]

@clacaputo do you mind also signing the external (cms-data/RecoTracker-FinalTrackSelectors#12) and then this can be merged.

[slava77]

@rappoccio @perrotta
it looks like the external PR was signed by reco.

[perrotta]

+1