Code to process phase2 version of deepTauID v2p5 [12_5_X] #40723

mbluj · 2023-02-08T12:57:52Z

PR description:

This PR adds code for a phase-2 version of DeepTau discriminator; it is a backport of #40622.
The neural network structure of the new phase-2 version is identical to that of Run-2/3 DeepTau v2p5, therefore code modifications are small. The changes include:

Adding scaling parameters and working points for the new training;
A new "year" identifier introduced to separate this new phase-2 training and current Run-2/3 v2p5 (2026 vs 2018);
The electron input collection used for the new phase2 deepTauID is sum of two collections: slimmedElectrons (gsfElectrons in EB) and slimmedElectronsHGC (HGCal electrons in endcaps). Notes: 1. The latter collection is called slimmedElectronsFromMultiCl in older CMSSW release series (and samples produced with them) which can cause some problems with running on old samples; 2. The same merged electron collection is used by old-style anti-electron tau discriminant for phase-2;

The performance of this new phase-2 DeepTau discriminator is documented in AN-22-090.

The new data-model was merged to cms-data in the following PR: cms-data/RecoTauTag-TrainingFiles#11

This PR allows to run the new version of DeepTau discriminator in private workflows (to analyse phase-2 samples with 12_5_X), but it does not enable the new version of DeepTau in official workflows. Therefore, differences in outputs are not expected.

To run standalone test, one can run the configuration file RecoTauTag/RecoTau/test/runDeepTauIDsOnMiniAOD.py after setting the flag phase2 = True in that file.

PR validation:

Validated with the standalone test mentioned above and with miniAOD workflows.

Backport PR

This is backport of #40622 to 12_5_X to facilitate tau studies for phase-2 with 12_5_X samples.

cmsbuild · 2023-02-08T12:58:16Z

A new Pull Request was created by @mbluj for CMSSW_12_5_X.

It involves the following packages:

PhysicsTools/PatAlgos (xpog, reconstruction)
RecoTauTag/RecoTau (reconstruction)

@cmsbuild, @mandrenguyen, @clacaputo, @swertz, @vlimant can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @hatakeyamak, @emilbols, @mbluj, @demuller, @seemasharmafnal, @mmarionncern, @missirol, @ahinzmann, @jdolen, @azotz, @jdamgov, @nhanvtran, @gkasieczka, @schoef, @andrzejnovak, @AlexDeMoor, @AnnikaStein, @JyothsnaKomaragiri, @gpetruc, @mariadalfonso this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

mandrenguyen · 2023-02-08T13:08:12Z

type tau

mandrenguyen · 2023-02-08T13:08:55Z

please test

cmsbuild · 2023-02-08T15:52:36Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-033c57/30503/summary.html
COMMIT: b588e8c
CMSSW: CMSSW_12_5_X_2023-02-05-0000/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40723/30503/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 2 lines from the logs
Reco comparison results: 20 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3724047
DQMHistoTests: Total failures: 25
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3723999
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -0.004 KiB( 50 files compared)
DQMHistoSizes: changed ( 312.0 ): -0.004 KiB MessageLogger/Warnings
Checked 216 log files, 167 edm output root files, 51 DQM output files
TriggerResults: no differences found

swertz · 2023-02-08T16:09:59Z

+1

mandrenguyen · 2023-02-09T18:00:40Z

+1

cmsbuild · 2023-02-09T18:01:08Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_5_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_13_0_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

mbluj · 2023-02-10T09:19:53Z

Just a comment: to make this PR useful also cms-dist of 12_5_X should be updated to contain cms-data state with data-model of the new phase-2 integrated in, i.e. with PR cms-data/RecoTauTag-TrainingFiles#11

perrotta · 2023-02-10T11:29:07Z

PhysicsTools/PatAlgos/python/slimming/miniAOD_tools.py

@@ -382,8 +382,8 @@ def _add_deepFlavour(process):
        toKeep = ['deepTau2017v2p1','deepTau2018v2p5']
    )
    from Configuration.Eras.Modifier_phase2_common_cff import phase2_common #Phase2 Tau MVA
-    phase2_common.toModify(tauIdEmbedder.toKeep, func=lambda t:t.append('newDMPhase2v1')) #Phase2 Tau isolation MVA
-    phase2_common.toModify(tauIdEmbedder.toKeep, func=lambda t:t.append('againstElePhase2v1')) #Phase2 Tau anti-e MVA
+    _tauIds_phase2 = ['newDMPhase2v1','againstElePhase2v1']


@mbluj newDMPhase2v1 was not kept in the master version. Just to try to understand: why?

@mbluj newDMPhase2v1 was not kept in the master version. Just to try to understand: why?

It has quite likely something to do with #40724 (comment), but I fail to find the connection (sorry...)

The idea is to to keep in master only phase-2 deepTauID ('deepTau2026v2p5') and remove/depreciate older, less powerful phase-2 tauIDs ('newDMPhase2v1' and 'againstElePhase2v1').

perrotta · 2023-02-10T11:32:49Z

please test with cms-sw/cmsdist#8308

cmsbuild · 2023-02-10T16:33:55Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-033c57/30578/summary.html
COMMIT: b588e8c
CMSSW: CMSSW_12_5_X_2023-02-05-0000/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40723/30578/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

The relvals timed out after 4 hours.

Comparison Summary

Summary:

You potentially added 5 lines to the logs
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3724047
DQMHistoTests: Total failures: 8
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3724017
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 216 log files, 167 edm output root files, 51 DQM output files
TriggerResults: no differences found

perrotta · 2023-02-10T17:12:50Z

+1

perrotta · 2023-02-10T17:13:35Z

merge

DennRoy and others added 13 commits January 27, 2023 13:41

Adding code to process Phase 2 DeepTau v2p5

0746c9f

Fix variable declaration

dbfd02f

Small fix

c3ae7ed

Add recipe for testing Phase2 DeepTau

e549328

Update testing configurations

ccbbb3b

Fix name of electron merger

c33e223

Integrate Phase2 DeepTau in MiniAOD

8575474

Fix geometry for Phase2

059c9c4

Common merged collection of phase-2 electrons for tauID

aca14d9

Fix phase2 tauIDs for miniAOD

d992c96

code-format fixes

40a1eb8

disable DeepTau for phase2 and re-enable old-style phase2 tauIDs

f7f280f

Use phase2 geometry accessible in 12_5

b588e8c

cmsbuild added this to the CMSSW_12_5_X milestone Feb 8, 2023

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending xpog-pending labels Feb 8, 2023

cmsbuild added the tau label Feb 8, 2023

cmsbuild added tests-started and removed tests-pending labels Feb 8, 2023

cmsbuild added tests-approved and removed tests-started labels Feb 8, 2023

cmsbuild removed the xpog-pending label Feb 8, 2023

cmsbuild added the xpog-approved label Feb 8, 2023

cmsbuild added reconstruction-approved fully-signed and removed reconstruction-pending pending-signatures labels Feb 9, 2023

perrotta reviewed Feb 10, 2023

View reviewed changes

cmsbuild added requires-external tests-started and removed tests-approved labels Feb 10, 2023

cmsbuild added tests-rejected and removed tests-started labels Feb 10, 2023

perrotta mentioned this pull request Feb 10, 2023

[12_5_X] Update tag for RecoTauTag-TrainingFiles to V00-07-00 cms-sw/cmsdist#8308

Merged

cmsbuild added orp-approved and removed orp-pending labels Feb 10, 2023

cmsbuild merged commit 1c0198a into cms-sw:CMSSW_12_5_X Feb 10, 2023

mbluj deleted the CMSSW_12_5_X_tau-pog_deepTauPh2 branch October 10, 2023 10:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Code to process phase2 version of deepTauID v2p5 [12_5_X] #40723

Code to process phase2 version of deepTauID v2p5 [12_5_X] #40723

mbluj commented Feb 8, 2023

cmsbuild commented Feb 8, 2023

mandrenguyen commented Feb 8, 2023

mandrenguyen commented Feb 8, 2023

cmsbuild commented Feb 8, 2023

swertz commented Feb 8, 2023

mandrenguyen commented Feb 9, 2023

cmsbuild commented Feb 9, 2023

mbluj commented Feb 10, 2023

perrotta Feb 10, 2023

perrotta Feb 10, 2023

mbluj Feb 10, 2023

perrotta commented Feb 10, 2023

cmsbuild commented Feb 10, 2023

perrotta commented Feb 10, 2023

perrotta commented Feb 10, 2023

Code to process phase2 version of deepTauID v2p5 [12_5_X] #40723

Code to process phase2 version of deepTauID v2p5 [12_5_X] #40723

Conversation

mbluj commented Feb 8, 2023

PR description:

PR validation:

Backport PR

cmsbuild commented Feb 8, 2023

mandrenguyen commented Feb 8, 2023

mandrenguyen commented Feb 8, 2023

cmsbuild commented Feb 8, 2023

Comparison Summary

swertz commented Feb 8, 2023

mandrenguyen commented Feb 9, 2023

cmsbuild commented Feb 9, 2023

mbluj commented Feb 10, 2023

perrotta Feb 10, 2023

Choose a reason for hiding this comment

perrotta Feb 10, 2023

Choose a reason for hiding this comment

mbluj Feb 10, 2023

Choose a reason for hiding this comment

perrotta commented Feb 10, 2023

cmsbuild commented Feb 10, 2023

RelVals-INPUT

Comparison Summary

perrotta commented Feb 10, 2023

perrotta commented Feb 10, 2023