Enabling the JetCore+DeepCore2.2.1 hybrid with MaxCand=30 (JC30DC) #43339

bouchamaouihichem · 2023-11-21T02:03:23Z

PR description:

JC30DC can now be used with the ProcModifier DeepCoreSeeding and is ready for validation.
Latest Track POG update 11/20: https://indico.cern.ch/event/1345175/contributions/5662814/attachments/2754969/4796681/DeepCore_TrackPOG_23_1120.pdf
Last merged PR and relevant material: DeepCore model 2.2.1 CMSSW_13_3_0_pre3 (fixed comments) #42851

PR validation:

Validation on workflow 11923 (high pT QCD PU) for Tracking performance and 11834 (inclusive TTBAR PU) for timing performance.

…th DeepCoreSeeding ProcModifiers

cmsbuild · 2023-11-21T02:08:58Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43339/37794

This PR adds an extra 16KB to repository

cmsbuild · 2023-11-21T02:09:26Z

A new Pull Request was created by @bouchamaouihichem for master.

It involves the following packages:

RecoTracker/IterativeTracking (reconstruction)

@cmsbuild, @jfernan2, @mandrenguyen can you please review it and eventually sign? Thanks.
@missirol, @mtosi, @mmusich, @VinInn, @GiacomoSguazzoni, @JanFSchulte, @rovere, @dgulhan, @gpetruc, @ebrondol, @VourMa, @felicepantaleo this is something you requested to watch as well.
@sextonkennedy, @rappoccio, @antoniovilela you are the release manager for this.

cms-bot commands are listed here

jfernan2 · 2023-11-21T08:47:34Z

please test

cmsbuild · 2023-11-21T11:25:03Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4a3e84/35973/summary.html
COMMIT: b2d0b7a
CMSSW: CMSSW_14_0_X_2023-11-20-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43339/35973/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 201 lines to the logs
Reco comparison results: 14 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3363868
DQMHistoTests: Total failures: 1394
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3362452
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 214 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

RecoTracker/IterativeTracking/python/JetCoreRegionalStep_cff.py

mmusich · 2023-11-22T09:52:28Z

I think this should be tested with one of the seedingDeepCore-modified workflows (offset number 0.17)

cmssw/Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py

Line 437 in 3b943c0

class UpgradeWorkflow_seedingDeepCore(UpgradeWorkflow):

jfernan2 · 2023-11-22T11:20:19Z

test parameters:

workflow = 11723.17

…re. Also updating to latest model JC20DC (MaxCand = 20)

jfernan2 · 2023-11-22T17:03:23Z

please test
With the right wf

cmsbuild · 2023-11-22T20:20:56Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4a3e84/36008/summary.html
COMMIT: b2d0b7a
CMSSW: CMSSW_14_0_X_2023-11-22-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/43339/36008/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 291 lines to the logs
Reco comparison results: 1818 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3467498
DQMHistoTests: Total failures: 2231
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3465244
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 218 log files, 172 edm output root files, 51 DQM output files
TriggerResults: no differences found

jfernan2 · 2023-11-23T15:46:12Z

@bouchamaouihichem could you please confirm if the changes introduced in the jets are as expected? https://tinyurl.com/yp38ebfl
Thanks

slava77 · 2023-11-23T15:55:18Z

RecoTracker/IterativeTracking/python/JetCoreRegionalStep_cff.py

@@ -158,7 +158,7 @@
                           maxPtForLooperReconstruction = 0.0)    
 jetCoreRegionalStepBarrelTrajectoryBuilder = RecoTracker.CkfPattern.GroupedCkfTrajectoryBuilder_cfi.GroupedCkfTrajectoryBuilderIterativeDefault.clone(
    trajectoryFilter = dict(refToPSet_ = 'jetCoreRegionalStepBarrelTrajectoryFilter'),
-    maxCand = 50,
+    maxCand = 30,


from a chat in the TRK mattermost thread, this change is not enough, another update is needed before this PR should be merged.

bouchamaouihichem · 2023-11-23T16:05:49Z

I am currently testing a fix and will update the PR (or make another one) soon

bouchamaouihichem · 2023-11-27T18:15:17Z

Added fix after after testing. Without the DeepCore ProcModifier JetCore will run with MaxCand =50. When the DeepCore ProcModifier is used, JetCore will run with MaxCand = 30 along with DeepCore.

cmsbuild · 2023-11-27T18:20:07Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-43339/37911

This PR adds an extra 24KB to repository

cmsbuild · 2023-11-27T18:20:30Z

Pull request #43339 was updated. @cmsbuild, @jfernan2, @mandrenguyen can you please check and sign again.

jfernan2 · 2023-11-27T19:14:27Z

please test

cmsbuild · 2023-11-27T22:32:59Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-4a3e84/36104/summary.html
COMMIT: 6086278
CMSSW: CMSSW_14_0_X_2023-11-27-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/43339/36104/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 89 lines from the logs
Reco comparison results: 1569 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3471548
DQMHistoTests: Total failures: 405
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3471120
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 218 log files, 172 edm output root files, 51 DQM output files
TriggerResults: no differences found

slava77 · 2023-11-28T15:09:28Z

Reco comparison results: 1569 differences found in the comparisons

as expected, the changes are only in the .17 (deep core) workflow, where more tracks appear with the jetCore algo ( jetCoreRegionalStep = 11 )

@cms-sw/reconstruction-l2
I think that this PR is OK from tracking POV;
please check this PR from your side.
Thank you.

jfernan2 · 2023-11-28T15:45:21Z

+1

cmsbuild · 2023-11-28T15:45:45Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @rappoccio, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

rappoccio · 2023-11-28T16:58:23Z

+1

Enabling the JetCore+DeepCore2.2.1 hybrid with MaxCand=30 (JC30DC) wi…

b2d0b7a

…th DeepCoreSeeding ProcModifiers

cmsbuild added this to the CMSSW_14_0_X milestone Nov 21, 2023

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending code-checks-pending tracking labels Nov 21, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 21, 2023

cmsbuild added tests-started and removed tests-pending labels Nov 21, 2023

cmsbuild added tests-approved and removed tests-started labels Nov 21, 2023

jfernan2 reviewed Nov 22, 2023

View reviewed changes

RecoTracker/IterativeTracking/python/JetCoreRegionalStep_cff.py Show resolved Hide resolved

Fixing Trajectory builder MaxCand for JetCore only and JetCore+DeepCo…

b986754

…re. Also updating to latest model JC20DC (MaxCand = 20)

cmsbuild added tests-started and removed tests-approved labels Nov 22, 2023

cmsbuild added tests-approved and removed tests-started labels Nov 22, 2023

slava77 reviewed Nov 23, 2023

View reviewed changes

Adding fix to keep default JetCore unchanged

6086278

cmsbuild added tests-pending code-checks-pending and removed tests-approved code-checks-approved labels Nov 27, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 27, 2023

cmsbuild added tests-started and removed tests-pending labels Nov 27, 2023

cmsbuild added tests-approved and removed tests-started labels Nov 27, 2023

cmsbuild added reconstruction-approved fully-signed and removed reconstruction-pending pending-signatures labels Nov 28, 2023

cmsbuild added orp-approved and removed orp-pending labels Nov 28, 2023

cmsbuild merged commit e548244 into cms-sw:master Nov 28, 2023

cmsbuild mentioned this pull request Nov 29, 2023

[LTO] Improve link time by using multiple process for linking cms-sw/cmsdist#8840

Merged

perrotta mentioned this pull request Nov 30, 2023

Spurious comparison differences in workflows 136.X, 141.044, 4.54, ... #43456

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enabling the JetCore+DeepCore2.2.1 hybrid with MaxCand=30 (JC30DC) #43339

Enabling the JetCore+DeepCore2.2.1 hybrid with MaxCand=30 (JC30DC) #43339

bouchamaouihichem commented Nov 21, 2023

cmsbuild commented Nov 21, 2023

cmsbuild commented Nov 21, 2023

jfernan2 commented Nov 21, 2023

cmsbuild commented Nov 21, 2023

mmusich commented Nov 22, 2023

jfernan2 commented Nov 22, 2023

jfernan2 commented Nov 22, 2023

cmsbuild commented Nov 22, 2023

jfernan2 commented Nov 23, 2023

slava77 Nov 23, 2023

bouchamaouihichem commented Nov 23, 2023

bouchamaouihichem commented Nov 27, 2023

cmsbuild commented Nov 27, 2023

cmsbuild commented Nov 27, 2023

jfernan2 commented Nov 27, 2023

cmsbuild commented Nov 27, 2023

slava77 commented Nov 28, 2023

jfernan2 commented Nov 28, 2023

cmsbuild commented Nov 28, 2023

rappoccio commented Nov 28, 2023

Enabling the JetCore+DeepCore2.2.1 hybrid with MaxCand=30 (JC30DC) #43339

Enabling the JetCore+DeepCore2.2.1 hybrid with MaxCand=30 (JC30DC) #43339

Conversation

bouchamaouihichem commented Nov 21, 2023

PR description:

PR validation:

cmsbuild commented Nov 21, 2023

cmsbuild commented Nov 21, 2023

jfernan2 commented Nov 21, 2023

cmsbuild commented Nov 21, 2023

Comparison Summary

mmusich commented Nov 22, 2023

jfernan2 commented Nov 22, 2023

jfernan2 commented Nov 22, 2023

cmsbuild commented Nov 22, 2023

Comparison Summary

jfernan2 commented Nov 23, 2023

slava77 Nov 23, 2023

Choose a reason for hiding this comment

bouchamaouihichem commented Nov 23, 2023

bouchamaouihichem commented Nov 27, 2023

cmsbuild commented Nov 27, 2023

cmsbuild commented Nov 27, 2023

jfernan2 commented Nov 27, 2023

cmsbuild commented Nov 27, 2023

Comparison Summary

slava77 commented Nov 28, 2023

jfernan2 commented Nov 28, 2023

cmsbuild commented Nov 28, 2023

rappoccio commented Nov 28, 2023