Addition of single iteration Patatrack and LST HLT configurations and workflows

[VourMa]

PR description

This is a procModifier-based implementation of LST in HLT. It reuses the trackingLST procModifier to enable LST track building, and it introduces a new one, seedingLST, which is not too useful for the HLT currently but it may become useful also for the offline setup in the near future.

Another modifier, singleIterPatatrack is introduced and it allows HLT to run only one iteration of tracking based on Patatrack pixel seeds. It works both with CKF and with LST building (when combined with the procModifier above).

Three new variations of the HLT timing workflow are also added (single iteration, single iteration LST, LST).

The most important results have been summarized in this presentation.

Instructions on how to run

cmsrel CMSSW_14_2_0_pre4
cd CMSSW_14_2_0_pre4/src
cmsenv
git cms-init
git cms-addpkg Configuration HLTrigger
git remote add SegLink [email protected]:SegmentLinking/cmssw.git
git fetch SegLink
git checkout CMSSW_14_2_0_pre4_singleIter_LST_HLT
scram b -j 16

# Rerun L1, needed because the HLT samples I am using are produced with 14_0_X
cmsDriver.py Phase2 -s L1,L1TrackTrigger \
--conditions auto:phase2_realistic_T33 \
--geometry ExtendedRun4D110 \
--era Phase2C17I13M9 \
--eventcontent FEVTDEBUGHLT \
--datatier GEN-SIM-DIGI-RAW-MINIAOD \
--customise SLHCUpgradeSimulations/Configuration/aging.customise_aging_1000,Configuration/DataProcessing/Utils.addMonitoring,L1Trigger/Configuration/customisePhase2FEVTDEBUGHLT.customisePhase2FEVTDEBUGHLT,L1Trigger/Configuration/customisePhase2TTOn110.customisePhase2TTOn110 \
--filein /store/mc/Phase2Spring24DIGIRECOMiniAOD/TT_TuneCP5_14TeV-powheg-pythia8/GEN-SIM-DIGI-RAW-MINIAOD/PU200_AllTP_140X_mcRun4_realistic_v4-v1/2560000/11d1f6f0-5f03-421e-90c7-b5815197fc85.root \
--fileout file:output_Phase2_L1T.root \
--inputCommands="keep *, drop l1tPFJets_*_*_*, drop l1tTrackerMuons_l1tTkMuonsGmt*_*_HLT" \
--outputCommands="drop l1tTrackerMuons_l1tTkMuonsGmt*_*_HLT" \
--mc -n 100 --nThreads 10

# Rerun L1 emulator + HLT (needed for any sample)
# In terms of procModifiers:
#    None -> Legacy seeding, CKF building
#    alpaka -> Patatrack seeding
#    singleIterPatatrack -> Single iteration tracking
#    trackingLST -> LST building for the initialStep
#    seedingLST -> LST seeding for the hightPtTripletStep
cmsDriver.py Phase2 -s L1P2GT,HLT:75e33 --processName=HLTX \
--conditions auto:phase2_realistic_T33 \
--geometry ExtendedRun4D110 \
--era Phase2C17I13M9 \
--eventcontent FEVTDEBUGHLT \
--customise SLHCUpgradeSimulations/Configuration/aging.customise_aging_1000 \
--filein file:output_Phase2_L1T.root \
--fileout file:Phase2_L1P2GT_HLT.root \
--inputCommands="keep *, drop *_hlt*_*_HLT, drop triggerTriggerFilterObjectWithRefs_l1t*_*_HLT" \
--outputCommands='keep *_*InitialStepTrack*LST*_*_*, keep *_*HighPtTripletStepTrack*_*_*' \
-n 100 --nThreads 10 \
--procModifiers alpaka,trackingLST,seedingLST

cmsDriver.py DQM -s VALIDATION:hltMultiTrackValidation \
--conditions auto:phase2_realistic_T33 \
--geometry ExtendedRun4D110 \
--era Phase2C17I13M9 \
--datatier DQMIO \
--eventcontent DQM \
--filein file:Phase2_L1P2GT_HLT.root \
--hltProcess HLTX \
--fileout DQM.root \
-n 100 --nThreads 10

cmsDriver.py HARVEST -s HARVESTING:@trackingOnlyValidation+@trackingOnlyDQM+postProcessorHLTtrackingSequence \
--filein file:DQM.root \
--scenario pp \
--filetype DQM \
--conditions auto:phase2_realistic_T33 \
--mc -n 100

mv DQM_V0001_R000000001__Global__CMSSW_X_Y_Z__RECO.root configName.root
makeTrackValidationPlots.py configName.root --extended

FYI @slava77 @rovere @lguzzi

[cmsbuild]

cms-bot internal usage

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46828/42842

• There are other open Pull requests which might conflict with changes you have proposed:
  • File Configuration/PyReleaseValidation/README.md modified in PR(s): Add run3 tracking low pu era #33532, ECAL RecHit producer Alpaka migration #46453
  • File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Add run3 tracking low pu era #33532, ECAL RecHit producer Alpaka migration #46453, Allowing Cluster Splitting with HLT Vertices #46694

[cmsbuild]

A new Pull Request was created by @VourMa for master.

It involves the following packages:

• Configuration/ProcessModifiers (operations)
• Configuration/PyReleaseValidation (upgrade, pdmv)
• HLTrigger/Configuration (hlt)
• RecoTracker/LSTCore (reconstruction)

@AdrianoDee, @Martin-Grunewald, @Moanwar, @antoniovilela, @cmsbuild, @davidlange6, @DickyChant, @fabiocos, @jfernan2, @mandrenguyen, @miquork, @mmusich, @rappoccio, @srimanob, @subirsarkar can you please review it and eventually sign? Thanks.
@GiacomoSguazzoni, @Martin-Grunewald, @SohamBhattacharya, @VinInn, @VourMa, @dgulhan, @fabiocos, @felicepantaleo, @gpetruc, @makortel, @missirol, @mmusich, @mtosi, @rovere, @silviodonato, @slomeo this is something you requested to watch as well.
@antoniovilela, @mandrenguyen, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

[VourMa]

I am not sure whether I should use the procModifiers introduced in this PR always with the alpaka procModifier. Feel free to give feedback on that and I can modify the necessary lines.

[mmusich]

test parameters:

• relvals_opt = --what upgrade
• workflows = 29634.753, 29634.754, 29634.755
• enable = hlt_p2_integration, hlt_p2_timing

[cmsbuild]

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46828/42846

• There are other open Pull requests which might conflict with changes you have proposed:
  • File Configuration/PyReleaseValidation/README.md modified in PR(s): Add run3 tracking low pu era #33532, ECAL RecHit producer Alpaka migration #46453
  • File Configuration/PyReleaseValidation/python/upgradeWorkflowComponents.py modified in PR(s): Add run3 tracking low pu era #33532, ECAL RecHit producer Alpaka migration #46453, Allowing Cluster Splitting with HLT Vertices #46694

[cmsbuild]

Pull request #46828 was updated. @AdrianoDee, @Martin-Grunewald, @Moanwar, @antoniovilela, @cmsbuild, @davidlange6, @DickyChant, @fabiocos, @jfernan2, @mandrenguyen, @miquork, @mmusich, @rappoccio, @srimanob, @subirsarkar can you please check and sign again.

[mmusich]

@cmsbuild, please test

[VourMa]

That's the gist of the discussion at #45508 (comment), where we decided to take this approach. Or do I misunderstand?

My point was having about a specific procModifier (e.g. patatrack and not the generic alpaka) to enable specific configurations that potentially change the physics. It's not about procModifiers vs. customizations (that's what the link in the comment points me to), which has been decided upon and is fine (in my opinion).

[mmusich]

to enable specific configurations that potentially change the physics.

the point here is that the legacy pixel tracking should not really exist in phase-2 (I think that's beyond discussion). I don't see the point of having a patatrack modifier to support a mode that is not envisaged in any possible scenario.

[cmsbuild]

+1

Size: This PR adds an extra 60KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-626af6/43196/summary.html
COMMIT: 0f32bd0
CMSSW: CMSSW_15_0_X_2024-12-02-1100/el8_amd64_gcc12
Additional Tests: HLT_P2_INTEGRATION,HLT_P2_TIMING
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/46828/43196/install.sh to create a dev area with all the needed externals and cmssw changes.

• HLT P2 Timing: chart

Comparison Summary

Summary:

• No significant changes to the logs found
• Reco comparison results: 59 differences found in the comparisons
• DQMHistoTests: Total files compared: 47
• DQMHistoTests: Total histograms compared: 3597272
• DQMHistoTests: Total failures: 2524
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3594728
• DQMHistoTests: Total skipped: 20
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 46 files compared)
• Checked 206 log files, 177 edm output root files, 47 DQM output files
• TriggerResults: found differences in 1 / 45 workflows

[mmusich]

+hlt

• for SegmentLinking@0f32bd0
• PR according to description and follow-up review: allows to run the novel LST algorithm in the Phase-2 HLT menu
• privately checked modifier-based integration tests, see Addition of single iteration Patatrack and LST HLT configurations and workflows #46828 (review)

[jfernan2]

+1

[AdrianoDee]

+pdmv

[mmusich]

@cms-sw/upgrade-l2 can you take a look?

[Moanwar]

+Upgrade

[cmsbuild]

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @mandrenguyen, @rappoccio, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

[mandrenguyen]

+1