add track algo and originalAlgo to PackedCandidates #41183

slava77 · 2023-03-25T00:23:50Z

This PR adds track algo and originalAlgo to PackedCandidates as uint8_t data members; the values are filled with the same selections as other track properties (mainly driven by pt> minPtForTrackProp, the cut is at 0.95 GeV)

The primary use case so far is in the tag-and-probe measurements of tracking performance where general and muon tracking iterations need to be separated and the analysis is forced to use AOD, while a switch to miniAOD should be possible.

More use cases can arise for improved track quality selection and lower level debugging in miniAOD.

Tested in CMSSW_13_0_0_pre4 on /store/relval/CMSSW_13_0_0_pre4/RelValTTbar_14TeV/GEN-SIM-RECO/PU_130X_mcRun3_2022_realistic_v2-v2/00000/{08b3cb94-2a3e-42c8-9374-46b4264dbef7.root,1a2b2a1d-133a-435c-b275-a4e3b651e71c.root,50823067-81cd-459a-baf7-aabc7851cce5.root,902c1d68-aa8e-49ea-94df-228edecdd3e9.root} with 4000 events running PAT step (step4 from wf 11834.21).

the sum of all products (using edmEventSize) compressed size increases by 238 B/ev or 0.27%
The packedPFCandidates collection size increases by 0.5% or 205 B/event (from 43.3 kiB/ev)

This was proposed to XPOG in the Jan 25 meeting (before the code was implemented and consequently before the file size measurements mentioned above)

slava77 · 2023-03-25T00:27:59Z

type tracking

@mmusich

cmsbuild · 2023-03-25T00:32:13Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41183/34870

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/PatCandidates/src/classes_def_objects.xml modified in PR(s): PFCandidate no longer inherits from CompositeCandidate #38999, Removing not used muon MVAs from miniAOD #40876
- File RecoBTag/FeatureTools/plugins/DeepBoostedJetTagInfoProducer.cc modified in PR(s): Custom Run 3 PFScouting NanoAOD #40438, [BTV] PUPPI ValueMap-compatible Deep taggers info producers #40803

cmsbuild · 2023-03-25T00:32:35Z

A new Pull Request was created by @slava77 (Slava Krutelyov) for master.

It involves the following packages:

DataFormats/PatCandidates (xpog, reconstruction)
PhysicsTools/PatAlgos (xpog, reconstruction)
RecoBTag/FeatureTools (reconstruction)

@swertz, @vlimant, @clacaputo, @cmsbuild, @simonepigazzini, @mandrenguyen can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @hatakeyamak, @emilbols, @mbluj, @demuller, @seemasharmafnal, @mmarionncern, @missirol, @ahinzmann, @jdolen, @azotz, @hqucms, @rovere, @jdamgov, @nhanvtran, @gkasieczka, @schoef, @andrzejnovak, @AlexDeMoor, @AnnikaStein, @JyothsnaKomaragiri, @gpetruc, @mariadalfonso this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

slava77 · 2023-03-25T00:32:55Z

@cmsbuild please test

cmsbuild · 2023-03-25T03:48:30Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cf8978/31588/summary.html
COMMIT: 8f215aa
CMSSW: CMSSW_13_1_X_2023-03-24-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41183/31588/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 17 lines from the logs
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3554286
DQMHistoTests: Total failures: 4
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3554260
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

mandrenguyen · 2023-03-25T12:02:43Z

@slava77 thealgo assignment is from the track merging, based on which iteration gives the highest quality track, right? originalAlgo is presumably the first iteration to find the track. Can you remind me why the latter is important to keep track of?

slava77 · 2023-03-25T12:23:30Z

@slava77 thealgo assignment is from the track merging, based on which iteration gives the highest quality track, right? originalAlgo is presumably the first iteration to find the track. Can you remind me why the latter is important to keep track of?

The algo is roughly as you describe but details may be more complicated (e.g. a more pure initialStep could be stopped earlier during building/fitting and may be overtaken with quality-driven assignment of a track made with a jetCore iteration in the final algo).

Knowledge of the originalAlgo turned out to be useful in the context of mkFit integration.
Also, it probably gives a smaller bias in the efficiency measurement compared to the algo when selecting purely tracker-driven iterations.

slava77 · 2023-03-25T13:12:38Z

hold

I think I'm not filling it right

cmsbuild · 2023-03-25T13:13:02Z

Pull request has been put on hold by @slava77
They need to issue an unhold command to remove the hold state or L1 can unhold it for all

… 8 bits only if algo!=original

slava77 · 2023-03-28T14:54:42Z

In the latest variant I

repacked the two iteration chars to a uint16_t, which provides a noticeable size decrease from +278 B/evt to +212 B/evt in 4K events of packedPFCandidates
save the upper byte only if the originalAlgo is not equal to algo, which happens only in 5% cases. This gives a small saving reducing total increase in packedPFCandidates to just +205 B/evt. Evidently the compression is already very effective in this case.

So, now over 4K events

the sum of all products (using edmEventSize) compressed size increases by 238 B/ev or 0.27%
The packedPFCandidates collection size increases by 0.5% or 205 B/event (from 43.3 kiB/ev)
(the PR description was updated)

cmsbuild · 2023-03-28T14:56:25Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41183/34914

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File DataFormats/PatCandidates/src/classes_def_objects.xml modified in PR(s): PFCandidate no longer inherits from CompositeCandidate #38999
- File RecoBTag/FeatureTools/plugins/DeepBoostedJetTagInfoProducer.cc modified in PR(s): Custom Run 3 PFScouting NanoAOD #40438

cmsbuild · 2023-03-28T14:56:49Z

Pull request #41183 was updated. @swertz, @vlimant, @clacaputo, @cmsbuild, @simonepigazzini, @mandrenguyen can you please check and sign again.

slava77 · 2023-03-28T15:21:33Z

BTW, the plan is to backport this to 13_0.

slava77 · 2023-03-28T15:22:15Z

@cmsbuild please test

cmsbuild · 2023-03-28T18:52:27Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cf8978/31653/summary.html
COMMIT: 1e3b629
CMSSW: CMSSW_13_1_X_2023-03-28-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41183/31653/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

There are some workflows for which there are errors in the baseline:
24234.61 step 2
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Summary:

You potentially added 2 lines to the logs
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3554236
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3554207
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

mandrenguyen · 2023-03-28T20:01:26Z

+reconstruction
Modest increase in miniAOD size as explained in #41183 (comment)
Adds an extra integer that specifies which iteration tracks that are linked to PF charged hadrons come from.
Needed for tracking studies.

slava77 · 2023-03-29T13:06:11Z

@cms-sw/xpog-l2
do you have additional comments for this PR?
Please check and comment or sign.
Thank you.

vlimant · 2023-03-29T15:52:41Z

+1

cmsbuild · 2023-03-29T15:53:06Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-03-29T19:49:29Z

+1

add track algo and originalAlgo to PackedCandidates

8f215aa

cmsbuild added this to the CMSSW_13_1_X milestone Mar 25, 2023

cmsbuild added reconstruction-pending pending-signatures tests-pending orp-pending code-checks-pending xpog-pending labels Mar 25, 2023

cmsbuild added the tracking label Mar 25, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 25, 2023

cmsbuild added tests-started and removed tests-pending labels Mar 25, 2023

cmsbuild added tests-approved and removed tests-started labels Mar 25, 2023

cmsbuild added the hold label Mar 25, 2023

add trk algo to copy methods as well

e6cbe22

cmsbuild added tests-pending code-checks-pending and removed tests-approved code-checks-approved labels Mar 26, 2023

pack algo and originalAlgo to a uint16_t, original saved in the upper…

1e3b629

… 8 bits only if algo!=original

cmsbuild added code-checks-pending and removed code-checks-approved labels Mar 28, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 28, 2023

cmsbuild added tests-started and removed tests-pending labels Mar 28, 2023

cmsbuild added tests-approved and removed tests-started labels Mar 28, 2023

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Mar 28, 2023

cmsbuild added fully-signed xpog-approved and removed pending-signatures xpog-pending labels Mar 29, 2023

cmsbuild added orp-approved and removed orp-pending labels Mar 29, 2023

cmsbuild merged commit b350d07 into cms-sw:master Mar 29, 2023

slava77 mentioned this pull request Mar 29, 2023

add track algo and originalAlgo to PackedCandidates [13_0_X] #41227

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add track algo and originalAlgo to PackedCandidates #41183

add track algo and originalAlgo to PackedCandidates #41183

slava77 commented Mar 25, 2023 •

edited

Loading

slava77 commented Mar 25, 2023

cmsbuild commented Mar 25, 2023

cmsbuild commented Mar 25, 2023

slava77 commented Mar 25, 2023

cmsbuild commented Mar 25, 2023

mandrenguyen commented Mar 25, 2023

slava77 commented Mar 25, 2023

slava77 commented Mar 25, 2023

cmsbuild commented Mar 25, 2023

slava77 commented Mar 28, 2023

cmsbuild commented Mar 28, 2023

cmsbuild commented Mar 28, 2023

slava77 commented Mar 28, 2023

slava77 commented Mar 28, 2023

cmsbuild commented Mar 28, 2023

mandrenguyen commented Mar 28, 2023

slava77 commented Mar 29, 2023

vlimant commented Mar 29, 2023

cmsbuild commented Mar 29, 2023

perrotta commented Mar 29, 2023

add track algo and originalAlgo to PackedCandidates #41183

add track algo and originalAlgo to PackedCandidates #41183

Conversation

slava77 commented Mar 25, 2023 • edited Loading

slava77 commented Mar 25, 2023

cmsbuild commented Mar 25, 2023

cmsbuild commented Mar 25, 2023

slava77 commented Mar 25, 2023

cmsbuild commented Mar 25, 2023

Comparison Summary

mandrenguyen commented Mar 25, 2023

slava77 commented Mar 25, 2023

slava77 commented Mar 25, 2023

cmsbuild commented Mar 25, 2023

slava77 commented Mar 28, 2023

cmsbuild commented Mar 28, 2023

cmsbuild commented Mar 28, 2023

slava77 commented Mar 28, 2023

slava77 commented Mar 28, 2023

cmsbuild commented Mar 28, 2023

Comparison Summary

mandrenguyen commented Mar 28, 2023

slava77 commented Mar 29, 2023

vlimant commented Mar 29, 2023

cmsbuild commented Mar 29, 2023

perrotta commented Mar 29, 2023

slava77 commented Mar 25, 2023 •

edited

Loading