Displaced tracking validation for Run 3 and Phase 2 #32157

jalimena · 2020-11-16T20:53:15Z

PR description:

This PR adds a collection to the tracking validation to validate displaced tracks. In particular, the usual tip and lip cuts on the tracking particles are relaxed. The collection is turned on only by use of a procModifier as shown below.

PR validation:

I've run the RECO, validation, and DQM steps (standard and trackingOnly) for ttbar and DisplacedSUSY samples, with Run 3 and Phase 2 conditions.

For example:
cmsDriver.py step3 --conditions auto:phase1_2021_realistic -s RAW2DIGI,L1Reco,RECO,RECOSIM,EI,PAT,VALIDATION:@standardValidation+@miniAODValidation,DQM:@standardDQM+@ExtraHLT+@miniAODDQM --datatier GEN-SIM-RECO,MINIAODSIM,DQMIO -n 200 --geometry DB:Extended --era Run3 --eventcontent RECOSIM,MINIAODSIM,DQM --procModifiers displacedTrackValidation --no_exec
run over the displaced SUSY sample in Run 3 conditions (0 PU), and then harvesting the DQM, produces these plots as output:
https://jalimena.web.cern.ch/jalimena/plots/track_validation_plots_dispSUSY_2021PU0_11_3_0_pre2_ProcModifier/

and
cmsDriver.py step3_trackingOnly --conditions auto:phase2_realistic_T15 -s RAW2DIGI,RECO:reconstruction_trackingOnly,VALIDATION:@trackingOnlyValidation,DQM:@trackingOnlyDQM --datatier GEN-SIM-RECO,DQMIO -n 200 --geometry Extended2026D49 --era Phase2C9 --eventcontent RECOSIM,DQM --procModifiers displacedTrackValidation --no_exec
run over a ttbar sample in phase 2 conditions with the trackingOnly sequence (0 PU), and then harvesting the DQM, produces these plots as output:
https://jalimena.web.cern.ch/jalimena/plots/track_validation_plots_ttbar_phase2PU0_trackingOnly_11_3_0_pre2_ProcModifier/

if this PR is a backport please specify the original PR and why you need to backport that PR:

not a backport

cmsbuild · 2020-11-16T20:59:47Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-32157/19851

This PR adds an extra 36KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Validation/RecoTrack/python/PostProcessorTracker_cfi.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel track reconstruction (10/N) #31722
- File Validation/RecoTrack/python/TrackValidation_cff.py modified in PR(s): Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel track reconstruction (10/N) #31722
- File Validation/RecoTrack/python/plotting/html.py modified in PR(s): DisplacedGeneralStep in tracking #32036, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel track reconstruction (10/N) #31722
- File Validation/RecoTrack/python/plotting/trackingPlots.py modified in PR(s): DisplacedGeneralStep in tracking #32036, Squash all Patatrack developments on top of CMSSW_11_3_0_pre5 #27983, Patatrack integration - Pixel track reconstruction (10/N) #31722
- File Validation/RecoTrack/test/trackingPerformanceValidation.py modified in PR(s): DisplacedGeneralStep in tracking #32036

cmsbuild · 2020-11-16T21:00:08Z

A new Pull Request was created by @jalimena (Juliette Alimena) for master.

It involves the following packages:

Validation/RecoTrack

@andrius-k, @kmaeshima, @ErnestaP, @cmsbuild, @jfernan2, @fioriNTU can you please review it and eventually sign? Thanks.
@makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @wmtford, @ebrondol, @mtosi, @dgulhan this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

jfernan2 · 2020-11-17T08:38:15Z

please test

cmsbuild · 2020-11-17T08:38:37Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-11-16-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/10798/console Started: 2020/11/17 11:11

cmsbuild · 2020-11-17T11:39:46Z

+1
Tested at: 99db796
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-57113f/10798/summary.html
CMSSW: CMSSW_11_2_X_2020-11-16-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-11-17T11:39:48Z

Comparison job queued.

cmsbuild · 2020-11-17T14:31:00Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-57113f/10798/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2529296
DQMHistoTests: Total failures: 3091
DQMHistoTests: Total nulls: 4
DQMHistoTests: Total successes: 2526179
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 246524.994 KiB( 34 files compared)
DQMHistoSizes: changed ( 10024.0,... ): 15600.468 KiB Tracking/TrackDisplaced
DQMHistoSizes: changed ( 10024.0,... ): 0.004 KiB MessageLogger/Errors
DQMHistoSizes: changed ( 10024.0,... ): 0.004 KiB MessageLogger/Warnings
DQMHistoSizes: changed ( 1306.0,... ): 12137.562 KiB Tracking/TrackDisplaced
DQMHistoSizes: changed ( 135.4,... ): 12088.062 KiB Tracking/TrackDisplaced
DQMHistoSizes: changed ( 23234.0,... ): 12285.731 KiB Tracking/TrackDisplaced
Checked 148 log files, 22 edm output root files, 35 DQM output files

jfernan2 · 2020-11-17T15:02:01Z

@jalimena this PR is adding around 20k new MEs, I wonder if such a huge number of plots and possibilities is actually needed for Validation, I can imagine what a handicap can be to check all of them with releases and samples.

On the other hand the configuration is creating the new plots for every MC workflow, could you please reduce it to just Phase2 and Run3 samples?

Thanks
All details about the changes introduced in the workflows and samples here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_11_2_X_2020-11-16-2300+57113f/39929/dqm-histo-comparison-summary.html

jalimena · 2020-11-17T15:46:57Z

@jalimena this PR is adding around 20k new MEs, I wonder if such a huge number of plots and possibilities is actually needed for Validation, I can imagine what a handicap can be to check all of them with releases and samples.

On the other hand the configuration is creating the new plots for every MC workflow, could you please reduce it to just Phase2 and Run3 samples?

Thanks
All details about the changes introduced in the workflows and samples here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisons/CMSSW_11_2_X_2020-11-16-2300+57113f/39929/dqm-histo-comparison-summary.html

Hi @jfernan2 indeed I don't want to add (or look at) 20k new plots! One thing I see I can do is to remove them for fastsim, but maybe that's not a great idea, given that fastsim is beginning to be used for long-lived particle signals? I also see in the code how I can add the collection just for phase 2 (i.e. with things like phase2_tracker.toModify() in Validation/RecoTrack/python/TrackValidation_cff.py), but I'm unsure how to do it for just Run 3 here - maybe with phase1Pixel.toModify() as I see elsewhere in the package? @mtosi or other tracking experts, can you help me? What do you recommend I do, in practice, to redo the number of plots?

mmusich · 2020-11-17T15:49:08Z

@jalimena

, but I'm unsure how to do it for just Run 3 here - maybe with phase1Pixel.toModify() as I see elsewhere in the package?

there is a run3 specific modifier:

https://github.com/cms-sw/cmssw/blob/master/Configuration/Eras/python/Modifier_run3_common_cff.py

cmsbuild · 2021-01-26T22:55:36Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-57113f/12553/summary.html
COMMIT: 0141ad3
CMSSW: CMSSW_11_3_X_2021-01-26-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32157/12553/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

The workflows 140.53 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

Summary:

No significant changes to the logs found
Reco comparison results: 1096 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2716596
DQMHistoTests: Total failures: 3611
DQMHistoTests: Total nulls: 19
DQMHistoTests: Total successes: 2712944
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -43.909 KiB( 36 files compared)
DQMHistoSizes: changed ( 1000.0,... ): 0.078 KiB Tracking/MessageLog
DQMHistoSizes: changed ( 140.53 ): -44.531 KiB Hcal/DigiRunHarvesting
DQMHistoSizes: changed ( 140.53 ): -1.172 KiB RPC/DCSInfo
Checked 156 log files, 37 edm output root files, 37 DQM output files

jfernan2 · 2021-01-27T08:36:35Z

There are a lot of changes in 140.53 unrelated to this PR.... has somebody seen it before?
Thanks

makortel · 2021-01-27T14:01:40Z

There are a lot of changes in 140.53 unrelated to this PR.... has somebody seen it before?
Thanks

Note

The workflows 140.53 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary.

in the comparison summary message.

jfernan2 · 2021-01-27T14:04:10Z

please test with #32036

jfernan2 · 2021-01-27T14:04:54Z

There are a lot of changes in 140.53 unrelated to this PR.... has somebody seen it before?
Thanks

Note

The workflows 140.53 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary.

in the comparison summary message.

Thanks @makortel I hadn't noticed it

makortel · 2021-01-27T14:10:18Z

There is some more information in #32755

cmsbuild · 2021-01-27T17:50:46Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-57113f/12568/summary.html
COMMIT: 0141ad3
CMSSW: CMSSW_11_3_X_2021-01-26-2300/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/32157/12568/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2716596
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2716567
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 1.794 KiB( 36 files compared)
DQMHistoSizes: changed ( 1000.0,... ): 0.078 KiB Tracking/MessageLog
Checked 156 log files, 37 edm output root files, 37 DQM output files

jfernan2 · 2021-01-27T18:04:00Z

Still to do:

modify Configuration/PyReleaseValidation appropriately to turn this validation sequence on for the processes we are interested in (I have asked for help by email)

confirm with PPD/PDMV and DQM conveners that this validation plan is reasonable (awaiting a response to Mia's email)

@jalimena do you plan to include those in this PR or in an independent one?

jalimena · 2021-01-27T18:11:55Z

Still to do:

modify Configuration/PyReleaseValidation appropriately to turn this validation sequence on for the processes we are interested in (I have asked for help by email)

confirm with PPD/PDMV and DQM conveners that this validation plan is reasonable (awaiting a response to Mia's email)

@jalimena do you plan to include those in this PR or in an independent one?

@jfernan2 If I get some advice from anyone soon (perhaps @mtosi ?) about how to modify Configuration/PyReleaseValidation, I will happily include it promptly in this PR. However, I guess it is not strictly necessary and I can open a new one for it. If you all are ready to sign and merge it like ~now, then I can for sure open a new PR for the small additions needed.

jfernan2 · 2021-01-27T18:13:36Z

Ok, so let's decouple the plots from their actual enabling. Thanks

jfernan2 · 2021-01-27T18:13:40Z

+1

jalimena · 2021-01-27T18:20:44Z

thanks to you!

silviodonato · 2021-01-29T18:11:34Z

+1

cmsbuild · 2021-01-29T18:11:56Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

cmsbuild added this to the CMSSW_11_2_X milestone Nov 16, 2020

cmsbuild added code-checks-pending comparison-pending dqm-pending orp-pending pending-signatures tests-pending labels Nov 16, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Nov 16, 2020

cmsbuild added tests-started and removed tests-pending labels Nov 17, 2020

cmsbuild added tests-approved and removed tests-started labels Nov 17, 2020

cmsbuild added comparison-available and removed comparison-pending labels Nov 17, 2020

cmsbuild added code-checks-pending comparison-pending and removed code-checks-approved comparison-available tests-approved labels Nov 17, 2020

cmsbuild added the tests-started label Jan 26, 2021

cmsbuild added tests-approved and removed tests-started labels Jan 26, 2021

cmsbuild added tests-started and removed tests-approved labels Jan 27, 2021

cmsbuild added tests-approved and removed tests-started labels Jan 27, 2021

cmsbuild added dqm-approved and removed dqm-pending labels Jan 27, 2021

cmsbuild added fully-signed operations-approved orp-approved and removed operations-pending orp-pending pending-signatures labels Jan 29, 2021

cmsbuild merged commit 197a6df into cms-sw:master Jan 29, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Displaced tracking validation for Run 3 and Phase 2 #32157

Displaced tracking validation for Run 3 and Phase 2 #32157

jalimena commented Nov 16, 2020 •

edited

Loading

cmsbuild commented Nov 16, 2020

cmsbuild commented Nov 16, 2020

jfernan2 commented Nov 17, 2020

cmsbuild commented Nov 17, 2020 •

edited

Loading

cmsbuild commented Nov 17, 2020

cmsbuild commented Nov 17, 2020

cmsbuild commented Nov 17, 2020

jfernan2 commented Nov 17, 2020

jalimena commented Nov 17, 2020

mmusich commented Nov 17, 2020

cmsbuild commented Jan 26, 2021

jfernan2 commented Jan 27, 2021

makortel commented Jan 27, 2021

jfernan2 commented Jan 27, 2021

jfernan2 commented Jan 27, 2021

makortel commented Jan 27, 2021

cmsbuild commented Jan 27, 2021

jfernan2 commented Jan 27, 2021

jalimena commented Jan 27, 2021

jfernan2 commented Jan 27, 2021

jfernan2 commented Jan 27, 2021

jalimena commented Jan 27, 2021

silviodonato commented Jan 29, 2021

cmsbuild commented Jan 29, 2021

Displaced tracking validation for Run 3 and Phase 2 #32157

Displaced tracking validation for Run 3 and Phase 2 #32157

Conversation

jalimena commented Nov 16, 2020 • edited Loading

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Nov 16, 2020

cmsbuild commented Nov 16, 2020

jfernan2 commented Nov 17, 2020

cmsbuild commented Nov 17, 2020 • edited Loading

cmsbuild commented Nov 17, 2020

cmsbuild commented Nov 17, 2020

cmsbuild commented Nov 17, 2020

jfernan2 commented Nov 17, 2020

jalimena commented Nov 17, 2020

mmusich commented Nov 17, 2020

cmsbuild commented Jan 26, 2021

Comparison Summary

jfernan2 commented Jan 27, 2021

makortel commented Jan 27, 2021

jfernan2 commented Jan 27, 2021

jfernan2 commented Jan 27, 2021

makortel commented Jan 27, 2021

cmsbuild commented Jan 27, 2021

Comparison Summary

jfernan2 commented Jan 27, 2021

jalimena commented Jan 27, 2021

jfernan2 commented Jan 27, 2021

jfernan2 commented Jan 27, 2021

jalimena commented Jan 27, 2021

silviodonato commented Jan 29, 2021

cmsbuild commented Jan 29, 2021

jalimena commented Nov 16, 2020 •

edited

Loading

cmsbuild commented Nov 17, 2020 •

edited

Loading