[10.6.X] Activate SiStrip Dynamical APV gain simulation for 2016 UL MC #28512

mmusich · 2019-11-29T16:32:57Z

PR description:

This is a follow-up to #28160.
In this PR the SiStrip APV saturation simulation is activated (assuming the 2 MC samples solution is used):

by switching on the includeAPVSimulation parameter using the tracker_apv_vfp30_2016 era modifier, hooked to the Era_Run2_2016_HIPM era on 100% of the events (ratio of the 2016 luminosity affected by the APV saturation vs the total is 53%);
providing the needed conditions of the APV baselines in the Global Tag (adding to the run2_mc & run2_mc_cosmics autoCond keys the appropriate tag: SiStripApvSimulationParameters_2016preVFP_v1). This corresponds to the APV baselines stored in https://patch-diff.githubusercontent.com/raw/cms-data/SimTracker-SiStripDigitizer/pull/2.patch.
modifying the event content to introduce a boolean flag (treated on equal footing for standard mixing and premixing) _simAPVsaturation_SimulatedAPVDynamicGain_ which propagates the notion if the Strip APV simulation has been run to the persisted event in all data-tiers, down to MINIAOD.
- this is done introducing an era modifier strips_vfp30_2016which is used in Era_Run2_2016 and excluded in Era_Run2_2017
the parameter includeAPVSimulation is turned off in the premix stage-1 and in the signal digitization of premix step-2.

PR validation:

Extensive technical validation has been carried out by the DPG (see presentation e.g. here) and through production of dedicated RelVals (see PDMVRELVALS-59).
Dedicated presentations have been given at:

7.11.2019: PPD general meeting Tracker/tracking for UL16
15.11.2019: RECO/AT meeting Tracker APV simulation in 2016
28.11.2019: PPD general meeting UL-16 tracking: APV/HIP data/MC comparisons

In addition, this PR has been technically tested with:

runTheMatrix.py -l limited
runTheMatrix.py -l 250400.0
runTheMatrix.py --what upgrade -l 20634.99 --ibeos

if this PR is a backport please specify the original PR:

This is a combined backport of #28355 and #28454

@pieterdavid @robervalwalsh @EmyrClement @mtosi @JanFSchulte @tsusa

- update 2016 Global Tags to include SiStripApvSimulationParametersRcd modified: Configuration/Eras/python/Era_Run2_2016_cff.py - introduce strips_vfp30_2016 modifier modified: Configuration/Eras/python/Era_Run2_2016_pA_cff.py - remove strips_vfp30_2016 modifier from era modified: Configuration/Eras/python/Era_Run2_2017_cff.py - remove strips_vfp30_2016 modifier from era new file: Configuration/Eras/python/Modifier_strips_vfp30_2016_cff.py - introduce strips_vfp30_2016 modifier modified: SimGeneral/MixingModule/python/SiStripSimParameters_cfi.py - activate APV simulation for 2016 pre-VFP change era

…defitions (down to miniAOD)

…2 premixing

…a_Run2_2016_HIPM, revert the other consequent changes and move fracOfEventsToSimAPV to be 100%

…V simulation

…-mixing

cmsbuild · 2019-11-29T16:33:30Z

A new Pull Request was created by @mmusich (Marco Musich) for CMSSW_10_6_X.

It involves the following packages:

Configuration/AlCa
Configuration/Eras
PhysicsTools/PatAlgos
SimGeneral/MixingModule
SimTracker/Configuration

@perrotta, @civanch, @slava77, @christopheralanwest, @mdhildreth, @cmsbuild, @franzoni, @tocheng, @tlampen, @pohsun, @santocch, @fabiocos, @kpedro88, @davidlange6 can you please review it and eventually sign? Thanks.
@echabert, @rappoccio, @gouskos, @pieterdavid, @robervalwalsh, @emilbols, @Martin-Grunewald, @ahinzmann, @threus, @peruzzim, @seemasharmafnal, @mmarionncern, @makortel, @smoortga, @acaudron, @mmusich, @jdolen, @prolay, @ferencek, @GiacomoSguazzoni, @rovere, @VinInn, @jdamgov, @nhanvtran, @gkasieczka, @tocheng, @schoef, @ebrondol, @dgulhan, @clelange, @HeinerTholen, @riga, @gbenelli, @JyothsnaKomaragiri, @mverzett, @gpetruc, @mariadalfonso, @pvmulder, @andrzejnovak this is something you requested to watch as well.
@davidlange6, @slava77, @fabiocos you are the release manager for this.

cms-bot commands are listed here

Configuration/AlCa/python/autoCond.py

mmusich · 2019-12-03T20:46:50Z

@fabiocos @srimanob
I do understand from discussion at the ORP today that this backport will be considered once technical validation of the content of 11_0_0_pre13 will be signed off, though would it be possible to start tests here to gain time (and exclude technical mistakes)? In particular would be nice to test this with #28513, including the workflows introduced there 25202.1,25202.2,250202.2,250202.3,250202.4,250202.5,1325.9,1325.91

cmsbuild · 2019-12-09T19:26:29Z

There was an issue with git-cms-merge-topic you can see the log here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-28512/3886/git-merge-result

fabiocos · 2019-12-09T21:43:28Z

please test workflow 25202.1,25202.2,250202.2,250202.3,250202.4,250202.5,1325.9,1325.91

cmsbuild · 2019-12-09T21:44:02Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/3888/console Started: 2019/12/09 22:44

cmsbuild · 2019-12-09T23:45:47Z

+1
Tested at: 69acd15
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cf8227/3888/summary.html

cmsbuild · 2019-12-09T23:45:49Z

Comparison job queued.

cmsbuild · 2019-12-10T01:17:31Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cf8227/3888/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-cf8227/1325.91_TTbar_13+TTbar_13+DIGIUP15APVSimu+RECOUP15_HIPM+HARVESTUP15+ALCATTUP15
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-cf8227/1325.9_TTbar_13+TTbar_13+DIGIUP15APVSimu+RECOUP15+HARVESTUP15+ALCATTUP15
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-cf8227/250202.2_TTbar_13+TTbar_13+DIGIPRMXUP15APVSimu_PU25+RECOPRMXUP15_PU25+HARVESTUP15_PU25
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-cf8227/250202.3_TTbar_13+TTbar_13+PREMIXUP15_PU25+DIGIPRMXLOCALUP15APVSimu_PU25+RECOPRMXUP15_PU25+HARVESTUP15_PU25
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-cf8227/250202.4_TTbar_13+TTbar_13+DIGIPRMXUP15APVSimu_PU25+RECOPRMXUP15_HIPM_PU25+HARVESTUP15_PU25
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-cf8227/250202.5_TTbar_13+TTbar_13+PREMIXUP15_PU25+DIGIPRMXLOCALUP15APVSimu_PU25+RECOPRMXUP15_HIPM_PU25+HARVESTUP15_PU25
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-cf8227/25202.1_TTbar_13+TTbar_13+DIGIUP15APVSimu_PU25+RECOUP15_PU25+HARVESTUP15_PU25
/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-cf8227/25202.2_TTbar_13+TTbar_13+DIGIUP15APVSimu_PU25+RECOUP15_PU25_HIPM+HARVESTUP15_PU25

Comparison Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 33
DQMHistoTests: Total histograms compared: 3212324
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3211989
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 32 files compared)
Checked 137 log files, 14 edm output root files, 33 DQM output files

slava77 · 2019-12-10T05:26:10Z

+1

for #28512 69acd15

reco-related parts from Activate SiStrip APV Simulation for 2016 MC #28355 and Fix alias in PreMixing configuration for APV simulation #28454 are backported consistently
jenkins tests pass and comparisons with the baseline show only the new product appearing in the tested standard workflows (bool_simAPVsaturation_SimulatedAPVDynamicGain); workflows with the APV saturation simulation run and their performance will be confirmed in a relval setup

cmsbuild · 2019-12-10T05:26:42Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_11_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @davidlange6, @slava77, @smuzaffar, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

fabiocos · 2019-12-10T08:28:20Z

+1

mmusich and others added 16 commits November 29, 2019 10:40

improve documentation of the fraction of events used for the simulation

cc94e95

persist SimulatedAPVDynamicGain flag in all SIM Tracker EventContent …

ee0daa5

…defitions (down to miniAOD)

use specific module instance names for SimulatedAPVDynamicGain

2a50fec

add line break for readability

18b7e42

set includeAPVSimulation to False for MixingModule instance of stage-…

9575dc9

…2 premixing

implement 2 MC solution, add the modifier strips_vfp30_2016 to era Er…

6438f37

…a_Run2_2016_HIPM, revert the other consequent changes and move fracOfEventsToSimAPV to be 100%

add alias for SimulatedAPVDynamicGain in std mixing and pre-mixing case

bfd6e25

revert mixData to mix in aliases_PreMix

673647e

use the tracker_apv_vfp30_2016 modifier to run the steering of the AP…

beab593

…V simulation

exclude strips_vfp30_2016 from Era_Run2_2017

255873d

make SimulatedAPVDynamicGain event product even in pre-mixing and std…

0e1b7b8

…-mixing

disable simAPVsaturation EDAlias for fast simulation

ed6d2e6

correct comment about stage2 premixing

247203b

fix the fastsim modifier declaration

5d542e4

Fix phase2 premixing aliases

7c35e19

cmsbuild added this to the CMSSW_10_6_X milestone Nov 29, 2019

cmsbuild added alca-pending analysis-pending comparison-pending operations-pending orp-pending pending-signatures reconstruction-pending simulation-pending tests-pending labels Nov 29, 2019

mmusich commented Nov 29, 2019

View reviewed changes

Configuration/AlCa/python/autoCond.py Outdated Show resolved Hide resolved

mmusich mentioned this pull request Nov 29, 2019

[10.6.X] Backport of APV Simulation RelVals #28513

Merged

cmsbuild added comparison-pending tests-pending tests-started and removed comparison-available tests-approved tests-pending labels Dec 9, 2019

cmsbuild added tests-pending and removed tests-started labels Dec 9, 2019

cmsbuild added tests-started and removed tests-pending labels Dec 9, 2019

cmsbuild added tests-approved and removed tests-started labels Dec 9, 2019

cmsbuild added comparison-available and removed comparison-pending labels Dec 10, 2019

cmsbuild added fully-signed reconstruction-approved and removed pending-signatures reconstruction-pending labels Dec 10, 2019

cmsbuild added orp-approved and removed orp-pending labels Dec 10, 2019

cmsbuild merged commit 6bba86c into cms-sw:CMSSW_10_6_X Dec 10, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[10.6.X] Activate SiStrip Dynamical APV gain simulation for 2016 UL MC #28512

[10.6.X] Activate SiStrip Dynamical APV gain simulation for 2016 UL MC #28512

mmusich commented Nov 29, 2019 •

edited

Loading

cmsbuild commented Nov 29, 2019

mmusich commented Dec 3, 2019

cmsbuild commented Dec 9, 2019

fabiocos commented Dec 9, 2019

cmsbuild commented Dec 9, 2019 •

edited

Loading

cmsbuild commented Dec 9, 2019

cmsbuild commented Dec 9, 2019

cmsbuild commented Dec 10, 2019

slava77 commented Dec 10, 2019

cmsbuild commented Dec 10, 2019

fabiocos commented Dec 10, 2019

[10.6.X] Activate SiStrip Dynamical APV gain simulation for 2016 UL MC #28512

[10.6.X] Activate SiStrip Dynamical APV gain simulation for 2016 UL MC #28512

Conversation

mmusich commented Nov 29, 2019 • edited Loading

PR description:

PR validation:

if this PR is a backport please specify the original PR:

cmsbuild commented Nov 29, 2019

mmusich commented Dec 3, 2019

cmsbuild commented Dec 9, 2019

fabiocos commented Dec 9, 2019

cmsbuild commented Dec 9, 2019 • edited Loading

cmsbuild commented Dec 9, 2019

cmsbuild commented Dec 9, 2019

cmsbuild commented Dec 10, 2019

slava77 commented Dec 10, 2019

cmsbuild commented Dec 10, 2019

fabiocos commented Dec 10, 2019

mmusich commented Nov 29, 2019 •

edited

Loading

cmsbuild commented Dec 9, 2019 •

edited

Loading