BTL simulation: OOT effects + premix fix #28757

casarsa · 2020-01-17T15:33:19Z

PR description:

This PR improves the BTL digitization including the effects on time resolution due to the photon tails of earlier OOT hits in the same crystal, like in PR #28433, and fixes the memory issue that had emerged when testing the premixing for that PR.

The cause of the problem was that the method that packs the SIM hits info into the premix bank expects times expressed modulo 25n, but that was not my case. As a consequence, the bit word with the SIM info was corrupted and the unpacking resulted into an overflowing index.

PR validation:

This branch has been tested in CMSSW_11_0_0_pre13 with the TTbar WFs 22234.0 and 22234.99.

WF 22234.0

step 2:

MemoryReport> Peak virtual size 7964.15 Mbytes
Key events increasing vsize:
[60] run: 1 lumi: 1 event: 60 vsize = 6286.11 deltaVsize = 640 rss = 3096.59 delta = 206.461
[249] run: 1 lumi: 1 event: 249 vsize = 7068.14 deltaVsize = 782 rss = 2927.33 delta = -169.258
[485] run: 1 lumi: 1 event: 485 vsize = 7964.15 deltaVsize = 896 rss = 3123.05 delta = 195.715
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[487] run: 1 lumi: 1 event: 487 vsize = 7964.15 deltaVsize = 0 rss = 3021.71 delta = -101.34
[486] run: 1 lumi: 1 event: 486 vsize = 7964.15 deltaVsize = 0 rss = 3107.3 delta = -15.75
[485] run: 1 lumi: 1 event: 485 vsize = 7964.15 deltaVsize = 896 rss = 3123.05 delta = 124.598
TimeReport> Time report complete in 44014 seconds
Time Summary:

Min event: 65.8531
Max event: 110.886
Avg event: 87.1839
Total loop: 43980.1
Total init: 33.3893
Total job: 44014
EventSetup Lock: 8.51154e-05
EventSetup Get: 76.1835
Event Throughput: 0.0113688 ev/s
CPU Summary:
Total loop: 41805.5
Total init: 11.4153
Total extra: 0
Total job: 41817.4

step 3

MemoryReport> Peak virtual size 12965.9 Mbytes
Key events increasing vsize:
[32] run: 1 lumi: 1 event: 32 vsize = 10146.7 deltaVsize = 1024 rss = 6783.16 delta = 280.848
[143] run: 1 lumi: 1 event: 143 vsize = 11426.8 deltaVsize = 1280 rss = 6918.56 delta = 135.406
[408] run: 1 lumi: 1 event: 408 vsize = 12962.9 deltaVsize = 1535 rss = 7269.84 delta = 351.277
[492] run: 1 lumi: 1 event: 492 vsize = 12965.9 deltaVsize = 3 rss = 7312.45 delta = 42.6055
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[494] run: 1 lumi: 1 event: 494 vsize = 12963.9 deltaVsize = -2 rss = 6955.95 delta = -356.496
[493] run: 1 lumi: 1 event: 493 vsize = 12963.9 deltaVsize = -2 rss = 7115.16 delta = -197.281
[492] run: 1 lumi: 1 event: 492 vsize = 12965.9 deltaVsize = 3 rss = 7312.45 delta = 42.6055
TimeReport> Time report complete in 89719 seconds
Time Summary:

Min event: 104.361
Max event: 346.13
Avg event: 175.392
Total loop: 89587.9
Total init: 121.492
Total job: 89719
EventSetup Lock: 0.000196457
EventSetup Get: 83.9121
Event Throughput: 0.00558111 ev/s
CPU Summary:
Total loop: 82191.3
Total init: 96.2134
Total extra: 0
Total job: 82296.5

WF 22234.99

premix bank size:

MTDSimAccumulator_mix_FTLBarrel_DIGI. 2.97066e+06 2.39297e+06
PMTDSimAccumulator_mix_FTLEndcap_DIGI. 1.33694e+06 779230

step 3

MemoryReport> Peak virtual size 5278.37 Mbytes
Key events increasing vsize:
[2] run: 1 lumi: 1 event: 2 vsize = 4446.24 deltaVsize = 12 rss = 2998.72 delta = -134.793
[16] run: 1 lumi: 1 event: 16 vsize = 4830.32 deltaVsize = 384 rss = 2981.2 delta = -17.5234
[458] run: 1 lumi: 1 event: 458 vsize = 5278.37 deltaVsize = 448 rss = 3136.12 delta = 154.93
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[460] run: 1 lumi: 1 event: 460 vsize = 5278.37 deltaVsize = 0 rss = 3103.74 delta = -32.3867
[459] run: 1 lumi: 1 event: 459 vsize = 5278.37 deltaVsize = 0 rss = 3062.41 delta = -73.7148
[458] run: 1 lumi: 1 event: 458 vsize = 5278.37 deltaVsize = 448 rss = 3136.12 delta = 3.17578
TimeReport> Time report complete in 24610.4 seconds
Time Summary:

Min event: 39.9079
Max event: 104.543
Avg event: 48.7014
Total loop: 24556
Total init: 53.7632
Total job: 24610.4
EventSetup Lock: 7.58171e-05
EventSetup Get: 67.2327
Event Throughput: 0.0203616 ev/s
CPU Summary:
Total loop: 22172.9
Total init: 12.3361
Total extra: 0
Total job: 22185.7

step 4

MemoryReport> Peak virtual size 9123.34 Mbytes
Key events increasing vsize:
[2] run: 1 lumi: 1 event: 2 vsize = 7422.96 deltaVsize = 660.016 rss = 5336.09 delta = -65.8555
[15] run: 1 lumi: 1 event: 15 vsize = 8227.02 deltaVsize = 768 rss = 5545.06 delta = 208.977
[154] run: 1 lumi: 1 event: 154 vsize = 9123.32 deltaVsize = 896 rss = 5983.21 delta = 438.148
[458] run: 1 lumi: 1 event: 458 vsize = 9123.34 deltaVsize = 0.0195312 rss = 6423.86 delta = 440.652
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[460] run: 1 lumi: 1 event: 460 vsize = 9123.34 deltaVsize = 0 rss = 6370.79 delta = -53.0742
[459] run: 1 lumi: 1 event: 459 vsize = 9123.34 deltaVsize = 0 rss = 6262.03 delta = -161.836
[458] run: 1 lumi: 1 event: 458 vsize = 9123.34 deltaVsize = 0.0195312 rss = 6423.86 delta = 440.652
TimeReport> Time report complete in 81246.8 seconds
Time Summary:

Min event: 89.7388
Max event: 467.405
Avg event: 158.661
Total loop: 81027.2
Total init: 166.451
Total job: 81246.8
EventSetup Lock: 0.000249386
EventSetup Get: 101.051
Event Throughput: 0.00617077 ev/s
CPU Summary:
Total loop: 75961.9
Total init: 120.744
Total extra: 0
Total job: 76134.6

=============================================

In the case of current code, as a reference:

WF 22234.0

step 2

MemoryReport> Peak virtual size 6286.12 Mbytes
Key events increasing vsize:
[4] run: 1 lumi: 1 event: 4 vsize = 5134.07 deltaVsize = 448 rss = 3084.34 delta = -16
[15] run: 1 lumi: 1 event: 15 vsize = 5646.11 deltaVsize = 512 rss = 3022.9 delta = -61.4336
[61] run: 1 lumi: 1 event: 61 vsize = 6286.12 deltaVsize = 640 rss = 3414.31 delta = 391.41
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[63] run: 1 lumi: 1 event: 63 vsize = 6286.12 deltaVsize = 0 rss = 3323.92 delta = -90.3945
[62] run: 1 lumi: 1 event: 62 vsize = 6286.12 deltaVsize = 0 rss = 3402.76 delta = -11.5547
[61] run: 1 lumi: 1 event: 61 vsize = 6286.12 deltaVsize = 640 rss = 3414.31 delta = -82.2305
TimeReport> Time report complete in 9059.45 seconds
Time Summary:

Min event: 73.8451
Max event: 114.501
Avg event: 89.0312
Total loop: 9016.34
Total init: 42.7557
Total job: 9059.45
EventSetup Lock: 9.46522e-05
EventSetup Get: 73.5861
Event Throughput: 0.011091 ev/s
CPU Summary:
Total loop: 8343.95
Total init: 21.0478
Total extra: 0
Total job: 8365.25

step 3

MemoryReport> Peak virtual size 10145.7 Mbytes
Key events increasing vsize:
[3] run: 1 lumi: 1 event: 3 vsize = 9089.55 deltaVsize = 4 rss = 6709.96 delta = 18.8125
[6] run: 1 lumi: 1 event: 6 vsize = 9121.55 deltaVsize = 32 rss = 6750.29 delta = 40.3281
[29] run: 1 lumi: 1 event: 29 vsize = 10145.7 deltaVsize = 1024 rss = 7010.46 delta = 260.172
[83] run: 1 lumi: 1 event: 83 vsize = 10145.7 deltaVsize = 0.0273438 rss = 7426.93 delta = 416.469
[46] run: 1 lumi: 1 event: 46 vsize = 10145.7 deltaVsize = 0.0078125 rss = 7203.63 delta = 193.168
[85] run: 1 lumi: 1 event: 85 vsize = 10145.7 deltaVsize = 0 rss = 7077.25 delta = -349.68
[84] run: 1 lumi: 1 event: 84 vsize = 10145.7 deltaVsize = 0 rss = 7218.67 delta = -208.262
[83] run: 1 lumi: 1 event: 83 vsize = 10145.7 deltaVsize = 0.0273438 rss = 7426.93 delta = 223.301
TimeReport> Time report complete in 19135.3 seconds
Time Summary:

Min event: 129.462
Max event: 314.8
Avg event: 185.461
Total loop: 18963.7
Total init: 162.735
Total job: 19135.3
EventSetup Lock: 0.000247478
EventSetup Get: 92.7902
Event Throughput: 0.00527324 ev/s
CPU Summary:
Total loop: 17414.4
Total init: 117.035
Total extra: 0
Total job: 17539.8

…f photon tails from earlier OOT hits

…patible with the premixing packing

…m the premixing bank

cmsbuild · 2020-01-17T15:33:50Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-01-17T15:39:46Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28757/13399

This PR adds an extra 28KB to repository

cmsbuild · 2020-01-17T15:40:10Z

A new Pull Request was created by @casarsa (Massimo Casarsa) for master.

It involves the following packages:

DataFormats/FTLDigi
SimFastTiming/FastTimingCommon

@cmsbuild, @civanch, @kpedro88, @mdhildreth can you please review it and eventually sign? Thanks.
@rovere this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

fabiocos · 2020-01-17T17:07:31Z

please test

cmsbuild · 2020-01-17T17:07:55Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4274/console Started: 2020/01/17 18:09

cmsbuild · 2020-01-17T18:19:21Z

+1
Tested at: 3a350ad
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ea5a91/4274/summary.html
CMSSW: CMSSW_11_1_X_2020-01-17-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-01-17T18:19:25Z

Comparison job queued.

cmsbuild · 2020-01-17T19:58:23Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ea5a91/4274/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 9761 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2697090
DQMHistoTests: Total failures: 8371
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2688373
DQMHistoTests: Total skipped: 346
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

silviodonato · 2020-01-21T08:58:35Z

cc: @trtomei @fwyzard This issue could affect the 11_0_0_patch1 production.

@casarsa since this PR is changing some constant values in DataFormats/FTLDigi/interface/PMTDSimAccumulator.h , the change could potentially create an incompatibility between CMSSW_11_1_X and CMSSW_11_0_X (we need this compatibility in order to run the rereco for the HLT TDR in CMSSW_11_1_X).
Could you check if the MTD digis produced without #28757 can be read correctly including #28757? Thanks!

makortel · 2020-01-21T15:16:07Z

@silviodonato For my education, is the plan to produce signal GEN-SIM and a premixed pileup library in 11_0_X, and run the overlay (premix stage2) in 11_1_X?

casarsa · 2020-01-27T09:25:55Z

@kpedro88 I've rerun the current code with the same number of events as for my branch tests:

WF 22234.0, CMSSW_11_0_0_pre13

step 2

MemoryReport> Peak virtual size 7054.16 Mbytes
Key events increasing vsize:
[15] run: 1 lumi: 1 event: 15 vsize = 5646.11 deltaVsize = 512 rss = 3068.17 delta = -40.4805
[70] run: 1 lumi: 1 event: 70 vsize = 6286.12 deltaVsize = 640 rss = 3314.41 delta = 246.242
[240] run: 1 lumi: 1 event: 240 vsize = 7054.15 deltaVsize = 768 rss = 3719.82 delta = 405.414
[268] run: 1 lumi: 1 event: 268 vsize = 7054.16 deltaVsize = 0.0117188 rss = 3582.66 delta = -137.168
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[270] run: 1 lumi: 1 event: 270 vsize = 7054.16 deltaVsize = 0 rss = 3610.56 delta = 27.9023
[269] run: 1 lumi: 1 event: 269 vsize = 7054.16 deltaVsize = 0 rss = 3634.46 delta = 51.8047
[268] run: 1 lumi: 1 event: 268 vsize = 7054.16 deltaVsize = 0.0117188 rss = 3582.66 delta = -137.168
TimeReport> Time report complete in 41886 seconds
Time Summary:

Min event: 68.2551
Max event: 108.076
Avg event: 82.8899
Total loop: 41845.4
Total init: 40.1771
Total job: 41886
EventSetup Lock: 0.000114441
EventSetup Get: 79.5282
Event Throughput: 0.0119487 ev/s
CPU Summary:
Total loop: 39577.6
Total init: 13.101
Total extra: 0
Total job: 39591.1

step 3

MemoryReport> Peak virtual size 13726.6 Mbytes
Key events increasing vsize:
[28] run: 1 lumi: 1 event: 28 vsize = 10145.4 deltaVsize = 1024 rss = 6748.63 delta = 115.246
[155] run: 1 lumi: 1 event: 155 vsize = 11425.5 deltaVsize = 1280 rss = 7780.5 delta = 1031.87
[453] run: 1 lumi: 1 event: 453 vsize = 13726.6 deltaVsize = 1533 rss = 8069.85 delta = 289.352
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[0] run: 0 lumi: 0 event: 0 vsize = 0 deltaVsize = 0 rss = 0 delta = 0
[455] run: 1 lumi: 1 event: 455 vsize = 13726.6 deltaVsize = 0 rss = 7639.76 delta = -430.086
[454] run: 1 lumi: 1 event: 454 vsize = 13726.6 deltaVsize = 0 rss = 7721.98 delta = -347.867
[453] run: 1 lumi: 1 event: 453 vsize = 13726.6 deltaVsize = 1533 rss = 8069.85 delta = -182.488
TimeReport> Time report complete in 94989.4 seconds
Time Summary:

Min event: 111.798
Max event: 668.458
Avg event: 185.843
Total loop: 94761.4
Total init: 204.078
Total job: 94989.4
EventSetup Lock: 0.000244617
EventSetup Get: 86.0717
Event Throughput: 0.00527641 ev/s
CPU Summary:
Total loop: 87265
Total init: 106.86
Total extra: 0
Total job: 87394.8

kpedro88 · 2020-01-27T19:27:59Z

@casarsa thanks, this looks reasonable to me.

kpedro88 · 2020-01-27T19:28:04Z

+upgrade

cmsbuild · 2020-01-27T19:28:21Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @davidlange6, @silviodonato, @fabiocos (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-01-27T20:01:46Z

please test workflow 23434.1001

cmsbuild · 2020-01-27T20:02:23Z

The tests are being triggered in jenkins.
Test Parameters:

MATRIX_EXTRAS = 23434.1001
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4373/console Started: 2020/01/27 21:17

cmsbuild · 2020-01-27T21:55:07Z

+1
Tested at: 3a350ad
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ea5a91/4373/summary.html
CMSSW: CMSSW_11_1_X_2020-01-27-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-01-27T21:55:11Z

Comparison job queued.

cmsbuild · 2020-01-27T23:31:37Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-ea5a91/4373/summary.html

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/results/JR-comparison/PR-ea5a91/23434.1001_TTbar_14TeV+RecoFullGlobalPU_TestOldDigi_2026D49PU+HARVESTFullGlobalPU_2026D49PU

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 9761 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2697090
DQMHistoTests: Total failures: 8371
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2688373
DQMHistoTests: Total skipped: 346
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

silviodonato · 2020-01-28T09:39:13Z

+1

casarsa added 4 commits December 5, 2019 12:12

Added a Gaussian smearing to the BTL time to account for the effect o…

e43a9b2

…f photon tails from earlier OOT hits

Times are saved in the MTDSimHitDataAccumulator modulo 25ns to be com…

35d029c

…patible with the premixing packing

Added a check on the index of the container of BTL hits extracted fro…

915730f

…m the premixing bank

Formatted the code

3a350ad

cmsbuild added this to the CMSSW_11_1_X milestone Jan 17, 2020

cmsbuild added code-checks-pending comparison-pending orp-pending pending-signatures simulation-pending tests-pending upgrade-pending labels Jan 17, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 17, 2020

cmsbuild added tests-started and removed tests-pending labels Jan 17, 2020

cmsbuild added tests-approved and removed tests-started labels Jan 17, 2020

cmsbuild added comparison-available and removed comparison-pending labels Jan 17, 2020

cmsbuild added simulation-approved and removed simulation-pending labels Jan 25, 2020

cmsbuild added fully-signed upgrade-approved and removed pending-signatures upgrade-pending labels Jan 27, 2020

cmsbuild added comparison-pending tests-pending and removed comparison-available tests-approved labels Jan 27, 2020

cmsbuild added tests-started and removed tests-pending labels Jan 27, 2020

cmsbuild added tests-approved and removed tests-started labels Jan 27, 2020

cmsbuild added comparison-available and removed comparison-pending labels Jan 27, 2020

cmsbuild added orp-approved and removed orp-pending labels Jan 28, 2020

cmsbuild merged commit 2e0839a into cms-sw:master Jan 28, 2020

casarsa deleted the mc-simBTL-OOT-fixPreMixing branch February 6, 2020 14:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BTL simulation: OOT effects + premix fix #28757

BTL simulation: OOT effects + premix fix #28757

casarsa commented Jan 17, 2020

cmsbuild commented Jan 17, 2020

cmsbuild commented Jan 17, 2020

cmsbuild commented Jan 17, 2020

fabiocos commented Jan 17, 2020

cmsbuild commented Jan 17, 2020 •

edited

Loading

cmsbuild commented Jan 17, 2020

cmsbuild commented Jan 17, 2020

cmsbuild commented Jan 17, 2020

silviodonato commented Jan 21, 2020

makortel commented Jan 21, 2020

casarsa commented Jan 27, 2020

kpedro88 commented Jan 27, 2020

kpedro88 commented Jan 27, 2020

cmsbuild commented Jan 27, 2020

silviodonato commented Jan 27, 2020

cmsbuild commented Jan 27, 2020 •

edited

Loading

cmsbuild commented Jan 27, 2020

cmsbuild commented Jan 27, 2020

cmsbuild commented Jan 27, 2020

silviodonato commented Jan 28, 2020

BTL simulation: OOT effects + premix fix #28757

BTL simulation: OOT effects + premix fix #28757

Conversation

casarsa commented Jan 17, 2020

PR description:

PR validation:

cmsbuild commented Jan 17, 2020

cmsbuild commented Jan 17, 2020

cmsbuild commented Jan 17, 2020

fabiocos commented Jan 17, 2020

cmsbuild commented Jan 17, 2020 • edited Loading

cmsbuild commented Jan 17, 2020

cmsbuild commented Jan 17, 2020

cmsbuild commented Jan 17, 2020

silviodonato commented Jan 21, 2020

makortel commented Jan 21, 2020

casarsa commented Jan 27, 2020

kpedro88 commented Jan 27, 2020

kpedro88 commented Jan 27, 2020

cmsbuild commented Jan 27, 2020

silviodonato commented Jan 27, 2020

cmsbuild commented Jan 27, 2020 • edited Loading

cmsbuild commented Jan 27, 2020

cmsbuild commented Jan 27, 2020

cmsbuild commented Jan 27, 2020

silviodonato commented Jan 28, 2020

cmsbuild commented Jan 17, 2020 •

edited

Loading

cmsbuild commented Jan 27, 2020 •

edited

Loading