avoid overflowing `firstStrip_` in case of approximated clusters #38735

mmusich · 2022-07-14T10:06:09Z

resolves #38729

PR description:

See #38729 (comment)

PR validation:

Tested running 140.58 successfully under CMSSW_12_5_ASAN_X_2022-07-13-1100

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

N/A

cmsbuild · 2022-07-14T10:11:56Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38735/31048

This PR adds an extra 12KB to repository

cmsbuild · 2022-07-14T10:12:18Z

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

DataFormats/SiStripCluster (reconstruction)

@jpata, @cmsbuild, @clacaputo can you please review it and eventually sign? Thanks.
@echabert, @swertz, @robervalwalsh, @rovere, @VinInn, @alesaggio, @threus this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

mmusich · 2022-07-14T10:21:24Z

test parameters:

workflow = 140.58

mmusich · 2022-07-14T10:21:39Z

@cmsbuild, please test for CMSSW_12_5_ASAN_X

perrotta · 2022-07-14T12:01:24Z

DataFormats/SiStripCluster/src/SiStripCluster.cc

@@ -28,7 +28,7 @@ SiStripCluster::SiStripCluster(const SiStripApproximateCluster cluster) : error_
  amplitudes_.resize(cluster.width(), cluster.avgCharge());

  //initialize firstStrip_
-  firstStrip_ = cluster.barycenter() - cluster.width() / 2;
+  firstStrip_ = std::max(0.f, cluster.barycenter() - cluster.width() / 2);


Maybe you can recycle here the same barycenter_ already computed a few lines above...

sure. I have more doubts though:

given cluster.width() here is a uint8_t

cmssw/DataFormats/SiStripCluster/interface/SiStripApproximateCluster.h

Line 25 in 9e6c192

uint8_t width() const { return width_; }

should it be divided by 2.f to avoid truncating the half-width of one half-strip?

I am not sure if there's a better (more efficient) way of computing the max since this should be a pretty intensive call in production.

@abaty @mandrenguyen

Hi,

I think Marco is correct on item 1, we should be dividing by a float here. I am not familiar with the most efficient way of implementing item 2, although we could check the timing if it is a strong concern.

plase 0.5f*float(cluster.width())
max is not slower than the above (and much faster than cluster.width() / 2.)

cmsbuild · 2022-07-14T13:22:34Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7fab06/26240/summary.html
COMMIT: 9330786
CMSSW: CMSSW_12_5_ASAN_X_2022-07-13-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38735/26240/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7fab06/26240/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7fab06/26240/git-merge-result

RelVals

140.58140.58_RunHI2018+RunHI2018+RAWPRIMEHI18+RECOHID18APPROXCLUSTERS+HARVESTDHI18/step3_RunHI2018+RunHI2018+RAWPRIMEHI18+RECOHID18APPROXCLUSTERS+HARVESTDHI18.log

mmusich · 2022-07-15T08:29:56Z

the other problem we have uncovered now is coming from the other "side" of the module, see log.

By adding:

diff --git a/RecoLocalTracker/SiStripClusterizer/src/SiStripClusterInfo.cc b/RecoLocalTracker/SiStripClusterizer/src/SiStripClusterInfo.cc
index dcdf774f0ef..58d1cc53b97 100644
--- a/RecoLocalTracker/SiStripClusterizer/src/SiStripClusterInfo.cc
+++ b/RecoLocalTracker/SiStripClusterizer/src/SiStripClusterInfo.cc
@@ -42,6 +42,8 @@ std::vector<float> SiStripClusterInfo::stripNoisesRescaledByGain() const {
   std::vector<float> results;
   results.reserve(width());
   for (size_t i = 0, e = width(); i < e; i++) {
+    std::cout << "i: " << i << " firstStrip: " << firstStrip() << " getting strip:" << firstStrip() + i << std::endl;
+
     results.push_back(siStripNoises_->getNoise(firstStrip() + i, detNoiseRange) /
                       siStripGain_->getStripGain(firstStrip() + i, detGainRange));
   }

I get:

i: 0 firstStrip: 757 getting strip:757
i: 1 firstStrip: 757 getting strip:758
i: 2 firstStrip: 757 getting strip:759
i: 3 firstStrip: 757 getting strip:760
i: 4 firstStrip: 757 getting strip:761
i: 5 firstStrip: 757 getting strip:762
i: 6 firstStrip: 757 getting strip:763
i: 7 firstStrip: 757 getting strip:764
i: 8 firstStrip: 757 getting strip:765
i: 9 firstStrip: 757 getting strip:766
i: 10 firstStrip: 757 getting strip:767
i: 11 firstStrip: 757 getting strip:768
=================================================================
==3626964==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x7f8e8c3717ff at pc 0x7f8f1bb0beba bp 0x7ffe3fda2080 sp 0x7ffe3fda2078
READ of size 1 at 0x7f8e8c3717ff thread T0

so it can happen that firstStrip() can exceed the total number of strips in the module...
I am not sure how to solve this, because one could clamp between 0.f and (total number of strips - cluster width), but the SiStripCluster is agnostic w.r.t. the DetId and the total number of strips depends on the DetId...

mmusich · 2022-07-15T11:17:28Z

with bd09b48 the failing workflow runs, but it doesn't address the case of the 4 APV modules:

I am not sure how to solve this, because one could clamp between 0.f and (total number of strips - cluster width), but the SiStripCluster is agnostic w.r.t. the DetId and the total number of strips depends on the DetId...

cmsbuild · 2022-07-15T11:24:04Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38735/31068

This PR adds an extra 12KB to repository

cmsbuild · 2022-07-15T11:24:37Z

Pull request #38735 was updated. @jpata, @cmsbuild, @clacaputo can you please check and sign again.

cmsbuild · 2022-07-19T11:09:42Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38735/31128

This PR adds an extra 16KB to repository

cmsbuild · 2022-07-19T11:10:08Z

Pull request #38735 was updated. @jpata, @cmsbuild, @clacaputo can you please check and sign again.

clacaputo · 2022-07-19T11:11:38Z

@cmsbuild, please test for CMSSW_12_5_ASAN_X

cmsbuild · 2022-07-19T14:58:57Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7fab06/26321/summary.html
COMMIT: 5895157
CMSSW: CMSSW_12_5_ASAN_X_2022-07-18-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38735/26321/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test test_ProduceSystematicMisalignment had ERRORS
---> test testAlignmentOfflineValidation had ERRORS
---> test test_PixelBaryCentreTool had ERRORS
---> test testPedeCampaign had ERRORS
and more ...

mmusich · 2022-07-19T17:38:31Z

-1
Failed Tests: UnitTests

Failures are unrelated, already seen in the IB: https://cmssdt.cern.ch/SDT/cgi-bin/showBuildLogs.py/el8_amd64_gcc10/www/mon/12.5.ASAN-mon-11/CMSSW_12_5_ASAN_X_2022-07-18-1100?utests

clacaputo · 2022-07-20T16:51:06Z

+reconstruction

the change proposed resolve Out of bounds read in SiStripMonitorCluster::analyze #38729
tested locally in CMSSW_12_5_ASAN_X_2022-07-18-1100
UnitTests failing are unrelated to this PR as stated here and tested locally

cmsbuild · 2022-07-20T16:51:30Z

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2022-07-20T17:16:19Z

please test
(As it was changed since #38735 (comment))

cmsbuild · 2022-07-20T21:59:00Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7fab06/26349/summary.html
COMMIT: 5895157
CMSSW: CMSSW_12_5_X_2022-07-20-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/38735/26349/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

@slava77 comparisons for the following workflows were not done due to missing matrix map:

/data/cmsbld/jenkins/workspace/compare-root-files-short-matrix/data/PR-7fab06/140.58_RunHI2018+RunHI2018+RAWPRIMEHI18+RECOHID18APPROXCLUSTERS+HARVESTDHI18

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 51
DQMHistoTests: Total histograms compared: 3723682
DQMHistoTests: Total failures: 7276
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3716384
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 50 files compared)
Checked 212 log files, 45 edm output root files, 51 DQM output files
TriggerResults: no differences found

qliphy · 2022-07-21T01:05:29Z

+1

cmsbuild added this to the CMSSW_12_5_X milestone Jul 14, 2022

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Jul 14, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 14, 2022

cmsbuild added tests-started and removed tests-pending labels Jul 14, 2022

mmusich changed the title ~~avoid overflowing firstStrip_ in case of approximated clusters~~ [RFC] avoid overflowing firstStrip_ in case of approximated clusters Jul 14, 2022

perrotta reviewed Jul 14, 2022

View reviewed changes

cmsbuild added tests-rejected and removed tests-started labels Jul 14, 2022

mmusich force-pushed the fixRawPrimeWorkflow branch from 9330786 to bd09b48 Compare July 15, 2022 11:16

cmsbuild added code-checks-pending tests-pending and removed tests-rejected code-checks-approved labels Jul 15, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 15, 2022

cmsbuild added code-checks-pending tests-pending and removed code-checks-approved labels Jul 19, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 19, 2022

cmsbuild added tests-started and removed tests-pending labels Jul 19, 2022

cmsbuild added tests-rejected and removed tests-started labels Jul 19, 2022

cmsbuild added fully-signed reconstruction-approved and removed reconstruction-pending pending-signatures labels Jul 20, 2022

cmsbuild added tests-started and removed tests-rejected labels Jul 20, 2022

cmsbuild added tests-approved and removed tests-started labels Jul 20, 2022

cmsbuild added orp-approved and removed orp-pending labels Jul 21, 2022

cmsbuild merged commit d24ce62 into cms-sw:master Jul 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

avoid overflowing `firstStrip_` in case of approximated clusters #38735

avoid overflowing `firstStrip_` in case of approximated clusters #38735

mmusich commented Jul 14, 2022

cmsbuild commented Jul 14, 2022

cmsbuild commented Jul 14, 2022

mmusich commented Jul 14, 2022

mmusich commented Jul 14, 2022

perrotta Jul 14, 2022

mmusich Jul 14, 2022 •

edited

Loading

abaty Jul 14, 2022

VinInn Jul 15, 2022 •

edited

Loading

cmsbuild commented Jul 14, 2022

mmusich commented Jul 15, 2022 •

edited

Loading

mmusich commented Jul 15, 2022

cmsbuild commented Jul 15, 2022

cmsbuild commented Jul 15, 2022

cmsbuild commented Jul 19, 2022

cmsbuild commented Jul 19, 2022

clacaputo commented Jul 19, 2022

cmsbuild commented Jul 19, 2022

mmusich commented Jul 19, 2022

clacaputo commented Jul 20, 2022

cmsbuild commented Jul 20, 2022

perrotta commented Jul 20, 2022

cmsbuild commented Jul 20, 2022

qliphy commented Jul 21, 2022

avoid overflowing firstStrip_ in case of approximated clusters #38735

avoid overflowing firstStrip_ in case of approximated clusters #38735

Conversation

mmusich commented Jul 14, 2022

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Jul 14, 2022

cmsbuild commented Jul 14, 2022

mmusich commented Jul 14, 2022

mmusich commented Jul 14, 2022

perrotta Jul 14, 2022

Choose a reason for hiding this comment

mmusich Jul 14, 2022 • edited Loading

Choose a reason for hiding this comment

abaty Jul 14, 2022

Choose a reason for hiding this comment

VinInn Jul 15, 2022 • edited Loading

Choose a reason for hiding this comment

cmsbuild commented Jul 14, 2022

RelVals

mmusich commented Jul 15, 2022 • edited Loading

mmusich commented Jul 15, 2022

cmsbuild commented Jul 15, 2022

cmsbuild commented Jul 15, 2022

cmsbuild commented Jul 19, 2022

cmsbuild commented Jul 19, 2022

clacaputo commented Jul 19, 2022

cmsbuild commented Jul 19, 2022

Unit Tests

mmusich commented Jul 19, 2022

clacaputo commented Jul 20, 2022

cmsbuild commented Jul 20, 2022

perrotta commented Jul 20, 2022

cmsbuild commented Jul 20, 2022

Comparison Summary

qliphy commented Jul 21, 2022

avoid overflowing `firstStrip_` in case of approximated clusters #38735

avoid overflowing `firstStrip_` in case of approximated clusters #38735

mmusich Jul 14, 2022 •

edited

Loading

VinInn Jul 15, 2022 •

edited

Loading

mmusich commented Jul 15, 2022 •

edited

Loading