fix peak mode SiStrip noise payload in 2018 cosmics simulation #41120

mmusich · 2023-03-21T15:54:05Z

resolves #41108

PR description:

In DBG IBs, the function SiStripNoises::getNoise verifies that the strip for which the noise is asked lies in the SiStripNoises::Range of that DetId:

cmssw/CondFormats/SiStripObjects/interface/SiStripNoises.h

Lines 73 to 78 in 9a02541

    
             static float getNoise(uint16_t strip, const Range& range) { 
        
           #ifdef EDM_ML_DEBUG 
        
               verify(strip, range); 
        
           #endif 
        
               return getNoiseFast(strip, range); 
        
             }

definition of SiStripNoises::verify

cmssw/CondFormats/SiStripObjects/src/SiStripNoises.cc

Lines 68 to 72 in 9a02541

    
           void SiStripNoises::verify(uint16_t strip, const Range& range) { 
        
             if (9 * strip >= (range.second - range.first) * 8) 
        
               throw cms::Exception("CorruptedData") 
        
                   << "[SiStripNoises::getNoise] looking for SiStripNoises for a strip out of range: strip " << strip; 
        
           }

if that's not verified, the code will throw an exception.
In normal builds if that doesn't happen, there is no exception thrown, though of course the used event setup data might be wrong.
This PR proposes a new peak mode noise payload that ensures that all per-strip noise data complies with the verify requirement.
@cms-sw/alca-l2 I propose here a candidate, please let me know if you prefer to transform it upfront into a full-fledged Global Tag, I can then subsequently pick it up here.

PR validation:

run successfully runTheMatrix.py -l 7.4

For reference here's a comparison of the bugged payload (SiStripNoise_PeakMode_2018_Minus20C_v0_mc) vs the fixed one (SiStripNoise_PeakMode_2018_Minus20C_v0_mc_fixed)

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

N/A

mmusich · 2023-03-21T16:00:13Z

type bug-fix

cmsbuild · 2023-03-21T16:02:38Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41120/34759

This PR adds an extra 12KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/AlCa/python/autoCond.py modified in PR(s): Include Phase2 TkAl Rcds in Phase2 MC GT with T21 tags #41119, HI 2023 relval wfs + realistic 2022 vtx smearing #41095

cmsbuild · 2023-03-21T16:03:01Z

A new Pull Request was created by @mmusich (Marco Musich) for master.

It involves the following packages:

Configuration/AlCa (alca)

@cmsbuild, @tvami, @saumyaphor4252, @francescobrivio can you please review it and eventually sign? Thanks.
@Martin-Grunewald, @missirol, @mmusich, @fabiocos, @tocheng this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

mmusich · 2023-03-21T16:03:17Z

@cmsbuild, please test for CMSSW_13_1_DBG_X

mmusich · 2023-03-22T09:20:28Z

is it normal that running the DBG relvals takes so long?

mmusich · 2023-03-22T10:29:07Z

@cmsbuild, please abort

mmusich · 2023-03-22T10:29:19Z

@cmsbuild, please test workflow 7.4

tvami · 2023-03-22T13:26:54Z

is it normal that running the DBG relvals takes so long?

@smuzaffar ?

cmsbuild · 2023-03-22T13:39:48Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e90a04/31506/summary.html
COMMIT: 3e98439
CMSSW: CMSSW_13_1_X_2023-03-21-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41120/31506/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 18 lines to the logs
Reco comparison results: 9 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3585875
DQMHistoTests: Total failures: 9
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3585844
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 218 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

tvami · 2023-03-22T13:49:32Z

@cmsbuild, please test workflow 7.4 for CMSSW_13_1_DBG_X

mmusich · 2023-03-22T13:52:07Z

please test workflow 7.4 for CMSSW_13_1_DBG_X

I don't see how this is going to help (I see it re-building everything from scratch again).

tvami · 2023-03-22T13:53:18Z

I was just hoping the issue with the DBG build was temporary. Anyway, Marco, I cut a new real GT for you 131X_upgrade2018cosmics_realistic_peak_v2

tvami · 2023-03-22T13:56:45Z

Also, will you submit a PR that has the code that ensures that all per-strip noise data complies with the verify requirement?

mmusich · 2023-03-22T14:02:57Z

Also, will you submit a PR that has the code that ensures that all per-strip noise data complies with the verify requirement?

sure, see #41127

smuzaffar · 2023-03-22T14:02:59Z

is it normal that running the DBG relvals takes so long?

yes , runtime for DBG is 2/3 time long so yes it can take a lot time. Also as we do not build DBG IB every day so you might pick up an old build (few days old) and then bot might have to rebuild a lot stuff

mmusich · 2023-03-22T14:05:06Z

I cut a new real GT for you 131X_upgrade2018cosmics_realistic_peak_v2

thanks, it's included now in this PR.

cmsbuild · 2023-03-22T14:06:03Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41120/34772

This PR adds an extra 12KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File Configuration/AlCa/python/autoCond.py modified in PR(s): Update BeamSpot for 2022 EOY MC #41123

cmsbuild · 2023-03-22T14:06:33Z

Pull request #41120 was updated. @cmsbuild, @tvami, @saumyaphor4252, @francescobrivio can you please check and sign again.

tvami · 2023-03-22T14:07:53Z

@cmsbuild, please test workflow 7.4 for CMSSW_13_1_DBG_X

and I guess we need to be patient with it

mmusich · 2023-03-22T14:11:34Z

and I guess we need to be patient with it

I guess you can also locally test and sign...

tvami · 2023-03-22T14:56:07Z

+alca

tested locally with the changes

cmsbuild · 2023-03-22T14:56:34Z

This pull request is fully signed and it will be integrated in one of the next master IBs after it passes the integration tests. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

mmusich · 2023-03-23T14:41:34Z

Waiting for what?

tvami · 2023-03-23T14:43:39Z

@cms-sw/orp-l2 can we merge this based on #41120 (comment) and that the std CMSSW test did pass #41120 (comment) ?

perrotta · 2023-03-23T15:02:55Z

@cmsbuild, please test workflow 7.4
(let have at least a round of "normal" bot tests run after the last update)

cmsbuild · 2023-03-23T18:05:18Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-e90a04/31565/summary.html
COMMIT: e4a92e9
CMSSW: CMSSW_13_1_X_2023-03-22-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41120/31565/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 9 lines to the logs
Reco comparison results: 9 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3585875
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3585847
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
Checked 218 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

perrotta · 2023-03-23T18:14:18Z

+1

cmsbuild added this to the CMSSW_13_1_X milestone Mar 21, 2023

cmsbuild added alca-pending pending-signatures tests-pending orp-pending code-checks-pending labels Mar 21, 2023

mmusich mentioned this pull request Mar 21, 2023

[DBG] Exception of category CorruptedData in RelVal wf 7.4 #41108

Closed

cmsbuild added the bug-fix label Mar 21, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 21, 2023

cmsbuild added tests-started and removed tests-pending labels Mar 21, 2023

mmusich changed the title ~~fix peak mode noise payload in 2018 cosmics simulation~~ fix peak mode SiStrip noise payload in 2018 cosmics simulation Mar 21, 2023

cmsbuild mentioned this pull request Mar 21, 2023

Update BeamSpot for 2022 EOY MC #41123

Merged

cmsbuild added tests-approved and removed tests-started labels Mar 22, 2023

fix peak mode noise payload in 2018 cosmics simulation

e4a92e9

cmsbuild added the code-checks-pending label Mar 22, 2023

mmusich mentioned this pull request Mar 22, 2023

introduce SiStripNoisesAndBadCompsChecker #41127

Merged

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 22, 2023

cmsbuild added tests-started and removed tests-pending labels Mar 22, 2023

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Mar 22, 2023

cmsbuild added tests-approved and removed tests-started labels Mar 23, 2023

cmsbuild added orp-approved and removed orp-pending labels Mar 23, 2023

cmsbuild merged commit d16a1cc into cms-sw:master Mar 23, 2023

mmusich deleted the fixPeakModeCosmicSimu_in_DBG_X_IBs branch March 23, 2023 18:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix peak mode SiStrip noise payload in 2018 cosmics simulation #41120

fix peak mode SiStrip noise payload in 2018 cosmics simulation #41120

mmusich commented Mar 21, 2023

mmusich commented Mar 21, 2023

cmsbuild commented Mar 21, 2023

cmsbuild commented Mar 21, 2023

mmusich commented Mar 21, 2023

mmusich commented Mar 22, 2023

mmusich commented Mar 22, 2023 •

edited

Loading

mmusich commented Mar 22, 2023

tvami commented Mar 22, 2023

cmsbuild commented Mar 22, 2023

tvami commented Mar 22, 2023

mmusich commented Mar 22, 2023

tvami commented Mar 22, 2023

tvami commented Mar 22, 2023

mmusich commented Mar 22, 2023

smuzaffar commented Mar 22, 2023

mmusich commented Mar 22, 2023

cmsbuild commented Mar 22, 2023

cmsbuild commented Mar 22, 2023

tvami commented Mar 22, 2023

mmusich commented Mar 22, 2023

tvami commented Mar 22, 2023

cmsbuild commented Mar 22, 2023

mmusich commented Mar 23, 2023

tvami commented Mar 23, 2023

perrotta commented Mar 23, 2023

cmsbuild commented Mar 23, 2023

perrotta commented Mar 23, 2023

	static float getNoise(uint16_t strip, const Range& range) {
	#ifdef EDM_ML_DEBUG
	verify(strip, range);
	#endif
	return getNoiseFast(strip, range);
	}

	void SiStripNoises::verify(uint16_t strip, const Range& range) {
	if (9 * strip >= (range.second - range.first) * 8)
	throw cms::Exception("CorruptedData")
	<< "[SiStripNoises::getNoise] looking for SiStripNoises for a strip out of range: strip " << strip;
	}

fix peak mode SiStrip noise payload in 2018 cosmics simulation #41120

fix peak mode SiStrip noise payload in 2018 cosmics simulation #41120

Conversation

mmusich commented Mar 21, 2023

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

mmusich commented Mar 21, 2023

cmsbuild commented Mar 21, 2023

cmsbuild commented Mar 21, 2023

mmusich commented Mar 21, 2023

mmusich commented Mar 22, 2023

mmusich commented Mar 22, 2023 • edited Loading

mmusich commented Mar 22, 2023

tvami commented Mar 22, 2023

cmsbuild commented Mar 22, 2023

Comparison Summary

tvami commented Mar 22, 2023

mmusich commented Mar 22, 2023

tvami commented Mar 22, 2023

tvami commented Mar 22, 2023

mmusich commented Mar 22, 2023

smuzaffar commented Mar 22, 2023

mmusich commented Mar 22, 2023

cmsbuild commented Mar 22, 2023

cmsbuild commented Mar 22, 2023

tvami commented Mar 22, 2023

mmusich commented Mar 22, 2023

tvami commented Mar 22, 2023

cmsbuild commented Mar 22, 2023

mmusich commented Mar 23, 2023

tvami commented Mar 23, 2023

perrotta commented Mar 23, 2023

cmsbuild commented Mar 23, 2023

Comparison Summary

perrotta commented Mar 23, 2023

mmusich commented Mar 22, 2023 •

edited

Loading