PPS Pixel DQM: assign errors to right plots #41649

fabferro · 2023-05-12T09:25:42Z

PR description:

This PR is a bug fix in filling the plots in PPS Pixel DQM. Using the DAQ map, errors coming from pixels are assigned to the right detector plane instead of being put in a unidentified-detector plot, unless strictly necessary.

PR validation:

With matrix wfs 138.4 (run3) and 136.874 (run2) and with brand new 2023 data. Errors are now put inside the right plots.

This fix is needed for the data taking asap. Backports are therefore needed.

cmsbuild · 2023-05-12T09:32:47Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41649/35528

This PR adds an extra 24KB to repository

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41649/35528/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41649/35528/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

cmsbuild · 2023-05-12T09:54:15Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41649/35529

This PR adds an extra 24KB to repository

cmsbuild · 2023-05-12T09:54:40Z

A new Pull Request was created by @fabferro (Fabrizio Ferro) for master.

It involves the following packages:

DQM/CTPPS (dqm)

@nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please review it and eventually sign? Thanks.
@grzanka this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

perrotta · 2023-05-13T04:05:30Z

please test

cmsbuild · 2023-05-13T08:05:57Z

-1

Failed Tests: RelVals
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0814e4/32604/summary.html
COMMIT: c44a7cb
CMSSW: CMSSW_13_2_X_2023-05-12-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/41649/32604/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

1306.0

----- Begin Fatal Exception 13-May-2023 09:11:31 CEST-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Calling method for module CTPPSPixelDQMSource/'ctppsPixelDQMOfflineSource'
Exception Message:
No "CTPPSPixelDAQMappingRcd" record found in the EventSetup.

 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

1330.0

----- Begin Fatal Exception 13-May-2023 09:12:30 CEST-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Calling method for module CTPPSPixelDQMSource/'ctppsPixelDQMOfflineSource'
Exception Message:
No "CTPPSPixelDAQMappingRcd" record found in the EventSetup.

 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

25202.0

----- Begin Fatal Exception 13-May-2023 09:19:24 CEST-----------------------
An exception of category 'NoRecord' occurred while
   [0] Processing  Event run: 1 lumi: 1 event: 1 stream: 0
   [1] Running path 'dqmoffline_step'
   [2] Calling method for module CTPPSPixelDQMSource/'ctppsPixelDQMOfflineSource'
Exception Message:
No "CTPPSPixelDAQMappingRcd" record found in the EventSetup.

 Please add an ESSource or ESProducer that delivers such a record.
----- End Fatal Exception -------------------------------------------------

fabferro · 2023-05-13T12:41:15Z

It looks like auto:run2_mc GT (ie: 123X_mcRun2_asymptotic_v1) does not contain CTPPSPixelDAQMappingRcd. It is needed in the updates introduced by this PR. It should be added. @wpcarvalho

emanueleusai · 2023-05-14T22:51:05Z

type ctpps

francescobrivio · 2023-05-15T11:53:31Z

Hi @fabferro

the CTPPSPixelDAQMappingRcd were introduced in the MC GTs only for 2018 and Run3, as detailed in #30890 (from 2020) and specifically requested in https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4315/1.html by PPS experts.

So something must be wrong in the PR if you are trying to consume that record in workflows older than 2018...maybe you need to restrict this to Run3 workflows only?

fabferro · 2023-05-15T12:20:48Z

Hi @fabferro

the CTPPSPixelDAQMappingRcd were introduced in the MC GTs only for 2018 and Run3, as detailed in #30890 (from 2020) and specifically requested in https://hypernews.cern.ch/HyperNews/CMS/get/calibrations/4315/1.html by PPS experts.

So something must be wrong in the PR if you are trying to consume that record in workflows older than 2018...maybe you need to restrict this to Run3 workflows only?

@francescobrivio : I did not remember that discussion. You are suggesting to use era_modifiers?

cmsbuild · 2023-05-16T17:45:45Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41649/35574

This PR adds an extra 24KB to repository

cmsbuild · 2023-05-16T17:46:09Z

Pull request #41649 was updated. @nothingface0, @emanueleusai, @cmsbuild, @pmandrik, @syuvivida, @tjavaid, @micsucmed, @rvenditti can you please check and sign again.

tvami · 2023-05-16T19:59:18Z

@cmsbuild , please test

cmsbuild · 2023-05-16T22:23:49Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-0814e4/32677/summary.html
COMMIT: aa92624
CMSSW: CMSSW_13_2_X_2023-05-16-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41649/32677/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

There are some workflows for which there are errors in the baseline:
140.031 step 3
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Summary:

You potentially removed 3 lines from the logs
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3222155
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3222130
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: -92481.324 KiB( 47 files compared)
DQMHistoSizes: changed ( 1306.0,... ): -16565.990 KiB CTPPS/TimingDiamond
DQMHistoSizes: changed ( 1306.0,... ): -11346.722 KiB CTPPS/TrackingPixel
DQMHistoSizes: changed ( 1306.0,... ): -2360.820 KiB CTPPS/DiamondSampic
DQMHistoSizes: changed ( 1306.0,... ): -553.576 KiB CTPPS/common
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: no differences found

emanueleusai · 2023-05-17T05:02:23Z

+1

cmsbuild · 2023-05-17T05:02:44Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-05-17T17:13:06Z

Reasoning with @wpcarvalho we recalled that, actually, the current PPS detectors were not even present in 2016. Thus, probably, the best option is to add a few lines in ctppsDQM_cff.py to prevent DQM to be called for 2016 data/mc processes. Like: from Configuration.Eras.Modifier_ctpps_2016_cff import ctpps_2016 ctpps_2016.toReplaceWith( _ctppsDQMOfflineSource, cms.Sequence( ) )

@fabferro I am puzzled: in #41649 (comment) is written that "the CTPPSPixelDAQMappingRcd were introduced in the MC GTs only for 2018 and Run3". Ok for clearing the ctppsDQMOfflineSource from runs in the Run2_2016 era, but how can this even work for a Run2_2017 era then?

fabferro · 2023-05-18T10:23:48Z

Reasoning with @wpcarvalho we recalled that, actually, the current PPS detectors were not even present in 2016. Thus, probably, the best option is to add a few lines in ctppsDQM_cff.py to prevent DQM to be called for 2016 data/mc processes. Like: from Configuration.Eras.Modifier_ctpps_2016_cff import ctpps_2016 ctpps_2016.toReplaceWith( _ctppsDQMOfflineSource, cms.Sequence( ) )

@fabferro I am puzzled: in #41649 (comment) is written that "the CTPPSPixelDAQMappingRcd were introduced in the MC GTs only for 2018 and Run3". Ok for clearing the ctppsDQMOfflineSource from runs in the Run2_2016 era, but how can this even work for a Run2_2017 era then?

Actually the campaign for updating PPS MC tags started from 2017, at least for pixels. I'm not 100% sure that ALL MC GTs were updated, but all used 131X_mc2017_* were.

perrotta · 2023-05-18T10:29:25Z

Actually the campaign for updating PPS MC tags started from 2017, at least for pixels. I'm not 100% sure that ALL MC GTs were updated, but all used 131X_mc2017_* were.

Ok, thank you @fabferro : this explains why we got no errors from the 2017 workflows

perrotta · 2023-05-18T10:29:53Z

+1

Assign errors to right plots

d48c7a7

cmsbuild added this to the CMSSW_13_2_X milestone May 12, 2023

cmsbuild added dqm-pending pending-signatures tests-pending orp-pending code-checks-pending labels May 12, 2023

cmsbuild added code-checks-rejected and removed code-checks-pending labels May 12, 2023

code format

c44a7cb

cmsbuild added code-checks-pending and removed code-checks-rejected labels May 12, 2023

fabferro mentioned this pull request May 12, 2023

[13_1_x] PPS Pixel DQM: assign errors to right plots #41650

Merged

cmsbuild added code-checks-approved and removed code-checks-pending labels May 12, 2023

fabferro mentioned this pull request May 12, 2023

[13_0_x] PPS Pixel DQM: assign errors to right plots #41651

Merged

cmsbuild added tests-started and removed tests-pending labels May 13, 2023

cmsbuild added tests-rejected and removed tests-started labels May 13, 2023

cmsbuild added the ctpps label May 14, 2023

cmsbuild added code-checks-rejected and removed code-checks-pending labels May 16, 2023

format fix

aa92624

cmsbuild added tests-pending code-checks-pending and removed tests-started code-checks-rejected labels May 16, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels May 16, 2023

cmsbuild added tests-started and removed tests-pending labels May 16, 2023

cmsbuild added tests-approved and removed tests-started labels May 16, 2023

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels May 17, 2023

cmsbuild added orp-approved and removed orp-pending labels May 18, 2023

cmsbuild merged commit c18ebe9 into cms-sw:master May 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PPS Pixel DQM: assign errors to right plots #41649

PPS Pixel DQM: assign errors to right plots #41649

fabferro commented May 12, 2023

cmsbuild commented May 12, 2023

cmsbuild commented May 12, 2023

cmsbuild commented May 12, 2023

perrotta commented May 13, 2023

cmsbuild commented May 13, 2023

fabferro commented May 13, 2023

emanueleusai commented May 14, 2023

francescobrivio commented May 15, 2023

fabferro commented May 15, 2023

cmsbuild commented May 16, 2023

cmsbuild commented May 16, 2023

tvami commented May 16, 2023

cmsbuild commented May 16, 2023

emanueleusai commented May 17, 2023

cmsbuild commented May 17, 2023

perrotta commented May 17, 2023

fabferro commented May 18, 2023 •

edited

Loading

perrotta commented May 18, 2023

perrotta commented May 18, 2023

PPS Pixel DQM: assign errors to right plots #41649

PPS Pixel DQM: assign errors to right plots #41649

Conversation

fabferro commented May 12, 2023

PR description:

PR validation:

cmsbuild commented May 12, 2023

cmsbuild commented May 12, 2023

cmsbuild commented May 12, 2023

perrotta commented May 13, 2023

cmsbuild commented May 13, 2023

RelVals

fabferro commented May 13, 2023

emanueleusai commented May 14, 2023

francescobrivio commented May 15, 2023

fabferro commented May 15, 2023

cmsbuild commented May 16, 2023

cmsbuild commented May 16, 2023

tvami commented May 16, 2023

cmsbuild commented May 16, 2023

Comparison Summary

emanueleusai commented May 17, 2023

cmsbuild commented May 17, 2023

perrotta commented May 17, 2023

fabferro commented May 18, 2023 • edited Loading

perrotta commented May 18, 2023

perrotta commented May 18, 2023

fabferro commented May 18, 2023 •

edited

Loading