Fix Tracks related Pixel quantities at HLT DQM (backport) #40875

arossi83 · 2023-02-24T10:51:52Z

PR description:

Monitor of the tracks related Pixel quantities at HLT DQM never works and it was disabled at some point, with this PR we introduce the changes needed (tracks refitting) in order to be able to monitor Pixel Tracks quantities at HLT DQM.
In this PR we also update the collection to be used by the Standard DQM Pixel Residuals Monitor in order to use the refitted tracks.

PR validation:

runTheMatrix.sh -l 140.063
Manually run the Online HLT client at P5 with old stored DQM stream file

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport of #40873

cmsbuild · 2023-02-24T10:52:15Z

A new Pull Request was created by @arossi83 (Alessandro Rossi) for CMSSW_13_0_X.

It involves the following packages:

DQM/Integration (dqm)
DQM/SiPixelPhase1Track (dqm)
DQMOffline/Trigger (dqm)

@emanueleusai, @cmsbuild, @syuvivida, @rvenditti, @micsucmed, @pmandrik can you please review it and eventually sign? Thanks.
@fioriNTU, @arossi83, @trocino, @batinkov, @hdelanno, @battibass, @sroychow, @jhgoh, @missirol, @HuguesBrun, @Fedespring, @threus, @jandrea, @idebruyn, @mmusich, @rociovilar, @mtosi, @cericeci, @francescobrivio this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Backported from Fix Tracks related Pixel quantities at HLT DQM #40873

mmusich · 2023-02-24T10:55:09Z

test parameters:

workflow = 140.063

mmusich · 2023-02-24T10:55:15Z

please test

mmusich · 2023-02-24T10:55:49Z

urgent

please @cms-sw/dqm-l2 consider this for integration in 13_0_0.

mmusich · 2023-02-24T12:18:17Z

@arossi83 the unit test TestDQMOnlineClient-hlt_dqm_sourceclient is failing with

----- Begin Fatal Exception 24-Feb-2023 12:25:10 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 355380 lumi: 19 event: 26716348 stream: 0
   [1] Running path 'hlt4vector'
   [2] Calling method for module SiPixelClusterShapeCacheProducer/'hltSiPixelClusterShapeCache'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: edmNew::DetSetVector<SiPixelCluster>
Looking for module label: hltSiPixelClusters
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

this happens because the input file for the test /store/express/Run2022B/ExpressPhysics/FEVT/Express-v1/000/355/380/00000/2b3f408f-883a-4f64-8083-9e5786589069.root is a FEVT file from StreamExpress and as such it doesn't have the necessary HLT collections.
The test has been running successfully in IB and PR tests so far only because all the (DQM module) consumers in its path are apparently well protected in terms of missing inputs, while SiPixelClusterShapeCacheProducer is not.
I think the right move would be to change the input of this unit test to use a file that instead does contain the HLT-produced collections, since otherwise the test is not testing anything, but I am not sure if there's any readily available such file. A FEVTHLTALL file from /HLTMonitor/Run2022*Express-v*/FEVTHLTALL would serve the purpose but I cannot find any on disk.
@cms-sw/hlt-l2 FYI.

missirol · 2023-02-24T13:30:49Z

Interesting.. I couldn't find any files from /HLTMonitor/Run2022*Express-v*/FEVTHLTALL either.

I could only find [1]; that might help for local tests, but I guess we should not rely on this non-standard path in CMSSW. Maybe, one quick workaround could be asking to copy this in a proper folder of the cmsbot cache (then CMSSW tests would see it). Not sure how to proceed.
[1]

/eos/cms/tier0/store/backfill/1/express/Tier0_REPLAY_2022/HLTMonitor/FEVTHLTALL/Express-v710/000/356/005/00000/0f207c66-8cda-4c3d-b4c8-3b063efc72e8.root

mmusich · 2023-02-24T13:36:09Z

DQMOffline/Trigger/python/SiPixel_OfflineMonitoring_cff.py

-    #+ hltSiPixelPhase1TrackClustersAnalyzer
+    hltSiPixelClusterShapeCache
+    + hltSiPixelPhase1ClustersAnalyzer
+    + refittedForPixelDQM


shouldn't this be

Suggested change

+ refittedForPixelDQM

+ hltrefittedForPixelDQM

?

Yes, It should. Just push a commit to fix it.

cmsbuild · 2023-02-24T13:51:09Z

-1

Failed Tests: UnitTests RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cc786b/30879/summary.html
COMMIT: 5f46737
CMSSW: CMSSW_13_0_X_2023-02-23-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40875/30879/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestDQMOnlineClient-hlt_dqm_sourceclient had ERRORS

RelVals-INPUT

140.54140.54_RunPA2013/step2_RunPA2013.log

Comparison Summary

There are some workflows for which there are errors in the baseline:
22034.911 step 1
23234.911 step 1
The results for the comparisons for these workflows could be incomplete
This means most likely that the IB is having errors in the relvals.The error does NOT come from this pull request

Summary:

You potentially added 84 lines to the logs
Reco comparison results: 14 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3608349
DQMHistoTests: Total failures: 7042
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3601285
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 20185.41 KiB( 49 files compared)
DQMHistoSizes: changed ( 11634.0,... ): 3364.235 KiB HLT/Pixel
Checked 217 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

mmusich · 2023-02-24T14:04:30Z

so this patch:

diff --git a/DQM/Integration/python/clients/hlt_dqm_sourceclient-live_cfg.py b/DQM/Integration/python/clients/hlt_dqm_sourceclient-live_cfg.py
index 3e6b28ecb1f..cc04e270abf 100644
--- a/DQM/Integration/python/clients/hlt_dqm_sourceclient-live_cfg.py
+++ b/DQM/Integration/python/clients/hlt_dqm_sourceclient-live_cfg.py
@@ -46,8 +46,8 @@ process.ClusterShapeHitFilterESProducer = cms.ESProducer( "ClusterShapeHitFilter
 )
 #SiStrip Local Reco
 process.load("CalibTracker.SiStripCommon.TkDetMapESProducer_cfi")
-#SiPixelTemplate
-process.load("CalibTracker.SiPixelESProducers.SiPixelTemplateDBObjectESProducer_cfi")
+#Track refitters
+process.load("RecoTracker.TrackProducer.TrackRefitters_cff")
 
 #---- for P5 (online) DB access
 process.load("DQM.Integration.config.FrontierCondition_GT_cfi")
diff --git a/DQMOffline/Trigger/python/SiPixel_OfflineMonitoring_cff.py b/DQMOffline/Trigger/python/SiPixel_OfflineMonitoring_cff.py
index 37714b63571..94184f2792e 100644
--- a/DQMOffline/Trigger/python/SiPixel_OfflineMonitoring_cff.py
+++ b/DQMOffline/Trigger/python/SiPixel_OfflineMonitoring_cff.py
@@ -6,11 +6,12 @@ from RecoPixelVertexing.PixelLowPtUtilities.siPixelClusterShapeCache_cfi import
 from DQM.SiPixelMonitorTrack.RefitterForPixelDQM import *
 
 hltSiPixelClusterShapeCache = siPixelClusterShapeCache.clone(src = 'hltSiPixelClusters')
-hltrefittedForPixelDQM = refittedForPixelDQM.clone(src ='hltMergedTracks')
+hltrefittedForPixelDQM = refittedForPixelDQM.clone(src ='hltMergedTracks',
+                                                   TTRHBuilder = cms.string('WithTrackAngle')) # no templates at HLT
 
 sipixelMonitorHLTsequence = cms.Sequence(
     hltSiPixelClusterShapeCache
     + hltSiPixelPhase1ClustersAnalyzer
-    + refittedForPixelDQM
+    + hltrefittedForPixelDQM
     + hltSiPixelPhase1TrackClustersAnalyzer
 )    
diff --git a/RecoPixelVertexing/PixelLowPtUtilities/plugins/SiPixelClusterShapeCacheProducer.cc b/RecoPixelVertexing/PixelLowPtUtilities/plugins/SiPixelClusterShapeCacheProducer.cc
index dba9894b23c..6322f9cc868 100644
--- a/RecoPixelVertexing/PixelLowPtUtilities/plugins/SiPixelClusterShapeCacheProducer.cc
+++ b/RecoPixelVertexing/PixelLowPtUtilities/plugins/SiPixelClusterShapeCacheProducer.cc
@@ -61,6 +61,13 @@ void SiPixelClusterShapeCacheProducer::produce(edm::StreamID, edm::Event& iEvent
   edm::Handle<InputCollection> input;
   iEvent.getByToken(token_, input);
 
+  if (!input.isValid()) {
+    edm::LogError("siPixelClusterShapeCache") << "input pixel cluster collection is not valid!";
+    auto output = std::make_unique<SiPixelClusterShapeCache>();
+    iEvent.put(std::move(output));
+    return;
+  }
+
   const auto& geom = &iSetup.getData(geomToken_);
 
   auto output = std::make_unique<SiPixelClusterShapeCache>(input);

technically makes the test scram b runtests_TestDQMOnlineClient-hlt_dqm_sourceclient use-ibeos run fine, even though it's not actually testing anything (see #40875 (comment)).
I suppose that this would also fix the problem with 140.54, though we probably want to run some sequence that doesn't run the HLT monitoring instead, since there's no re-HLT in it.

cmsbuild · 2023-02-24T14:12:23Z

Pull request #40875 was updated. @emanueleusai, @cmsbuild, @syuvivida, @rvenditti, @micsucmed, @pmandrik can you please check and sign again.

…lable

cmsbuild · 2023-02-28T08:34:31Z

Pull request #40875 was updated. @micsucmed, @emanueleusai, @clacaputo, @cmsbuild, @syuvivida, @pmandrik, @mandrenguyen, @rvenditti can you please check and sign again.

mmusich · 2023-02-28T08:35:03Z

@cmsbuild, please test

cmsbuild · 2023-02-28T11:42:24Z

-1

Failed Tests: RelVals-INPUT
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cc786b/30940/summary.html
COMMIT: d943f6f
CMSSW: CMSSW_13_0_X_2023-02-27-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/40875/30940/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals-INPUT

20834.020834.0_TTbar_14TeV+2026D88/step2_TTbar_14TeV+2026D88.log
20834.2120834.21_TTbar_14TeV+2026D88_ProdLike/step2_TTbar_14TeV+2026D88_ProdLike.log
20834.10320834.103_TTbar_14TeV+2026D88Aging3000/step2_TTbar_14TeV+2026D88Aging3000.log

Expand to see more relval errors ...

Comparison Summary

Summary:

You potentially added 7 lines to the logs
Reco comparison results: 14 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3647977
DQMHistoTests: Total failures: 7043
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3640912
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 23549.645 KiB( 49 files compared)
DQMHistoSizes: changed ( 140.063,... ): 3364.235 KiB HLT/Pixel
Checked 217 log files, 167 edm output root files, 50 DQM output files
TriggerResults: no differences found

arossi83 · 2023-02-28T13:44:29Z

Test failing because of #40889. Backport of #40873 with the addition of #40891 (see last commit here) (both PR already merged in master)

mmusich · 2023-02-28T16:07:10Z

@emanueleusai @mandrenguyen @perrotta w.r.t last signature this PR just removed cms type specification in one configuration file. Can you consider this for inclusion in 13_0_0?

mandrenguyen · 2023-02-28T16:10:15Z

+reconstruction
Disregarding relval input failure

emanueleusai · 2023-03-01T02:12:38Z

+1

differences in DQM understood (see master PR)

cmsbuild · 2023-03-01T02:13:04Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_13_0_X IBs (but tests are reportedly failing) and once validation in the development release cycle CMSSW_13_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-03-01T08:43:55Z

+1

perrotta · 2023-03-01T08:44:27Z

merge

Alessandro Rossi added 2 commits February 22, 2023 17:46

Introduce track refitting in HLT DQM for Pixel Tracks quantities

4c89858

Add Pixel Template to HLT Online client

5f46737

cmsbuild added this to the CMSSW_13_0_X milestone Feb 24, 2023

cmsbuild added dqm-pending pending-signatures tests-pending orp-pending trk labels Feb 24, 2023

cmsbuild added tests-started and removed tests-pending labels Feb 24, 2023

cmsbuild added the urgent label Feb 24, 2023

mmusich reviewed Feb 24, 2023

View reviewed changes

cmsbuild added tests-rejected and removed tests-started labels Feb 24, 2023

Fix a typo on SiPixel_OfflineMonitor sequence

0e9baea

cmsbuild added tests-pending and removed tests-rejected labels Feb 24, 2023

mmusich mentioned this pull request Feb 24, 2023

Fix Tracks related Pixel quantities at HLT DQM #40873

Merged

Modification to protect the process when some collection are not avai…

014e922

…lable

cmsbuild added reconstruction-pending tracking labels Feb 25, 2023

cmsbuild added reconstruction-pending dqm-pending pending-signatures tests-pending and removed fully-signed tests-approved labels Feb 28, 2023

cmsbuild added tests-started and removed tests-pending labels Feb 28, 2023

cmsbuild added tests-rejected and removed tests-started labels Feb 28, 2023

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Feb 28, 2023

cmsbuild added dqm-approved fully-signed and removed dqm-pending pending-signatures labels Mar 1, 2023

cmsbuild added orp-approved and removed orp-pending labels Mar 1, 2023

cmsbuild merged commit 9c46630 into cms-sw:CMSSW_13_0_X Mar 1, 2023

cmsbuild mentioned this pull request Mar 2, 2023

[BuildRules] Enable alpaka ROCm backend [13.0.x] cms-sw/cmsdist#8347

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix Tracks related Pixel quantities at HLT DQM (backport) #40875

Fix Tracks related Pixel quantities at HLT DQM (backport) #40875

arossi83 commented Feb 24, 2023

cmsbuild commented Feb 24, 2023 •

edited

Loading

mmusich commented Feb 24, 2023

mmusich commented Feb 24, 2023

mmusich commented Feb 24, 2023

mmusich commented Feb 24, 2023 •

edited

Loading

missirol commented Feb 24, 2023

mmusich Feb 24, 2023

arossi83 Feb 24, 2023

cmsbuild commented Feb 24, 2023

mmusich commented Feb 24, 2023

cmsbuild commented Feb 24, 2023

cmsbuild commented Feb 28, 2023

mmusich commented Feb 28, 2023

cmsbuild commented Feb 28, 2023

arossi83 commented Feb 28, 2023

mmusich commented Feb 28, 2023

mandrenguyen commented Feb 28, 2023

emanueleusai commented Mar 1, 2023

cmsbuild commented Mar 1, 2023

perrotta commented Mar 1, 2023

perrotta commented Mar 1, 2023

Fix Tracks related Pixel quantities at HLT DQM (backport) #40875

Fix Tracks related Pixel quantities at HLT DQM (backport) #40875

Conversation

arossi83 commented Feb 24, 2023

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

cmsbuild commented Feb 24, 2023 • edited Loading

mmusich commented Feb 24, 2023

mmusich commented Feb 24, 2023

mmusich commented Feb 24, 2023

mmusich commented Feb 24, 2023 • edited Loading

missirol commented Feb 24, 2023

mmusich Feb 24, 2023

Choose a reason for hiding this comment

arossi83 Feb 24, 2023

Choose a reason for hiding this comment

cmsbuild commented Feb 24, 2023

Unit Tests

RelVals-INPUT

Comparison Summary

mmusich commented Feb 24, 2023

cmsbuild commented Feb 24, 2023

cmsbuild commented Feb 28, 2023

mmusich commented Feb 28, 2023

cmsbuild commented Feb 28, 2023

RelVals-INPUT

Comparison Summary

arossi83 commented Feb 28, 2023

mmusich commented Feb 28, 2023

mandrenguyen commented Feb 28, 2023

emanueleusai commented Mar 1, 2023

cmsbuild commented Mar 1, 2023

perrotta commented Mar 1, 2023

perrotta commented Mar 1, 2023

cmsbuild commented Feb 24, 2023 •

edited

Loading

mmusich commented Feb 24, 2023 •

edited

Loading