fix GEM inputs for `L1REPACK:FullMC` #40461

missirol · 2023-01-09T17:54:31Z

PR description:

It looks like L1REPACK:FullMC does not work as expected on EDM inputs without the DIGI data tier, as [1] returns [2].

(This would suggest that, up to now, users were simply reading the GEM digis from the EDM inputs when running L1REPACK:FullMC.)

This PR is a tentative fix (I'm not a domain expert). It is based on what is currently done for L1REPACK:Full (which is for real data, as opposed to L1REPACK:FullMC which is for MC), e.g.

cmssw/Configuration/StandardSequences/python/SimL1EmulatorRepack_Full_cff.py

Lines 168 to 170 in a8e4cca

    
           _SimL1EmulatorTaskWithGEM = SimL1EmulatorTask.copy() 
        
           _SimL1EmulatorTaskWithGEM.add(unpackGEM) 
        
           (stage2L1Trigger & run3_GEM).toReplaceWith(SimL1EmulatorTask, _SimL1EmulatorTaskWithGEM)

(L1REPACK:Full was fixed in #36133, based on #36133 (comment))

[1]

cmsDriver.py step2 -s L1REPACK:FullMC --conditions auto:phase1_2022_realistic_postEE --era Run3 \
  --datatier RAW --eventcontent RAW --geometry DB:Extended --nThreads 1 --nStreams 0 -n 10 \
  --filein /store/mc/Run3Summer22EEDR/QCD_Pt-120To170_TuneCP5_13p6TeV-pythia8/GEN-SIM-RAW/Poisson70KeepRAW_124X_mcRun3_2022_realistic_postEE_v1-v1/80000/55194757-7f7a-4fac-83b2-fc6e0b5fe7f5.root

[2]

----- Begin Fatal Exception 09-Jan-2023 18:08:53 CET-----------------------
An exception of category 'ProductNotFound' occurred while
   [0] Processing  Event run: 1 lumi: 104 event: 103002 stream: 0
   [1] Running path 'RAWoutput_step'
   [2] Prefetching for module PoolOutputModule/'RAWoutput'
   [3] Prefetching for module RawDataCollectorByLabel/'rawDataCollector'
   [4] Prefetching for module L1TDigiToRaw/'packGmtStage2'
   [5] Prefetching for module L1TMuonOverlapPhase1TrackProducer/'simOmtfDigis'
   [6] Prefetching for module CSCTriggerPrimitivesProducer/'simCscTriggerPrimitiveDigis'
   [7] Prefetching for module GEMPadDigiClusterProducer/'simMuonGEMPadDigiClusters'
   [8] Calling method for module GEMPadDigiProducer/'simMuonGEMPadDigis'
Exception Message:
Principal::getByToken: Found zero products matching all criteria
Looking for type: MuonDigiCollection<GEMDetId,GEMDigi>
Looking for module label: simMuonGEMDigis
Looking for productInstanceName: 

   Additional Info:
      [a] If you wish to continue processing events after a ProductNotFound exception,
add "SkipEvent = cms.untracked.vstring('ProductNotFound')" to the "options" PSet in the configuration.

----- End Fatal Exception -------------------------------------------------

PR validation:

Nothing beyond the use case described in the PR description.

If this PR is a backport, please specify the original PR and why you need to backport that PR. If this PR will be backported, please specify to which release cycle the backport is meant for:

~~Plan to backport to (at least) 12_6_X to aid HLT studies.~~

CMSSW_12_4_X
CMSSW_12_5_X
CMSSW_12_6_X

cmsbuild · 2023-01-09T18:00:29Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40461/33624

This PR adds an extra 16KB to repository

cmsbuild · 2023-01-09T18:00:55Z

A new Pull Request was created by @missirol (Marino Missiroli) for master.

It involves the following packages:

Configuration/StandardSequences (operations)

@cmsbuild, @perrotta, @rappoccio, @fabiocos, @davidlange6 can you please review it and eventually sign? Thanks.
@fabiocos, @VourMa, @makortel, @felicepantaleo, @GiacomoSguazzoni, @JanFSchulte, @rovere, @VinInn, @Martin-Grunewald, @lecriste, @mmusich, @sameasy, @mtosi, @dgulhan, @slomeo this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

missirol · 2023-01-09T18:01:19Z

assign l1

cmsbuild · 2023-01-09T18:01:33Z

New categories assigned: l1

@epalencia,@rekovic,@cecilecaillol you have been requested to review this Pull request/Issue and eventually sign? Thanks

missirol · 2023-01-09T18:02:58Z

please test

@cms-sw/l1-l2 , is there any wf testing L1REPACK:FullMC in CMSSW ?

cmsbuild · 2023-01-09T21:49:35Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1dd99c/29854/summary.html
COMMIT: e290d42
CMSSW: CMSSW_13_0_X_2023-01-09-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40461/29854/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 12 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3555538
DQMHistoTests: Total failures: 160
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3555356
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 211 log files, 162 edm output root files, 49 DQM output files
TriggerResults: no differences found

missirol · 2023-01-10T12:45:22Z

@cms-sw/l1-l2

Could you please have a look, and check if this fix is correct? If so, it would be useful to have this in 13_0_0_pre3 (deadline: today), and it will have to be backported. Thanks!

cecilecaillol · 2023-01-10T13:50:11Z

+l1

missirol · 2023-01-10T14:00:51Z

type bugfix

perrotta · 2023-01-10T14:33:24Z

Configuration/StandardSequences/python/SimL1EmulatorRepack_FullMC_cff.py

@@ -60,6 +67,8 @@ def _print(ignored):
 # CSC TPs
 simCscTriggerPrimitiveDigis.CSCComparatorDigiProducer = 'unpackCSC:MuonCSCComparatorDigi'
 simCscTriggerPrimitiveDigis.CSCWireDigiProducer       = 'unpackCSC:MuonCSCWireDigi'
+# GEM
+run3_GEM.toModify(simMuonGEMPadDigis, InputCollection = 'unpackGEM')


This is the same as what currently in SimL1EmulatorRepack_Full_cff.py, therefore I imagine it may work and my question is probably silly...

But what happens if run3_GEM and not stage2L1Trigger? 'unpackGEM' is feed as InputCollection to simMuonGEMPadDigis [1], but the SimL1EmulatorTask is not updates with the unpackGEM module in it [2]

Well, I can only guess as a non-L1T expert.

This file only really works for stage2L1Trigger, according to the comments up top ("only supports Stage 2 eras for now."), modulo the fact that the comments are mistakenly swapped (the first comment, "use a legacy version", should show for ~stage2L1Trigger, not for stage2L1Trigger)

In practise, I think stage2L1Trigger applies to nearly anything after 2015, so "run3_GEM and not stage2L1Trigger" should never happen (in principle..).

Let me fix the comments, since I would like to backport this..

Thank you Marino, this is also my understanding.
However, even if so, the clean way to implement I imagine it should be

Suggested change

run3_GEM.toModify(simMuonGEMPadDigis, InputCollection = 'unpackGEM')

(stage2L1Trigger & run3_GEM).toModify(simMuonGEMPadDigis, InputCollection = 'unpackGEM')

Following what I see elsewhere, e.g.

cmssw/L1Trigger/L1TMuon/python/simDigis_cff.py

Line 119 in a45ef05

(stage2L1Trigger & run3_GEM).toReplaceWith( SimL1TMuonTask, cms.Task(simMuonGEMPadTask,_run3_SimL1TMuonTask) )

I can change run3_GEM to (stage2L1Trigger & run3_GEM) also above (and in "Full").

However, even if so, the clean way to implement I imagine it should be

I cross-posted. Yes, will do (soon).

cmsbuild · 2023-01-10T16:51:22Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40461/33642

This PR adds an extra 16KB to repository

cmsbuild · 2023-01-10T16:56:39Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-40461/33643

This PR adds an extra 16KB to repository

cmsbuild · 2023-01-10T16:57:05Z

Pull request #40461 was updated. @perrotta, @rappoccio, @epalencia, @cecilecaillol, @cmsbuild, @rekovic, @fabiocos, @davidlange6 can you please check and sign again.

perrotta · 2023-01-10T17:11:03Z

please test

cmsbuild · 2023-01-10T21:00:44Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-1dd99c/29892/summary.html
COMMIT: 633c942
CMSSW: CMSSW_13_0_X_2023-01-10-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/40461/29892/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 35 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3555538
DQMHistoTests: Total failures: 1194
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3554322
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 211 log files, 162 edm output root files, 49 DQM output files
TriggerResults: no differences found

missirol · 2023-01-11T08:04:41Z

@cms-sw/l1-l2, could you please check again, and re-sign? Thanks!

The diff of this PR since your previous signature is below:
https://github.com/cms-sw/cmssw/compare/e290d429c270dae5dc61c0c2db06d311f3c18fb5..633c942bdab6a2318b7fda5bd266180520c0057b

cecilecaillol · 2023-01-11T09:19:10Z

+l1

perrotta · 2023-01-11T09:58:35Z

+1

cmsbuild · 2023-01-11T09:59:00Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will be automatically merged.

cmsbuild added this to the CMSSW_13_0_X milestone Jan 9, 2023

cmsbuild added code-checks-pending operations-pending orp-pending pending-signatures tests-pending labels Jan 9, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 9, 2023

cmsbuild added the l1-pending label Jan 9, 2023

cmsbuild added tests-started tests-pending and removed tests-pending tests-started labels Jan 9, 2023

cmsbuild added tests-approved and removed tests-started labels Jan 9, 2023

cmsbuild added l1-approved and removed l1-pending labels Jan 10, 2023

cmsbuild added the bug-fix label Jan 10, 2023

perrotta reviewed Jan 10, 2023

View reviewed changes

fix GEM inputs for L1REPACK:FullMC

633c942

missirol force-pushed the devel_fixGEMRawToDigiInL1TFullMC branch from 7cb44b3 to 633c942 Compare January 10, 2023 16:51

cmsbuild added code-checks-approved and removed code-checks-pending labels Jan 10, 2023

cmsbuild added tests-started and removed tests-pending labels Jan 10, 2023

This was referenced Jan 10, 2023

fix GEM inputs for L1REPACK:FullMC [12_6_X] #40468

Merged

fix GEM inputs for L1REPACK:FullMC [12_5_X] #40469

Merged

fix GEM inputs for L1REPACK:FullMC [12_4_X] #40470

Merged

cmsbuild added tests-approved and removed tests-started labels Jan 10, 2023

cmsbuild added l1-approved and removed l1-pending labels Jan 11, 2023

cmsbuild added fully-signed operations-approved orp-approved and removed operations-pending pending-signatures orp-pending labels Jan 11, 2023

cmsbuild merged commit fe42e8d into cms-sw:master Jan 11, 2023

missirol deleted the devel_fixGEMRawToDigiInL1TFullMC branch January 11, 2023 10:02

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix GEM inputs for `L1REPACK:FullMC` #40461

fix GEM inputs for `L1REPACK:FullMC` #40461

missirol commented Jan 9, 2023 •

edited

Loading

cmsbuild commented Jan 9, 2023

cmsbuild commented Jan 9, 2023

missirol commented Jan 9, 2023

cmsbuild commented Jan 9, 2023

missirol commented Jan 9, 2023 •

edited

Loading

cmsbuild commented Jan 9, 2023

missirol commented Jan 10, 2023

cecilecaillol commented Jan 10, 2023

missirol commented Jan 10, 2023

perrotta Jan 10, 2023

missirol Jan 10, 2023

perrotta Jan 10, 2023

missirol Jan 10, 2023

missirol Jan 10, 2023

cmsbuild commented Jan 10, 2023

cmsbuild commented Jan 10, 2023

cmsbuild commented Jan 10, 2023

perrotta commented Jan 10, 2023

cmsbuild commented Jan 10, 2023

missirol commented Jan 11, 2023

cecilecaillol commented Jan 11, 2023

perrotta commented Jan 11, 2023

cmsbuild commented Jan 11, 2023

	_SimL1EmulatorTaskWithGEM = SimL1EmulatorTask.copy()
	_SimL1EmulatorTaskWithGEM.add(unpackGEM)
	(stage2L1Trigger & run3_GEM).toReplaceWith(SimL1EmulatorTask, _SimL1EmulatorTaskWithGEM)

	run3_GEM.toModify(simMuonGEMPadDigis, InputCollection = 'unpackGEM')
	(stage2L1Trigger & run3_GEM).toModify(simMuonGEMPadDigis, InputCollection = 'unpackGEM')

fix GEM inputs for L1REPACK:FullMC #40461

fix GEM inputs for L1REPACK:FullMC #40461

Conversation

missirol commented Jan 9, 2023 • edited Loading

PR description:

PR validation:

If this PR is a backport, please specify the original PR and why you need to backport that PR. If this PR will be backported, please specify to which release cycle the backport is meant for:

cmsbuild commented Jan 9, 2023

cmsbuild commented Jan 9, 2023

missirol commented Jan 9, 2023

cmsbuild commented Jan 9, 2023

missirol commented Jan 9, 2023 • edited Loading

cmsbuild commented Jan 9, 2023

Comparison Summary

missirol commented Jan 10, 2023

cecilecaillol commented Jan 10, 2023

missirol commented Jan 10, 2023

perrotta Jan 10, 2023

Choose a reason for hiding this comment

missirol Jan 10, 2023

Choose a reason for hiding this comment

perrotta Jan 10, 2023

Choose a reason for hiding this comment

missirol Jan 10, 2023

Choose a reason for hiding this comment

missirol Jan 10, 2023

Choose a reason for hiding this comment

cmsbuild commented Jan 10, 2023

cmsbuild commented Jan 10, 2023

cmsbuild commented Jan 10, 2023

perrotta commented Jan 10, 2023

cmsbuild commented Jan 10, 2023

Comparison Summary

missirol commented Jan 11, 2023

cecilecaillol commented Jan 11, 2023

perrotta commented Jan 11, 2023

cmsbuild commented Jan 11, 2023

fix GEM inputs for `L1REPACK:FullMC` #40461

fix GEM inputs for `L1REPACK:FullMC` #40461

missirol commented Jan 9, 2023 •

edited

Loading

missirol commented Jan 9, 2023 •

edited

Loading