Add CSC/DT rechits to AOD[backport] #34066

kakwok · 2021-06-09T16:49:39Z

PR description:

This PR adds CSC/DT rechits to the AOD datatier, which are the essential inputs for searches of LLP decaying in the muon system.

PR validation:

Currently underway.

if this PR is a backport please specify the original PR and why you need to backport that PR:

Before submitting your pull requests, make sure you followed this checklist:

verify that the PR is really intended for the chosen branch
verify that changes follow CMS Naming, Coding, And Style Rules
verify that the PR passes the basic test procedure suggested in the CMSSW PR instructions

cmsbuild · 2021-06-09T16:50:00Z

A new Pull Request was created by @kakwok for CMSSW_10_6_X.

It involves the following packages:

RecoLocalMuon/Configuration

@perrotta, @jpata, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@bellan, @abbiendi, @jhgoh, @battibass this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

Backported from Add CSC/DT rechits to AOD #34072

mariadalfonso · 2021-06-09T17:49:09Z

this PR is intended for the B-Parking
#33917

slava77 · 2021-06-09T18:08:37Z

is there a master branch PR?

slava77 · 2021-06-09T18:11:12Z

RecoLocalMuon/Configuration/python/RecoLocalMuon_EventContent_cff.py

-    outputCommands = cms.untracked.vstring(
+    outputCommands = cms.untracked.vstring('keep *_dt1DRecHits_*_*', 
        'keep *_dt4DSegments_*_*', 
        'keep *_dt4DCosmicSegments_*_*',
        'keep *_cscSegments_*_*', 
+        'keep *_csc2DRecHits_*_*', 


for 10_6_X (this PR) this part should be done below for bParking modifier

_updateOutput(bParking, RecoLocalMuonAOD, ['keep *_dt1DRecHits_*_*', 'keep *_csc2DRecHits_*_*'])

BTW, was it confirmed that dt1DRecHits work OK, or should this be dt1DCosmicRecHits instead?

We checked with the analysis team who uses the DT rechits, we do not need dt1DCosmicRecHits

@slava77 bparking modifier added in 7632eaf

slava77 · 2021-06-09T19:29:09Z

test parameters:

workflows = 136.898,1304.181

slava77 · 2021-06-09T19:29:43Z

@cmsbuild please test

slava77 · 2021-06-09T19:57:33Z

looking at somewhat random aod files I get a total of about 1%:

ZEE + 2018 PU from UL RunIISummer20ULPrePremix aod 100 events: dt1DRecHits add 0.13%, csc2DRecHits add 1.0%; the total increase is about 1.2%
136.898 input (Run2018B ParkingBPH5 run 317661 LS 393) aod 100 events: dt1DRecHits add 0.88%, csc2DRecHits add 0.79%; the total increase is about 1.7%
- I think that this is more representative of the total

Considering that this will stay mostly on tape, I guess that an increase in AOD size by 1.7% is acceptable.

kakwok · 2021-06-09T20:50:57Z

Adding output from 100 events of workflow 136.898

File step3_AOD.root Events 100
Branch Name | Average Uncompressed Size (Bytes/Event) | Average Compressed Size (Bytes/Event)
CSCDetIdCSCSegmentsOwnedRangeMap_cscSegments__reRECO. 9680.49 5144.9
CSCDetIdCSCRecHit2DsOwnedRangeMap_csc2DRecHits__reRECO. 13457 5097.81

The two collections add ~9 KB/event out of 560 KB/event (~1.9%)

cmsbuild · 2021-06-09T21:37:33Z

Pull request #34066 was updated. @perrotta, @jpata, @cmsbuild, @slava77 can you please check and sign again.

kakwok · 2021-06-09T22:22:37Z

is there a master branch PR?

I've created the PR for master branch #34072

slava77 · 2021-06-10T00:14:29Z

@cmsbuild please test

cmsbuild · 2021-06-10T00:49:44Z

-1

Failed Tests: RelVals RelVals-INPUT AddOn
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19e50b/15840/summary.html
COMMIT: 490faa3
CMSSW: CMSSW_10_6_X_2021-06-09-2300/slc7_amd64_gcc700
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34066/15840/install.sh to create a dev area with all the needed externals and cmssw changes.

RelVals

4.534.53_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT/step2_RunPhoton2012B+RunPhoton2012B+HLTD+RECODR1reHLT+HARVESTDR1reHLT.log
5.15.1_TTbar+TTbarFS+HARVESTFS/step1_TTbar+TTbarFS+HARVESTFS.log
4.224.22_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC/step2_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC.log

Expand to see more relval errors ...

RelVals-INPUT

4.174.17_RunMinBias2011A+RunMinBias2011A+HLTD+RECODR1reHLT+HARVESTDR1reHLT+SKIMDreHLT/step2_RunMinBias2011A+RunMinBias2011A+HLTD+RECODR1reHLT+HARVESTDR1reHLT+SKIMDreHLT.log
4.224.22_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC/step2_RunCosmics2011A+RunCosmics2011A+RECOCOSD+ALCACOSD+SKIMCOSD+HARVESTDC.log
4.264.26_ZMuSkim2011A+ZMuSkim2011A+HLTDSKIM+RECODR1reHLT+HARVESTDR1reHLT/step2_ZMuSkim2011A+ZMuSkim2011A+HLTDSKIM+RECODR1reHLT+HARVESTDR1reHLT.log

Expand to see more relval errors ...

AddOn Tests

fastsimcmsDriver.py TTbar_8TeV_TuneCUETP8M1_cfi --conditions auto:run1_mc --fast -n 100 --eventcontent AODSIM,DQM --relval 100000,1000 -s GEN,SIM,RECOBEFMIX,DIGI:pdigi_valid,L1,DIGI2RAW,L1Reco,RECO,EI,VALIDATION --customise=HLTrigger/Configuration/CustomConfigs.L1THLT --datatier GEN-SIM-DIGI-RECO,DQMIO --beamspot Realistic8TeVCollision : FAILED - time: date Thu Jun 10 02:45:04 2021-date Thu Jun 10 02:45:02 2021 s - exit: 256
hlt_data_Fake2cmsDriver.py RelVal -s L1REPACK:Full --data --scenario=pp -n 10 --conditions auto:run2_hlt_Fake2 --relval 9000,50 --datatier "RAW" --eventcontent RAW --customise=HLTrigger/Configuration/CustomConfigs.L1T --era Run2_2016 --fileout file:RelVal_Raw_Fake2_DATA.root --filein /store/data/Run2016B/JetHT/RAW/v1/000/272/762/00000/C666CDE2-E013-E611-B15A-02163E011DBE.root : FAILED - time: date Thu Jun 10 02:45:21 2021-date Thu Jun 10 02:45:10 2021 s - exit: 256
hlt_mc_Fake2

----- Begin Fatal Exception 10-Jun-2021 02:45:49 CEST-----------------------
An exception of category 'FileOpenError' occurred while
   [0] Constructing the EventProcessor
   [1] Constructing input source of type PoolSource
   [2] Calling RootFileSequenceBase::initTheFile()
   [3] Calling StorageFactory::open()
   [4] Calling File::sysopen()
Exception Message:
Failed to open the file 'RelVal_Raw_Fake2_MC.root'
   Additional Info:
      [a] Input file file:RelVal_Raw_Fake2_MC.root could not be opened.
      [b] open() failed with system error 'No such file or directory' (error code 2)
----- End Fatal Exception -------------------------------------------------

Expand to see more addon errors ...

slava77 · 2021-06-10T01:05:42Z

RecoLocalMuon/Configuration/python/RecoLocalMuon_EventContent_cff.py

 _outputs = [RecoLocalMuonFEVT, RecoLocalMuonRECO, RecoLocalMuonAOD]
+_updateOutput(bParking, RecoLocalMuonAOD, ['keep *_dt1DRecHits_*_*', 'keep *_csc2DRecHits_*_*'])


Suggested change

_updateOutput(bParking, RecoLocalMuonAOD, ['keep *_dt1DRecHits_*_*', 'keep *_csc2DRecHits_*_*'])

_updateOutput(bParking, [RecoLocalMuonAOD], ['keep *_dt1DRecHits_*_*', 'keep *_csc2DRecHits_*_*'])

I guess this is going to fix the crashes.
Please double-check in a local test (parsing a config is enough)

perrotta · 2021-06-10T05:24:48Z

backport of #34072

cmsbuild · 2021-06-10T06:13:08Z

Pull request #34066 was updated. @perrotta, @jpata, @cmsbuild, @slava77 can you please check and sign again.

slava77 · 2021-06-10T06:14:56Z

RecoLocalMuon/Configuration/python/RecoLocalMuon_EventContent_cff.py

@@ -22,7 +22,7 @@
 )
 # AOD content
 RecoLocalMuonAOD = cms.PSet(
-    outputCommands = cms.untracked.vstring(
+    outputCommands = cms.untracked.vstring( 
        'keep *_dt4DSegments_*_*', 
        'keep *_dt4DCosmicSegments_*_*',
        'keep *_cscSegments_*_*', 


BTW, both DT segments and CSC segments are already available in AOD.
It would be nice to see some slides/material showing that segments do not work.

@slava77 For the record, we checked in simulation that rechits provides much better signal-background discrimination. A more detailed discussion is also documented in the AN of EXO-20-015, section 4.1

slava77 · 2021-06-10T06:15:36Z

@cmsbuild please test

cmsbuild · 2021-06-10T11:01:41Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-19e50b/15844/summary.html
COMMIT: 571953e
CMSSW: CMSSW_10_6_X_2021-06-09-2300/slc7_amd64_gcc700
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/34066/15844/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 3215678
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3215343
DQMHistoTests: Total skipped: 334
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 143 log files, 29 edm output root files, 35 DQM output files
TriggerResults: no differences found

slava77 · 2021-06-11T00:20:16Z

+reconstruction

for #34066 571953e

code changes are in line with the PR description and the follow up review
- this PR is functionally consistent with the master version Add CSC/DT rechits to AOD #34072 (merged) : the python edits are not identical due to differences/refactoring of the configuration made in 11_1_X in Remove drop and update the event content formats #29158
- the event content update is restricted to the bParking era/configuration
  - @kakwok please update the PR description to clarify this point
jenkins tests pass and comparisons with the baseline show no differences
event size analysis was done in Add CSC/DT rechits to AOD[backport] #34066 (comment) and Add CSC/DT rechits to AOD[backport] #34066 (comment) : about 1.7% increase in the AOD event size is expected

cmsbuild · 2021-06-11T00:20:39Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_10_6_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_0_X is complete. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2021-06-11T06:14:27Z

+1

Add CSC/DT rechits to AOD

e462180

cmsbuild added this to the CMSSW_10_6_X milestone Jun 9, 2021

cmsbuild added orp-pending pending-signatures reconstruction-pending tests-pending labels Jun 9, 2021

slava77 reviewed Jun 9, 2021

View reviewed changes

slava77 mentioned this pull request Jun 9, 2021

features for bParking UL rereco and miniAOD in 10_6_X (10_6_26?) #33917

Closed

14 tasks

cmsbuild added tests-started and removed tests-pending labels Jun 9, 2021

Martin added 2 commits June 9, 2021 16:05

update output for bparking era only

7632eaf

update bparking

490faa3

cmsbuild added tests-pending and removed tests-started labels Jun 9, 2021

kakwok changed the title ~~Add CSC/DT rechits to AOD~~ Add CSC/DT rechits to AOD[backport] Jun 9, 2021

kakwok mentioned this pull request Jun 9, 2021

Add CSC/DT rechits to AOD #34072

Merged

cmsbuild added tests-started and removed tests-pending labels Jun 10, 2021

cmsbuild removed the tests-started label Jun 10, 2021

cmsbuild added the tests-rejected label Jun 10, 2021

slava77 reviewed Jun 10, 2021

View reviewed changes

cmsbuild added the backport label Jun 10, 2021

fix syntax

571953e

cmsbuild added tests-pending and removed tests-rejected labels Jun 10, 2021

slava77 reviewed Jun 10, 2021

View reviewed changes

cmsbuild added tests-started and removed tests-pending labels Jun 10, 2021

cmsbuild added tests-approved and removed tests-started labels Jun 10, 2021

cmsbuild added fully-signed reconstruction-approved backport-ok and removed pending-signatures reconstruction-pending backport labels Jun 11, 2021

cmsbuild added orp-approved and removed orp-pending labels Jun 11, 2021

cmsbuild merged commit f253ec2 into cms-sw:CMSSW_10_6_X Jun 11, 2021

kakwok mentioned this pull request Dec 6, 2022

add DT and CSC rechits to AOD content #40251

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add CSC/DT rechits to AOD[backport] #34066

Add CSC/DT rechits to AOD[backport] #34066

kakwok commented Jun 9, 2021

cmsbuild commented Jun 9, 2021 •

edited

Loading

mariadalfonso commented Jun 9, 2021

slava77 commented Jun 9, 2021

slava77 Jun 9, 2021

slava77 Jun 9, 2021

kakwok Jun 9, 2021

kakwok Jun 9, 2021 •

edited

Loading

slava77 commented Jun 9, 2021

slava77 commented Jun 9, 2021

slava77 commented Jun 9, 2021

kakwok commented Jun 9, 2021

cmsbuild commented Jun 9, 2021

kakwok commented Jun 9, 2021

slava77 commented Jun 10, 2021

cmsbuild commented Jun 10, 2021

slava77 Jun 10, 2021

perrotta commented Jun 10, 2021

cmsbuild commented Jun 10, 2021

slava77 Jun 10, 2021

kakwok Jun 15, 2021

slava77 commented Jun 10, 2021

cmsbuild commented Jun 10, 2021

slava77 commented Jun 11, 2021

cmsbuild commented Jun 11, 2021

qliphy commented Jun 11, 2021

		_outputs = [RecoLocalMuonFEVT, RecoLocalMuonRECO, RecoLocalMuonAOD]
		_updateOutput(bParking, RecoLocalMuonAOD, ['keep _dt1DRecHits__', 'keep _csc2DRecHits__'])

	_updateOutput(bParking, RecoLocalMuonAOD, ['keep _dt1DRecHits__', 'keep _csc2DRecHits__'])
	_updateOutput(bParking, [RecoLocalMuonAOD], ['keep _dt1DRecHits__', 'keep _csc2DRecHits__'])

Add CSC/DT rechits to AOD[backport] #34066

Add CSC/DT rechits to AOD[backport] #34066

Conversation

kakwok commented Jun 9, 2021

PR description:

PR validation:

if this PR is a backport please specify the original PR and why you need to backport that PR:

cmsbuild commented Jun 9, 2021 • edited Loading

mariadalfonso commented Jun 9, 2021

slava77 commented Jun 9, 2021

slava77 Jun 9, 2021

Choose a reason for hiding this comment

slava77 Jun 9, 2021

Choose a reason for hiding this comment

kakwok Jun 9, 2021

Choose a reason for hiding this comment

kakwok Jun 9, 2021 • edited Loading

Choose a reason for hiding this comment

slava77 commented Jun 9, 2021

slava77 commented Jun 9, 2021

slava77 commented Jun 9, 2021

kakwok commented Jun 9, 2021

cmsbuild commented Jun 9, 2021

kakwok commented Jun 9, 2021

slava77 commented Jun 10, 2021

cmsbuild commented Jun 10, 2021

RelVals

RelVals-INPUT

AddOn Tests

slava77 Jun 10, 2021

Choose a reason for hiding this comment

perrotta commented Jun 10, 2021

cmsbuild commented Jun 10, 2021

slava77 Jun 10, 2021

Choose a reason for hiding this comment

kakwok Jun 15, 2021

Choose a reason for hiding this comment

slava77 commented Jun 10, 2021

cmsbuild commented Jun 10, 2021

Comparison Summary

slava77 commented Jun 11, 2021

cmsbuild commented Jun 11, 2021

qliphy commented Jun 11, 2021

cmsbuild commented Jun 9, 2021 •

edited

Loading

kakwok Jun 9, 2021 •

edited

Loading