Update event content structure in RecoEcal and RecoVertex #29299

jeongeun · 2020-03-25T12:30:43Z

PR description:

Update event content definitions to explicitly have RECO to be a superset of AOD and for FEVT to be a superset of RECO. Clean up 'drop' not really used in RecoEcalRECO.
(The previous tasks are PR#29025, PR#29158, and PR#29225.)
3 files changed :
+RecoEcal_EventContent_cff.py
+RecoVertex_EventContent_cff.py
+BeamSpot_EventContent_cff.py

PR validation:

Event Content comparison check was also done and there is no change with these updates.
Tested in CMSSW_11_1_X_2020_03-23-1100, the basic test all passed in the CMSSW PR instructions.

cmsbuild · 2020-03-25T12:31:08Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-03-25T12:36:54Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-29299/14363

This PR adds an extra 16KB to repository

cmsbuild · 2020-03-25T12:37:14Z

A new Pull Request was created by @jeongeun (JeongEun Lee) for master.

It involves the following packages:

RecoEcal/Configuration
RecoVertex/BeamSpotProducer
RecoVertex/Configuration

@perrotta, @slava77, @christopheralanwest, @tocheng, @cmsbuild, @tlampen, @pohsun can you please review it and eventually sign? Thanks.
@varuns23, @Sam-Harper, @makortel, @jainshilpi, @afiqaize, @GiacomoSguazzoni, @tocheng, @lgray, @sobhatta, @rovere, @VinInn, @ebrondol, @argiro, @mmusich, @dgulhan this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

perrotta · 2020-03-25T13:23:04Z

please test

cmsbuild · 2020-03-25T13:23:26Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5375/console Started: 2020/03/25 14:30

cmsbuild · 2020-03-25T14:41:45Z

+1
Tested at: 14ed839
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-61785f/5375/summary.html
CMSSW: CMSSW_11_1_X_2020-03-25-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-03-25T14:41:48Z

Comparison job queued.

cmsbuild · 2020-03-25T16:07:51Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-61785f/5375/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2692110
DQMHistoTests: Total failures: 38
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2691753
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

pohsun · 2020-03-28T09:21:26Z

+1

perrotta

These are all the comments I have for this PR

perrotta · 2020-04-06T08:38:05Z

RecoEcal/Configuration/python/RecoEcal_EventContent_cff.py

+        'keep recoSuperClusters_correctedHybridSuperClusters_*_*',
+	# Endcap clusters
+        'keep *_multi5x5SuperClusters_*_*',
+        'keep recoSuperClusters_multi5x5SuperClusters_*_*',


This is a subset of the line above... But ok, it doesn't hurt

perrotta · 2020-04-06T08:40:55Z

RecoEcal/Configuration/python/RecoEcal_EventContent_cff.py

+        # Particle Flow superclusters
+        'keep *_particleFlowSuperClusterECAL_*_*',
+        'keep *_particleFlowSuperClusterOOTECAL_*_*',
+	# DROP statements # not used


How did you verify that these drop statements are not used?
@afiqaize @SohamBhattacharya

By my event content comparison check, these drop statements don't really act on the RECO/FEVT event content. When I checked every event content without these drop statements like this modified file, there is no difference in event contents.
@afiqaize @SohamBhattacharya Please confirm if these 4 drop statements can be removed.

For now, it would be good to have these drop statements in RecoEcalRECO PSets as the original. I'll not remove them. It will safe in the new commit.

perrotta · 2020-04-06T08:47:12Z

RecoEcal/Configuration/python/RecoEcal_EventContent_cff.py

+	# Barrel clusters
+        'keep *_correctedHybridSuperClusters_*_*',
+	# Endcap clusters
+        'keep *_multi5x5*_*_*',


Can this get expanded without the wild card (and without blowing the list too much)?

If this is in FEVT, I think that an exception can be made, considering that this datatier is more debug-like than something for sustained production

I've found 5 modules in RECO/FEVT event contents :
multi5x5BasicClustersCleaned
multi5x5SuperClusters
multi5x5SuperClustersCleaned
multi5x5SuperClustersWithPreshower
multi5x5PreshowerClusterShape
with different types and labels.

vector<reco::CaloCluster> multi5x5BasicClustersCleaned * * vector<reco::SuperCluster> multi5x5BasicClustersCleaned * * vector<reco::CaloCluster> multi5x5SuperClusters * * vector<reco::SuperCluster> multi5x5SuperClusters * * vector<reco::PreshowerCluster> multi5x5SuperClustersWithPreshower * * vector<reco::SuperCluster> multi5x5SuperClustersWithPreshower * * vector<reco::PreshowerClusterShape> multi5x5PreshowerClusterShape * *

perrotta · 2020-04-06T08:49:05Z

RecoEcal/Configuration/python/RecoEcal_EventContent_cff.py

+        'keep *_correctedHybridSuperClusters_*_*',
+	# Endcap clusters
+        'keep *_multi5x5*_*_*',
+        'keep *_correctedMulti5x5*_*_*')


Is there anything else beside correctedMulti5x5SuperClustersWithPreshower already saved in the AOD event content?

No, only vector<reco::SuperCluster> "correctedMulti5x5SuperClustersWithPreshower" "" "RECO" exists in AOD event content (also in RECO/FEVT event contents). So this line is redundant so it can be deleted.

cmsbuild · 2020-04-20T12:55:41Z

Pull request #29299 was updated. @perrotta, @slava77, @christopheralanwest, @tocheng, @cmsbuild, @tlampen, @pohsun can you please check and sign again.

perrotta · 2020-04-20T15:14:25Z

please test

cmsbuild · 2020-04-20T15:14:50Z

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/5787/console Started: 2020/04/20 17:20

cmsbuild · 2020-04-20T16:29:56Z

+1
Tested at: c960422
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-61785f/5787/summary.html
CMSSW: CMSSW_11_1_X_2020-04-20-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-04-20T16:29:59Z

Comparison job queued.

cmsbuild · 2020-04-20T18:00:23Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-61785f/5787/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 34
DQMHistoTests: Total histograms compared: 2696435
DQMHistoTests: Total failures: 2
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2696114
DQMHistoTests: Total skipped: 319
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
Checked 147 log files, 16 edm output root files, 34 DQM output files

perrotta · 2020-04-20T19:07:00Z

+1

EventContent structure updated as planned, without changing the list of saved products in AOD/RECO/FEVT outputs
Jenkins tests pass and show no differences

pohsun · 2020-04-21T16:05:55Z

+1

cmsbuild · 2020-04-21T16:24:36Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo (and backports should be raised in the release meeting by the corresponding L2)

silviodonato · 2020-04-21T17:05:06Z

+1

update event content structure and remove no use drops

14ed839

cmsbuild added this to the CMSSW_11_1_X milestone Mar 25, 2020

cmsbuild added alca-pending code-checks-pending comparison-pending orp-pending pending-signatures reconstruction-pending tests-pending labels Mar 25, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 25, 2020

cmsbuild added tests-started and removed tests-pending labels Mar 25, 2020

cmsbuild added tests-approved and removed tests-started labels Mar 25, 2020

cmsbuild added comparison-available and removed comparison-pending labels Mar 25, 2020

cmsbuild added alca-approved and removed alca-pending labels Mar 28, 2020

perrotta reviewed Apr 6, 2020

View reviewed changes

jeongeun mentioned this pull request Apr 14, 2020

Update RecoParticleFlow eventcontent format and remove drop #29470

Merged

cmsbuild added the code-checks-approved label Apr 20, 2020

cmsbuild added tests-started and removed tests-pending labels Apr 20, 2020

cmsbuild added tests-approved and removed tests-started labels Apr 20, 2020

cmsbuild added comparison-available and removed comparison-pending labels Apr 20, 2020

cmsbuild added reconstruction-approved and removed reconstruction-pending labels Apr 20, 2020

cmsbuild added alca-approved fully-signed and removed alca-pending pending-signatures labels Apr 21, 2020

cmsbuild added orp-approved and removed orp-pending labels Apr 21, 2020

cmsbuild merged commit 43535d4 into cms-sw:master Apr 21, 2020

This was referenced Apr 22, 2020

Update eventcontent structure in RecoPixelVertexing #29535

Merged

Update eventcontent structure in RecoTauTag RecoMTD #29571

Merged

Update eventcontent structure in RecoHI packages #29576

Merged

jeongeun mentioned this pull request May 13, 2020

Update event content structure for cosmic in RecoEcal #29821

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update event content structure in RecoEcal and RecoVertex #29299

Update event content structure in RecoEcal and RecoVertex #29299

jeongeun commented Mar 25, 2020 •

edited

Loading

cmsbuild commented Mar 25, 2020

cmsbuild commented Mar 25, 2020

cmsbuild commented Mar 25, 2020

perrotta commented Mar 25, 2020

cmsbuild commented Mar 25, 2020 •

edited

Loading

cmsbuild commented Mar 25, 2020

cmsbuild commented Mar 25, 2020

cmsbuild commented Mar 25, 2020

pohsun commented Mar 28, 2020

perrotta left a comment

perrotta Apr 6, 2020

perrotta Apr 6, 2020

jeongeun Apr 8, 2020

jeongeun Apr 20, 2020 •

edited

Loading

perrotta Apr 6, 2020

slava77 Apr 6, 2020

jeongeun Apr 7, 2020 •

edited

Loading

perrotta Apr 6, 2020

jeongeun Apr 7, 2020

cmsbuild commented Apr 20, 2020

perrotta commented Apr 20, 2020

cmsbuild commented Apr 20, 2020 •

edited

Loading

cmsbuild commented Apr 20, 2020

cmsbuild commented Apr 20, 2020

cmsbuild commented Apr 20, 2020

perrotta commented Apr 20, 2020

pohsun commented Apr 21, 2020

cmsbuild commented Apr 21, 2020

silviodonato commented Apr 21, 2020

Update event content structure in RecoEcal and RecoVertex #29299

Update event content structure in RecoEcal and RecoVertex #29299

Conversation

jeongeun commented Mar 25, 2020 • edited Loading

PR description:

PR validation:

cmsbuild commented Mar 25, 2020

cmsbuild commented Mar 25, 2020

cmsbuild commented Mar 25, 2020

perrotta commented Mar 25, 2020

cmsbuild commented Mar 25, 2020 • edited Loading

cmsbuild commented Mar 25, 2020

cmsbuild commented Mar 25, 2020

cmsbuild commented Mar 25, 2020

pohsun commented Mar 28, 2020

perrotta left a comment

Choose a reason for hiding this comment

perrotta Apr 6, 2020

Choose a reason for hiding this comment

perrotta Apr 6, 2020

Choose a reason for hiding this comment

jeongeun Apr 8, 2020

Choose a reason for hiding this comment

jeongeun Apr 20, 2020 • edited Loading

Choose a reason for hiding this comment

perrotta Apr 6, 2020

Choose a reason for hiding this comment

slava77 Apr 6, 2020

Choose a reason for hiding this comment

jeongeun Apr 7, 2020 • edited Loading

Choose a reason for hiding this comment

perrotta Apr 6, 2020

Choose a reason for hiding this comment

jeongeun Apr 7, 2020

Choose a reason for hiding this comment

cmsbuild commented Apr 20, 2020

perrotta commented Apr 20, 2020

cmsbuild commented Apr 20, 2020 • edited Loading

cmsbuild commented Apr 20, 2020

cmsbuild commented Apr 20, 2020

cmsbuild commented Apr 20, 2020

perrotta commented Apr 20, 2020

pohsun commented Apr 21, 2020

cmsbuild commented Apr 21, 2020

silviodonato commented Apr 21, 2020

jeongeun commented Mar 25, 2020 •

edited

Loading

cmsbuild commented Mar 25, 2020 •

edited

Loading

jeongeun Apr 20, 2020 •

edited

Loading

jeongeun Apr 7, 2020 •

edited

Loading

cmsbuild commented Apr 20, 2020 •

edited

Loading