Adding offlineVerticesWithBS to miniAOD #33778

AdrianoDee · 2021-05-19T11:34:20Z

PR description:

This PR adds a new collection of offlineSlimmedPrimaryVerticesWithBS to miniAOD format. This collection comes from AOD offlinePrimaryVerticesWithBS with the same slimming and association used for offlineSlimmedPrimaryVertices.

The relative size increase is checked on a a set of Run3 ttbar samples with multiple :

The size per event:

The v3 shown here is the version with minPtForLowQualityTrackProperties=0.0 with #33777

Further checks in the references below.

PR Validation and Further references

BPH Jamboree December 2020 here

xPOG Meeting February 2021 here

Further reference here

For a complete documentation, trk4bph meetings: 1, 2, 3 , 4, 5, 6

cmsbuild · 2021-05-19T11:41:59Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33778/22745

This PR adds an extra 16KB to repository

cmsbuild · 2021-05-19T11:42:21Z

A new Pull Request was created by @AdrianoDee for master.

It involves the following packages:

PhysicsTools/PatAlgos

@perrotta, @jpata, @cmsbuild, @slava77 can you please review it and eventually sign? Thanks.
@rappoccio, @gouskos, @jdolen, @JyothsnaKomaragiri, @ahinzmann, @schoef, @emilbols, @swozniewski, @jdamgov, @mbluj, @nhanvtran, @gkasieczka, @clelange, @hatakeyamak, @ferencek, @gpetruc, @andrzejnovak, @mariadalfonso, @seemasharmafnal, @mmarionncern this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

slava77

it looks like use cases for most of the introduced producers are missing.
shouldn't there be at least a keep statement for the output event format?

slava77 · 2021-05-19T12:39:44Z

PhysicsTools/PatAlgos/python/slimming/primaryVertexAssociation_cfi.py

+
+run2_miniAOD_pp_on_AA_103X.toModify(primaryVertexWithBSAssociation,particles = "cleanedParticleFlow")
+primaryVertexWithBSAssociationCleaned = primaryVertexWithBSAssociation.clone(particles = "cleanedParticleFlow:removed")


is this really needed?

Sincerely I placed it for consistency, for me we may erase it.

@mandrenguyen @abaty could you please give your advice about the possible need of these offlineVerticesWithBS for HI?

It would be nice to have offlinePrimaryVerticesWithBS in HI miniAOD as well. I checked the increase in size on 100 events of wf 140.5611 and it looks negligible, around 0.1%. PU is so low in PbPb that multiple vertices are more likely to come from vertex splitting. So perhaps this is not unexpected.

It would be nice to have offlinePrimaryVerticesWithBS in HI miniAOD as well. I checked the increase in size on 100 events of wf 140.5611 and it looks negligible, around 0.1%. PU is so low in PbPb that multiple vertices are more likely to come from vertex splitting. So perhaps this is not unexpected.

Great, thank you Matt,
Therefore, this part can remain as it is,

…thBS

AdrianoDee · 2021-05-19T17:07:25Z

it looks like use cases for most of the introduced producers are missing.
shouldn't there be at least a keep statement for the output event format?

Indeed you are right, I added a keep to the MicroEventContent (for miniAOD)

cmsbuild · 2021-05-19T17:11:15Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33778/22755

This PR adds an extra 20KB to repository

cmsbuild · 2021-05-19T17:11:39Z

Pull request #33778 was updated. @perrotta, @jpata, @cmsbuild, @slava77 can you please check and sign again.

perrotta · 2021-05-21T09:36:13Z

PhysicsTools/PatAlgos/python/slimming/primaryVertexAssociation_cfi.py

@@ -4,10 +4,17 @@
 primaryVertexAssociation = sortedPFPrimaryVertices.clone(
  qualityForPrimary = cms.int32(2),
  produceSortedVertices = cms.bool(False),
-  producePileUpCollection  = cms.bool(False),  
+  producePileUpCollection  = cms.bool(False),


Please drop type specifications for all parameters which already exist.
This is a safer way to protect from parameter name mistakes and will also help in possible parameter migrations.

primaryVertexAssociation = sortedPFPrimaryVertices.clone( qualityForPrimary = 2, produceSortedVertices = False, producePileUpCollection = False, produceNoPileUpCollection = False )

perrotta · 2021-05-21T09:36:34Z

PhysicsTools/PatAlgos/python/slimming/primaryVertexAssociation_cfi.py

  produceNoPileUpCollection = cms.bool(False)
 )

+primaryVertexWithBSAssociation = primaryVertexAssociation.clone(
+  vertices = cms.InputTag("offlinePrimaryVerticesWithBS")


Suggested change

vertices = cms.InputTag("offlinePrimaryVerticesWithBS")

vertices = "offlinePrimaryVerticesWithBS"

perrotta · 2021-05-21T09:40:29Z

PhysicsTools/PatAlgos/python/slimming/primaryVertexAssociation_cfi.py

+
+run2_miniAOD_pp_on_AA_103X.toModify(primaryVertexWithBSAssociation,particles = "cleanedParticleFlow")
+primaryVertexWithBSAssociationCleaned = primaryVertexWithBSAssociation.clone(particles = "cleanedParticleFlow:removed")


@mandrenguyen @abaty could you please give your advice about the possible need of these offlineVerticesWithBS for HI?

perrotta · 2021-05-21T09:41:25Z

PhysicsTools/PatAlgos/python/slimming/slimming_cff.py

    cms.Task(slimmingTask.copy(), packedCandidateMuonID, packedPFCandidateTrackChi2, lostTrackChi2, centralityBin, hiHFfilters))
 from Configuration.ProcessModifiers.run2_miniAOD_pp_on_AA_103X_cff import run2_miniAOD_pp_on_AA_103X
 run2_miniAOD_pp_on_AA_103X.toReplaceWith(slimmingTask,cms.Task(primaryVertexAssociationCleaned,slimmingTask.copy()))
+run2_miniAOD_pp_on_AA_103X.toReplaceWith(slimmingTask,cms.Task(primaryVertexWithBSAssociationCleaned,slimmingTask.copy()))


Same here: insertion to be agreed with @mandrenguyen @abaty

perrotta · 2021-05-21T09:43:18Z

assign xpog
(in order to evaluate about the miniAOD file size increase reported in the PR description)

cmsbuild · 2021-05-21T09:43:37Z

New categories assigned: xpog

@fgolf,@mariadalfonso,@gouskos you have been requested to review this Pull request/Issue and eventually sign? Thanks

cmsbuild · 2021-05-21T11:37:52Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33778/22796

This PR adds an extra 16KB to repository

cmsbuild · 2021-05-28T14:04:49Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-33778/22915

This PR adds an extra 20KB to repository

cmsbuild · 2021-05-28T14:05:13Z

Pull request #33778 was updated. @perrotta, @gouskos, @cmsbuild, @fgolf, @slava77, @jpata, @mariadalfonso can you please check and sign again.

perrotta · 2021-05-28T14:06:28Z

please test

perrotta · 2021-05-28T14:12:57Z

+1

(Since the last update only consists in removing some unneeded comment lines I can sign it now, even though the tests are still running)
offlineVerticesWithBS added to miniAOD, as planned
The same addition propagates to HI miniAOD, and this has been approved by the HI reco contact
Increase in the miniAOD size seems manageable (but I'll let @cms-sw/xpog-l2 to better evaluate about it)
Jenkins tests pass, with no differences in the reco outputs besides the addition of the new collection

AdrianoDee · 2021-05-28T14:14:29Z

@perrotta thanks

mariadalfonso · 2021-05-28T14:54:39Z

+xpog

miniAOD size increase in the description inline with the discussion of the budget in the xPOG

cmsbuild · 2021-05-28T14:55:06Z

This pull request is fully signed and it will be integrated in one of the next master IBs after it passes the integration tests. This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild · 2021-05-28T17:23:14Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3bd2dd/15387/summary.html
COMMIT: 6689203
CMSSW: CMSSW_12_0_X_2021-05-28-1100/slc7_amd64_gcc900
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/33778/15387/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
ROOTFileChecks: Some differences in event products or their sizes found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 37
DQMHistoTests: Total histograms compared: 2650486
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2650457
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 36 files compared)
Checked 155 log files, 37 edm output root files, 37 DQM output files
TriggerResults: no differences found

silviodonato · 2021-05-28T19:52:28Z

+1

Adding offlineVerticesWithBS to miniAOD

0e16911

cmsbuild added this to the CMSSW_12_0_X milestone May 19, 2021

cmsbuild added code-checks-pending orp-pending pending-signatures reconstruction-pending tests-pending labels May 19, 2021

AdrianoDee mentioned this pull request May 19, 2021

Making miniAOD low pt tracks threshold tunable #33777

Merged

AdrianoDee marked this pull request as ready for review May 19, 2021 11:36

cmsbuild added code-checks-approved and removed code-checks-pending labels May 19, 2021

slava77 reviewed May 19, 2021

View reviewed changes

Updating MicroEventContent to include offlineSlimmedPrimaryVerticesWi…

3958a8a

…thBS

cmsbuild added code-checks-pending and removed code-checks-approved labels May 19, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels May 19, 2021

perrotta reviewed May 21, 2021

View reviewed changes

cmsbuild added the xpog-pending label May 21, 2021

Cleaning Type Specifications

1b82c33

cmsbuild added code-checks-pending and removed code-checks-approved labels May 21, 2021

cmsbuild added code-checks-pending tests-pending and removed tests-approved labels May 28, 2021

cmsbuild added code-checks-approved and removed code-checks-pending labels May 28, 2021

cmsbuild added tests-started and removed tests-pending labels May 28, 2021

cmsbuild added reconstruction-approved and removed reconstruction-pending labels May 28, 2021

cmsbuild added fully-signed xpog-approved and removed pending-signatures xpog-pending labels May 28, 2021

cmsbuild mentioned this pull request May 28, 2021

Unify vertex association between CHS and PUPPI #33885

Merged

cmsbuild added tests-approved and removed tests-started labels May 28, 2021

cmsbuild added orp-approved and removed orp-pending labels May 28, 2021

cmsbuild merged commit e8a1469 into cms-sw:master May 28, 2021

AdrianoDee mentioned this pull request May 31, 2021

[Backport] Adding offlineVerticesWithBS to miniAOD to 10_6_X #33922

Merged

slava77 mentioned this pull request Jun 2, 2021

features for bParking UL rereco and miniAOD in 10_6_X (10_6_26?) #33917

Closed

14 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding offlineVerticesWithBS to miniAOD #33778

Adding offlineVerticesWithBS to miniAOD #33778

AdrianoDee commented May 19, 2021 •

edited

Loading

cmsbuild commented May 19, 2021

cmsbuild commented May 19, 2021

slava77 left a comment

slava77 May 19, 2021

AdrianoDee May 19, 2021

perrotta May 21, 2021

mandrenguyen May 28, 2021

perrotta May 28, 2021

AdrianoDee commented May 19, 2021

cmsbuild commented May 19, 2021

cmsbuild commented May 19, 2021

perrotta May 21, 2021

perrotta May 21, 2021

perrotta May 21, 2021

perrotta May 21, 2021

perrotta commented May 21, 2021

cmsbuild commented May 21, 2021

cmsbuild commented May 21, 2021

cmsbuild commented May 28, 2021

cmsbuild commented May 28, 2021

perrotta commented May 28, 2021

perrotta commented May 28, 2021

AdrianoDee commented May 28, 2021

mariadalfonso commented May 28, 2021

cmsbuild commented May 28, 2021

cmsbuild commented May 28, 2021

silviodonato commented May 28, 2021


		run2_miniAOD_pp_on_AA_103X.toModify(primaryVertexWithBSAssociation,particles = "cleanedParticleFlow")
		primaryVertexWithBSAssociationCleaned = primaryVertexWithBSAssociation.clone(particles = "cleanedParticleFlow:removed")

	vertices = cms.InputTag("offlinePrimaryVerticesWithBS")
	vertices = "offlinePrimaryVerticesWithBS"

Adding offlineVerticesWithBS to miniAOD #33778

Adding offlineVerticesWithBS to miniAOD #33778

Conversation

AdrianoDee commented May 19, 2021 • edited Loading

PR description:

PR Validation and Further references

cmsbuild commented May 19, 2021

cmsbuild commented May 19, 2021

slava77 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AdrianoDee commented May 19, 2021

cmsbuild commented May 19, 2021

cmsbuild commented May 19, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

perrotta commented May 21, 2021

cmsbuild commented May 21, 2021

cmsbuild commented May 21, 2021

cmsbuild commented May 28, 2021

cmsbuild commented May 28, 2021

perrotta commented May 28, 2021

perrotta commented May 28, 2021

AdrianoDee commented May 28, 2021

mariadalfonso commented May 28, 2021

cmsbuild commented May 28, 2021

cmsbuild commented May 28, 2021

Comparison Summary

silviodonato commented May 28, 2021

AdrianoDee commented May 19, 2021 •

edited

Loading