Integration of the first version of muon DPG nanoAOD ntuples #38226

battibass · 2022-06-03T11:10:44Z

PR description:

This pull request is a follow-up of the plans about the migration of muon DPG ntuple code to nanoAOD, and its consequent integration in CMSSW.
The project was discussed in previous cross-POG meetings.

As there is no existing place in CMSSW where muon DPG offline analysis code is stored, we propose to add a new package. This was briefly discussed with @perrotta.
The package contains only muon DPG-related nanoAOD stuff, hence it was suggested to add @cms-sw/xpog-l2 as reviewers.

PR validation:

The PR integrates the code, but it does not include modules in workflows for now.
It was tested that code compiles and passes code-format and code-checks.

A test recipe was used to validate the ntuple production:

cmsenv

cd DPGAnalysis/MuonTools/test

voms-proxy-init -voms cms -valid 192:0

cmsRun muDpgNtuples_cfg.py

$CMSSW_RELEASE_BASE/src/PhysicsTools/NanoAOD/test/inspectNanoFile.py MuDPGNtuple_nanoAOD_ZMMPU.root --docmd=doc.md --sizemd=size.md

At the moment of the PR, the recipe runs fine in the latest IB, and produces an ntuple whose type, number and size of branches, match well what shown at cross-POG (total size of event branches, in Run3 ZMM MC including pile-up, is around 3.3 kB / ev)

cmsbuild · 2022-06-03T11:17:23Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38226/30380

This PR adds an extra 44KB to repository

cmsbuild · 2022-06-03T11:17:49Z

A new Pull Request was created by @battibass (Carlo Battilana) for master.

It involves the following packages:

DPGAnalysis/MuonTools (****)

The following packages do not have a category, yet:

DPGAnalysis/MuonTools
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@cmsbuild can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

mariadalfonso · 2022-06-03T14:27:55Z

assign xpog

cmsbuild · 2022-06-03T14:28:12Z

New categories assigned: xpog

@mariadalfonso,@gouskos,@fgolf you have been requested to review this Pull request/Issue and eventually sign? Thanks

mariadalfonso · 2022-06-07T17:47:55Z

@battibass

how we do test this centrally ?

cmsbuild · 2023-04-05T14:38:28Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38226/35053

This PR adds an extra 140KB to repository
Found files with invalid states:
- DPGAnalysis/MuonTools/plugins/MuNtupleGEMMuonFiller.h:
  - Added: 1ca1828
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleDTTPGThetaFiller.cc:
  - Added: 1ca1828
  - Deleted: 2c9d5bd
- DPGAnalysis/MuonTools/plugins/MuNtupleGEMRecHitFiller.h:
  - Added: 1ca1828
  - Modified: 0477982
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleRPCDigiFiller.cc:
  - Added: 1ca1828
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleRPCDigiFiller.h:
  - Added: 1ca1828
  - Modified: 0477982
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleCSCALCTDigiFiller.cc:
  - Added: 607c236
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/test/muDpgNtuples_cfg.py:
  - Added: 1ca1828
  - Modified: fa3c362, c4e1d3c
  - Deleted: 2c9d5bd
- DPGAnalysis/MuonTools/plugins/MuNtupleCSCWireDigiFiller.h:
  - Added: 607c236
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleMuonFiller.cc:
  - Added: 1ca1828
  - Modified: 0477982, ba2012e
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleGEMRecHitFiller.cc:
  - Added: 1ca1828
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleDTSegmentFiller.cc:
  - Added: 1ca1828
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleDTDigiFiller.cc:
  - Added: 1ca1828
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleCSCTnPFiller.h:
  - Added: fa3c362
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleGEMSegmentFiller.cc:
  - Added: 1ca1828
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleCSCALCTDigiFiller.h:
  - Added: 607c236
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleGEMDigiFiller.cc:
  - Added: 1ca1828
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleDTDigiFiller.h:
  - Added: 1ca1828
  - Modified: 0477982
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuRecoTableProducers.cc:
  - Added: 2c9d5bd
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleGEMSegmentFiller.h:
  - Added: 1ca1828
  - Modified: 0477982
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleBmtfFiller.cc:
  - Added: 1ca1828
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleRPCRecHitFiller.h:
  - Added: 1ca1828
  - Modified: 0477982
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleRPCRecHitFiller.cc:
  - Added: 1ca1828
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/src/MuRecObjBaseProducer.h:
  - Added: 2c9d5bd
  - Modified: a4872b0
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleCSCTnPFiller.cc:
  - Added: fa3c362
  - Modified: ba2012e
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleDTTPGThetaFiller.h:
  - Added: 1ca1828
  - Modified: 0477982
  - Deleted: 2c9d5bd
- DPGAnalysis/MuonTools/plugins/MuNtupleGEMMuonFiller.cc:
  - Added: 1ca1828
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleCSCWireDigiFiller.cc:
  - Added: 607c236
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleMuonFiller.h:
  - Added: 1ca1828
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleDTSegmentFiller.h:
  - Added: 1ca1828
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleBmtfFiller.h:
  - Added: 1ca1828
  - Modified: 0477982
  - Deleted: ddf5a8a
- DPGAnalysis/MuonTools/plugins/MuNtupleDTTPGPhiFiller.cc:
  - Added: 1ca1828
  - Deleted: 2c9d5bd
- DPGAnalysis/MuonTools/plugins/MuNtupleDTTPGPhiFiller.h:
  - Added: 1ca1828
  - Modified: 0477982
  - Deleted: 2c9d5bd
- DPGAnalysis/MuonTools/plugins/MuNtupleGEMDigiFiller.h:
  - Added: 1ca1828
  - Modified: 0477982
  - Deleted: ddf5a8a

cmsbuild · 2023-04-05T14:38:57Z

Pull request #38226 was updated. @cmsbuild, @simonepigazzini, @swertz, @vlimant can you please check and sign again.

battibass · 2023-04-05T14:39:17Z

As discussed during the cross-POG meeting on Feb 8th, here follow a few updates related to the muon DPG NANO.

In a nutshell:

a generic template table producer was added and used to write all DT/CSC/RPC/GEM digi tables;
a generic template table producer was added and used to write all DT/RPC/GEM local-reco tables;
a simpleFlatTableProducer was used to populate the "basic" table for reco muon objects;
(when needed) dedicated producers were slimmed to produce only information that can't be generated by the above table producers;
configuration in python modules, was also adapted to allow for central testing with cmsDriver.py.

Example commands are:

$ cmsDriver.py nano_mu_zmuskim --conditions auto:run3_data_relval -s RAW2DIGI,NANO:DPGAnalysis/MuonTools/muNtupleProducer_cff.muNtupleProducer --datatier NANOAOD --eventcontent NANOAOD --data --process muNANO --scenario pp --era Run3 -n 100 --filein file:/eos/cms/store/data/Run2022E/Muon/RAW-RECO/ZMu-10Dec2022-v2/330000/159d8e9d-7887-4543-a637-203f4c874c48.root

and:

$ cmsDriver.py nano_mu_bkg --conditions auto:run3_data_relval -s RAW2DIGI,NANO:DPGAnalysis/MuonTools/muNtupleProducerBkg_cff.muNtupleProducerBkg --datatier NANOAOD --eventcontent NANOAOD --data --process muNANO --scenario pp --era Run3 -n 100 --filein file:/eos/cms/store/data/Run2022E/Muon/RAW-RECO/ZMu-10Dec2022-v2/330000/159d8e9d-7887-4543-a637-203f4c874c48.root

cmsbuild · 2023-06-01T16:34:12Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38226/35747

This PR adds an extra 56KB to repository
Found files with invalid states:
- DPGAnalysis/MuonTools/plugins/MuDTSegmentExtTableProducer.h:
  - Added: 4fb3bc4
  - Deleted: 2ceed12
- DPGAnalysis/MuonTools/src/MuBaseFlatTableProducer.h:
  - Added: 4fb3bc4
  - Deleted: 2ceed12
- DPGAnalysis/MuonTools/src/MuNtupleUtils.h:
  - Added: 4fb3bc4
  - Deleted: 2ceed12
- DPGAnalysis/MuonTools/plugins/MuDTMuonExtTableProducer.h:
  - Added: 4fb3bc4
  - Deleted: 2ceed12
- DPGAnalysis/MuonTools/plugins/MuCSCTnPFlatTableProducer.h:
  - Added: 4fb3bc4
  - Deleted: 2ceed12
- DPGAnalysis/MuonTools/plugins/MuDTTPGPhiFlatTableProducer.h:
  - Added: 4fb3bc4
  - Deleted: 2ceed12
- DPGAnalysis/MuonTools/plugins/MuGEMMuonExtTableProducer.h:
  - Added: 4fb3bc4
  - Deleted: 2ceed12
- DPGAnalysis/MuonTools/src/MuDigiBaseProducer.h:
  - Added: 4fb3bc4
  - Deleted: 2ceed12
- DPGAnalysis/MuonTools/plugins/MuDTTPGThetaFlatTableProducer.h:
  - Added: 4fb3bc4
  - Deleted: 2ceed12
- DPGAnalysis/MuonTools/src/MuLocalRecoBaseProducer.h:
  - Added: 4fb3bc4
  - Deleted: 2ceed12

cmsbuild · 2023-06-01T16:34:43Z

Pull request #38226 was updated. @cmsbuild, @simonepigazzini, @vlimant can you please check and sign again.

battibass · 2023-06-01T16:36:49Z

please test

cmsbuild · 2023-06-01T19:14:19Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-5b5595/32938/summary.html
COMMIT: 307ff10
CMSSW: CMSSW_13_2_X_2023-06-01-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38226/32938/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 21 lines to the logs
Reco comparison results: 59 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3221457
DQMHistoTests: Total failures: 2234
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3219201
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 207 log files, 159 edm output root files, 48 DQM output files
TriggerResults: no differences found

simonepigazzini · 2023-06-05T08:16:46Z

+1

cmsbuild · 2023-06-05T08:17:11Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-06-05T10:02:00Z

+1

According to: cms-sw#38226 (comment)

cmsbuild added this to the CMSSW_12_5_X milestone Jun 3, 2022

cmsbuild added code-checks-pending new-package-pending orp-pending pending-signatures tests-pending labels Jun 3, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 3, 2022

cmsbuild added the xpog-pending label Jun 3, 2022

smuzaffar modified the milestones: CMSSW_12_5_X, CMSSW_12_6_X Aug 28, 2022

cmsbuild modified the milestones: CMSSW_12_6_X, CMSSW_13_0_X Nov 24, 2022

mariadalfonso mentioned this pull request Jan 25, 2023

Add prototype HcalNanoAOD #40489

Closed

smuzaffar modified the milestones: CMSSW_13_0_X, CMSSW_13_1_X Feb 11, 2023

battibass force-pushed the nanoAOD_muonDPGNtuples branch from 9512dd1 to e59c083 Compare April 5, 2023 14:31

cmsbuild added code-checks-pending and removed code-checks-approved labels Apr 5, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 5, 2023

cmsbuild removed the code-checks-approved label Apr 5, 2023

battibass force-pushed the nanoAOD_muonDPGNtuples branch from 7f0ff59 to 307ff10 Compare June 1, 2023 16:27

cmsbuild added tests-pending code-checks-pending and removed tests-rejected new-package-pending code-checks-approved labels Jun 1, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Jun 1, 2023

cmsbuild added tests-started and removed tests-pending labels Jun 1, 2023

cmsbuild added tests-approved and removed tests-started labels Jun 1, 2023

cmsbuild added fully-signed xpog-approved and removed pending-signatures xpog-pending labels Jun 5, 2023

cmsbuild added orp-approved and removed orp-pending labels Jun 5, 2023

cmsbuild merged commit 7c42f22 into cms-sw:master Jun 5, 2023

cmsbuild mentioned this pull request Jun 6, 2023

Update tensorflow to 2.11 cms-sw/cmsdist#8533

Closed

battibass pushed a commit to battibass/cmssw that referenced this pull request Jun 8, 2023

Rearrange file position and remove .h in plugins/

614c4e2

According to: cms-sw#38226 (comment)

battibass pushed a commit to battibass/cmssw that referenced this pull request Oct 14, 2023

Rearrange file position and remove .h in plugins/

6e14491

According to: cms-sw#38226 (comment)

battibass deleted the nanoAOD_muonDPGNtuples branch August 4, 2024 16:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integration of the first version of muon DPG nanoAOD ntuples #38226

Integration of the first version of muon DPG nanoAOD ntuples #38226

battibass commented Jun 3, 2022

cmsbuild commented Jun 3, 2022

cmsbuild commented Jun 3, 2022

mariadalfonso commented Jun 3, 2022

cmsbuild commented Jun 3, 2022

mariadalfonso commented Jun 7, 2022

cmsbuild commented Apr 5, 2023

cmsbuild commented Apr 5, 2023

battibass commented Apr 5, 2023

cmsbuild commented Jun 1, 2023

cmsbuild commented Jun 1, 2023

battibass commented Jun 1, 2023

cmsbuild commented Jun 1, 2023

simonepigazzini commented Jun 5, 2023

cmsbuild commented Jun 5, 2023

perrotta commented Jun 5, 2023

Integration of the first version of muon DPG nanoAOD ntuples #38226

Integration of the first version of muon DPG nanoAOD ntuples #38226

Conversation

battibass commented Jun 3, 2022

PR description:

PR validation:

cmsbuild commented Jun 3, 2022

cmsbuild commented Jun 3, 2022

mariadalfonso commented Jun 3, 2022

cmsbuild commented Jun 3, 2022

mariadalfonso commented Jun 7, 2022

cmsbuild commented Apr 5, 2023

cmsbuild commented Apr 5, 2023

battibass commented Apr 5, 2023

cmsbuild commented Jun 1, 2023

cmsbuild commented Jun 1, 2023

battibass commented Jun 1, 2023

cmsbuild commented Jun 1, 2023

Comparison Summary

simonepigazzini commented Jun 5, 2023

cmsbuild commented Jun 5, 2023

perrotta commented Jun 5, 2023