Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement an EDFilter to select events based on their bunch crossing number (10.3.x) #25019

Closed
wants to merge 2 commits into from
Closed

Implement an EDFilter to select events based on their bunch crossing number (10.3.x) #25019

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
618cd1f
Implement an EDFilter to select events based on their bunch crossing
fwyzard Oct 26, 2018
cccd39a
Use a local skimmed file as the test input
fwyzard Dec 14, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions FWCore/Modules/src/BunchCrossingFilter.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@

#include "FWCore/Framework/interface/global/EDFilter.h"
#include "FWCore/Framework/interface/MakerMacros.h"
#include "FWCore/Framework/interface/Event.h"
#include "FWCore/ParameterSet/interface/ConfigurationDescriptions.h"
#include "FWCore/ParameterSet/interface/ParameterSet.h"
#include "FWCore/ParameterSet/interface/ParameterSetDescription.h"

template <typename T>
std::vector<T> sorted(std::vector<T> v) {
std::sort(v.begin(), v.end());
return v;
}

namespace edm {
class BunchCrossingFilter : public global::EDFilter<> {
public:
explicit BunchCrossingFilter(ParameterSet const& config);

static void fillDescriptions(ConfigurationDescriptions& descriptions);
bool filter(StreamID, Event& event, EventSetup const&) const final;

private:
const std::vector<unsigned int> bunches_;
};

BunchCrossingFilter::BunchCrossingFilter(ParameterSet const& config) :
bunches_(sorted(config.getParameter<std::vector<unsigned int>>("bunches")))
{
}

bool BunchCrossingFilter::filter(StreamID, Event& event, EventSetup const&) const
{
return std::binary_search(bunches_.begin(), bunches_.end(), event.bunchCrossing());
}

void
BunchCrossingFilter::fillDescriptions(ConfigurationDescriptions& descriptions) {
ParameterSetDescription desc;
desc.add<std::vector<unsigned int>>("bunches", {})->setComment("List of bunch crossings for which events should be accepted [1-3564].");
descriptions.add("bunchCrossingFilter", desc);
}
}

using edm::BunchCrossingFilter;
DEFINE_FWK_MODULE(BunchCrossingFilter);
78 changes: 78 additions & 0 deletions FWCore/Modules/test/testBunchCrossingFilter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
# simple test for the BunchCrossingFilter

# colliding bunches in run 317435, fill 6759, scheme 25ns_2556b_2544_2215_2332_144bpi_20injV2:
# 81-128, 136-183
# 215-262, 270-317, 325-372,
# 404-451, 459-506, 514-561,
# 593-640, 648-695, 703-750,
# 786-833, 841-888,
# 920-967, 975-1022, 1030-1077,
# 1109-1156, 1164-1211, 1219-1266,
# 1298-1345, 1353-1400, 1408-1455,
# 1487-1534, 1542-1589, 1597-1644,
# 1680-1727, 1735-1782,
# 1814-1861, 1869-1916, 1924-1971,
# 2003-2050, 2058-2105, 2113-2160,
# 2192-2239, 2247-2294, 2302-2349,
# 2381-2428, 2436-2483, 2491-2538,
# 2574-2621, 2629-2676,
# 2708-2755, 2763-2810, 2818-2865,
# 2897-2944, 2952-2999, 3007-3054,
# 3086-3133, 3141-3188, 3196-3243,
# 3275-3322, 3330-3377, 3385-3432
# (see https://lpc.web.cern.ch/fillingSchemes/2018/25ns_2556b_2544_2215_2332_144bpi_20injV2.csv)

import FWCore.ParameterSet.Config as cms

process = cms.Process('TEST')

process.maxEvents = cms.untracked.PSet(
input = cms.untracked.int32(-1)
)

process.options = cms.untracked.PSet(
wantSummary = cms.untracked.bool(True)
)

process.load('FWCore.MessageService.MessageLogger_cfi')

# input data
process.source = cms.Source('PoolSource',
fileNames = cms.untracked.vstring('/store/data/Run2018B/ZeroBias/RAW/v1/000/317/435/00000/10FA8B3F-5A68-E811-A1D9-FA163E481185.root')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to avoid getting files from storage for framework tests since I do run these test on my laptop without internet connections. We normally either create new input files for the test or we store tiny ROOT files into the repository. Would any of the files in IOPool/Input/testdata work for your test?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the files under IOPool/Input/testdata have at most one event, so it is hard to show the various ways how to use the filter.
Of course if this is a requirement, I can simplify the test case and adapt it to that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could stick a tiny file in IOPool/Input/testdata if you want, or one in FWCore/Modules/testdata

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do I create a tiny file ?
Even if I drop all collections ("drop *"), a file with 1 event is ~1.3 MB.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the following

import FWCore.ParameterSet.Config as cms

process = cms.Process("SLIM")

process.source = cms.Source("PoolSource", fileNames = cms.untracked.vstring("file:10FA8B3F-5A68-E811-A1D9-FA163E481185.root"))

process.out = cms.OutputModule("PoolOutputModule", fileName = cms.untracked.string("slim.root"),
                               dropMetaData = cms.untracked.string("ALL"),
                               outputCommands = cms.untracked.vstring("drop *")
                               )

process.maxEvents = cms.untracked.PSet(input = cms.untracked.int32(10))

process.o = cms.EndPath(process.out)

the generated file was 1.5MB in size. Looking at the internals, the ParameterSets branch is 1.47MB in size. I'll see if I can come up with a simple way to fake that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you checkout IOPool/Provenance and edit IOPool/Provenance/src/CommonProvenanceFiller.cc and comment out the loop

https://github.com/cms-sw/cmssw/blob/master/IOPool/Provenance/src/CommonProvenanceFiller.cc#L24

and rebuild IOPool/Output. At that point, using the configuration I tested, you can create a slim.root file which is only 40KB in length. I was successfully able to read back that tiny file from a simple cmsRun job.

)

from FWCore.Modules.bunchCrossingFilter_cfi import bunchCrossingFilter as _bunchCrossingFilter

# empty input, do not select any bunch crossings
process.selectNone = _bunchCrossingFilter.clone(
bunches = cms.vuint32(),
)

# full range of possible bunch crossings [1,3564]
process.selectAll = _bunchCrossingFilter.clone(
bunches = cms.vuint32(range(1,3565))
)

# select bx 536
process.selectSingle = _bunchCrossingFilter.clone(
bunches = cms.vuint32(536)
)

# select the whole train 514-561
process.selectTrain = _bunchCrossingFilter.clone(
bunches = cms.vuint32(range(514,562))
)

# inverted to veto (non-colliding) bx 1
process.selectEmpty = _bunchCrossingFilter.clone(
bunches = cms.vuint32(1)
)

process.SelectNone = cms.Path( process.selectNone )
process.SelectAll = cms.Path( process.selectAll )
process.SelectSingle = cms.Path( process.selectSingle )
process.SelectTrain = cms.Path( process.selectTrain )
process.VetoEmpty = cms.Path( ~ process.selectEmpty )
process.VetoSingle = cms.Path( ~ process.selectSingle )
process.VetoTrain = cms.Path( ~ process.selectTrain )
process.SelectTrainButOne = cms.Path( process.selectTrain * ~ process.selectSingle )