Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EventSetup consumes migration for MuonServiceProxy #28840

Merged
merged 1 commit into from
Feb 11, 2020

Conversation

wddgit
Copy link
Contributor

@wddgit wddgit commented Jan 31, 2020

PR description:

This completes the migration of MuonServiceProxy to call
consumes for the EventSetup products it gets. An earlier
commit added support for this if a new constructor was
called, but the old constructor still existed and most clients
were still using the old constructor. This commit migrates all
clients to use the new constructor and deletes the old one.

The main change is that a ConsumesCollector needs to
be passed to the MuonServiceProxy constructor as an argument.
Another change is that MuonServiceProxy now allows calling
update at beginRun because a few modules were doing that
instead of calling update during the event.

Note some modules were constructing and updating a MuonServiceProxy
object, but then never using it. Instead of migrating those, I deleted
MuonServiceProxy from those modules.

Note that this deletes the old constructor. This will break code
outside the repository that uses MuonServiceProxy, but eventually
support for that mode will be dropped by the Framework anyway.

PR validation:

Relies on existing tests. Nothing should change in the output.

This completes the migration of MuonServiceProxy to call
consumes for the EventSetup products it gets. An earlier
commit added support for this if a new constructor was
called but the old constructor still existed.
This commit migrates all clients to use the new constructor
and deletes the old one.

The main change is that a ConsumesCollector needs to
be passed to the MuonServiceProxy constructor.

Note some modules were constructing and updating a MuonServiceProxy
object, but then never using it. Instead of migrating those, I deleted
MuonServiceProxy from those modules.

Note that this deletes the old constructor. This will break code
outside the repository that uses MuonServiceProxy, but eventually
support for that mode will be dropped by the Framework anyway.
@cmsbuild
Copy link
Contributor

The code-checks are being triggered in jenkins.

@cmsbuild
Copy link
Contributor

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-28840/13583

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @wddgit (W. David Dagenhart) for master.

It involves the following packages:

Alignment/OfflineValidation
DQM/MuonMonitor
DQMOffline/Muon
FastSimulation/MuonSimHitProducer
RecoHI/HiMuonAlgos
RecoLocalMuon/CSCEfficiency
RecoMuon/CosmicMuonProducer
RecoMuon/GlobalMuonProducer
RecoMuon/GlobalTrackingTools
RecoMuon/L2MuonProducer
RecoMuon/L2MuonSeedGenerator
RecoMuon/L3MuonProducer
RecoMuon/MuonIdentification
RecoMuon/MuonIsolation
RecoMuon/MuonSeedGenerator
RecoMuon/StandAloneMuonProducer
RecoMuon/TrackerSeedGenerator
RecoMuon/TrackingTools
Validation/RecoMuon

@perrotta, @Martin-Grunewald, @ssekmen, @lveldere, @kmaeshima, @schneiml, @civanch, @sbein, @christopheralanwest, @andrius-k, @mdhildreth, @cmsbuild, @franzoni, @jfernan2, @tlampen, @fioriNTU, @slava77, @tocheng, @fwyzard, @pohsun can you please review it and eventually sign? Thanks.
@abbiendi, @echapon, @jazzitup, @tlampen, @cericeci, @battibass, @jhgoh, @dgulhan, @HuguesBrun, @ptcox, @trocino, @yetkinyilmaz, @barvic, @tocheng, @bellan, @mschrode, @mmusich, @amagitte, @adewit, @mandrenguyen, @MiheeJo, @Fedespring, @calderona, @yenjie, @bachtis, @kurtejung, @matt-komm, @rociovilar, @folguera this is something you requested to watch as well.
@davidlange6, @silviodonato, @fabiocos you are the release manager for this.

cms-bot commands are listed here

@wddgit
Copy link
Contributor Author

wddgit commented Jan 31, 2020

please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 31, 2020

The tests are being triggered in jenkins.
https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/4451/console Started: 2020/01/31 16:44

@cmsbuild
Copy link
Contributor

+1
Tested at: 65ec28c
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3ddfe3/4451/summary.html
CMSSW: CMSSW_11_1_X_2020-01-31-1100
SCRAM_ARCH: slc7_amd64_gcc820

@cmsbuild
Copy link
Contributor

Comparison job queued.

Comment on lines +40 to +45
enum class UseEventSetupIn { Run, Event, RunAndEvent };

/// Constructor
MuonServiceProxy(const edm::ParameterSet&, edm::ConsumesCollector&&);
MuonServiceProxy(const edm::ParameterSet&,
edm::ConsumesCollector&&,
UseEventSetupIn useEventSetupIn = UseEventSetupIn::Event);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure why these alternatives are needed.
The updates made per beginRun were likely made in attempt to optimize CPU use and perhaps also to cache some values.

Having, e.g. separate MagneticField per run and per event sounds rather questionable to me.
There should be just one (self)consistent ES product.

Or did I miss the subtlety in the framework and we have to declare separate tokens for the products which can be accessed in the BeginRun transition?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@slava77 So there are two different questions here:

  1. Does the framework require separate tokens if a module needs to get an EventSetup product in the event and also in beginRun? Yes. There have to be separate tokens for objects retrieved in beginRun and in the Event. The Transition is a template parameter of the esConsumes function.

https://cmssdt.cern.ch/lxr/source/FWCore/Framework/interface/EDConsumerBase.h#0168

One might be able to handle it with templates, but the complexity of that seemed worse than having 2 tokens and also if you are retrieving in both beginRun and Event in the same job with the same module, then you have to have 2 tokens.

  1. Why are we getting objects in the run in some modules, in the event in some modules, and in both run and event in some modules? Partially the answer is I just maintained the existing behavior. That is what the modules were doing before this PR. I'm not an expert in this part of the code and I didn't want to spend time trying to understand this behavior and fix it. Actually, all the modules only call update in the event except for two modules.

A. FastSimulation/MuonSimHitProducer/src/MuonSimHitProducer.cc gets the products by calling update only during beginRun, then there are function calls using MuonServiceProxy in both beginRun and the Event. I don't know what they are doing and whether this can be changed. One also might consider changing all the other modules to have this behavior. If the EventSetup products never change in the middle of a run (and never will in the future), then this might improve performance slightly.

B. RecoMuon/TrackerSeedGenerator/plugins/TSGFromL2Muon.cc calls update both during the run and during the event. And after update is called it uses the MuonServiceProxy in both the Event and beginRun. I don't know exactly what it is doing. It is quite possible it would be good enough to only get the data during the run and delete the update call during the event. I don't know. I was trying to avoid things that might possibly change results.

If an expert who understands this part of the code gives me some direction and takes responsibility for it, I would be happy to modify the code to behave in a different way. If I just guess based on my past experience and intuition, then I would say it is probable the data is the same for a run and all the lumps and events in that run. But I don't know and am not an expert in this area in CMS. This could also be addressed in a future PR. I am really focused on the consumes migration so we gain the ability to run EventSetup modules concurrently.

@cmsbuild
Copy link
Contributor

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3ddfe3/4451/summary.html

Comparison Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 34
  • DQMHistoTests: Total histograms compared: 2697068
  • DQMHistoTests: Total failures: 1
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 2696721
  • DQMHistoTests: Total skipped: 346
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 33 files compared)
  • Checked 147 log files, 16 edm output root files, 34 DQM output files

@slava77
Copy link
Contributor

slava77 commented Jan 31, 2020

+1

for #28840 65ec28c

  • technical
  • jenkins tests pass

@slava77
Copy link
Contributor

slava77 commented Jan 31, 2020

I added #28843 to follow up on #28840 (comment)

@pohsun
Copy link

pohsun commented Feb 2, 2020

+1

@jfernan2
Copy link
Contributor

jfernan2 commented Feb 3, 2020

+1
No dqm changes

@Martin-Grunewald
Copy link
Contributor

+1

@silviodonato
Copy link
Contributor

Kind reminder for fastsim: @civanch @lveldere @mdhildreth @sbein @ssekmen

@silviodonato
Copy link
Contributor

silviodonato commented Feb 10, 2020

Kind reminder for fastsim: @civanch @lveldere @mdhildreth @sbein @ssekmen
The deadline for CMSSW_11_1_0_pre3 is tomorrow (Feb 11)

@silviodonato
Copy link
Contributor

urgent

@silviodonato
Copy link
Contributor

merge
@civanch @lveldere @mdhildreth @sbein @ssekmen please have a look when you have time

@cmsbuild cmsbuild merged commit 6ba826d into cms-sw:master Feb 11, 2020
@wddgit wddgit deleted the esConsumesMuonServiceProxy2 branch May 21, 2020 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants