Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[12_4_X] Use "best matching" EDAlias in module dependence checks when placing ConditionalTask modules in Path #39530

Closed
wants to merge 1 commit into from
Closed

[12_4_X] Use "best matching" EDAlias in module dependence checks when placing ConditionalTask modules in Path #39530

wants to merge 1 commit into from

Conversation

makortel
Copy link
Contributor

PR description:

Backport of #39409. From original description

This PR fixes the problem reported in #36938 (comment). If the EDAlias has wildcards for one module, and more specific aliases for another module, we should use the one that has the most constrained match. There is one ambiguous case left, for which an exception is thrown. I hope we won't encounter that situation in practice.

PR validation:

Tests of #39409, also the original development

@makortel
Use "best matching" EDAlias in module dependence checks when placing …
708b8d3 
…ConditionalTask modules in Path

If the EDAlias has wildcards for one module, and more specific aliases
for another module, we should use the one that has the most
constrained match. There is one ambiguous case left, for which an
exception is thrown. I hope we won't encounter that situation in
practice.

The problem was encountered with a SwitchProducer having the EDAlias
case, but I think the problem could happen also without
SwitchProducer.
@cmsbuild cmsbuild added this to the CMSSW_12_4_X milestone Sep 28, 2022
@cmsbuild
Copy link
Contributor

cmsbuild commented Sep 28, 2022 •
edited
Loading

A new Pull Request was created by @makortel (Matti Kortelainen) for CMSSW_12_4_X.

It involves the following packages:

  • FWCore/Framework (core)
  • FWCore/Integration (core)

@cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please review it and eventually sign? Thanks.
@missirol, @wddgit this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

@makortel makortel changed the title [12_4_X]Use "best matching" EDAlias in module dependence checks when placing ConditionalTask modules in Path [12_4_X] Use "best matching" EDAlias in module dependence checks when placing ConditionalTask modules in Path Sep 28, 2022
@makortel
Copy link
Contributor Author

type bug-fix

@makortel
Copy link
Contributor Author

backport of #39409

@makortel
Copy link
Contributor Author

@cmsbuild, please test

This was referenced Sep 28, 2022
Open
Closed
@cmsbuild
Copy link
Contributor

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-7e98ac/27830/summary.html
COMMIT: 708b8d3
CMSSW: CMSSW_12_4_X_2022-09-28-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/39530/27830/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 4 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 3677402
  • DQMHistoTests: Total failures: 8
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 3677372
  • DQMHistoTests: Total skipped: 22
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 208 log files, 45 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

@makortel
Copy link
Contributor Author

+1

@cmsbuild
Copy link
Contributor

This pull request is fully signed and it will be integrated in one of the next CMSSW_12_4_X IBs (tests are also fine) and once validation in the development release cycle CMSSW_12_6_X is complete. This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta
perrotta reviewed Oct 1, 2022
View reviewed changes
Copy link
Contributor

@perrotta perrotta left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please tell where this difference wrt what merged in master and in 12_5_X comes from?
Moreover, could you explain why is this needed for 12_4_X: will HLT request a new release for the last few weeks of 2022 data taking with this fix?

FWCore/Framework/src/StreamSchedule.cc
@@ -143,6 +144,74 @@ namespace edm {

return workerManager.getWorker(*modpset, preg, prealloc, processConfiguration, moduleLabel);
}

std::optional<std::string> findBestMatchingAlias(
std::multimap<std::string, edm::BranchDescription const*> const& conditionalModuleBranches,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two are std::unordered_multimap in master and 12_5_X

@makortel
Copy link
Contributor Author

makortel commented Oct 3, 2022

Could you please tell where this difference wrt what merged in master and in 12_5_X comes from?

The multimap vs unordered_multimap difference comes from #38790 that was merged for 12_5_X. I didn't backport it at the time because the performance difference looked negligible (ok, one could ask if it would make sense to backport it now with the argument of keeping the commit of this PR the same as in 12_5_X and master).

Moreover, could you explain why is this needed for 12_4_X: will HLT request a new release for the last few weeks of 2022 data taking with this fix?

I blindly provided a fix for the release where the problem was seen. I leave it to @cms-sw/hlt-l2 to comment if it is really needed in 12_4_X for the remaining 2022 data taking. (and if not, I'm fine with just closing this PR).

@Martin-Grunewald
Copy link
Contributor

Currently the HLT menu uses your fix of providing the collection labels explicitely, rather than using * so I suppose it is not strictly needed for 12_4.

@makortel
Copy link
Contributor Author

makortel commented Oct 4, 2022

Closing following the discussion in ORP. If need arises nevertheless, I can reopen the PR.

@makortel makortel closed this Oct 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@perrotta perrotta perrotta left review comments

Assignees
No one assigned
Labels
backport-ok bug-fix core-approved fully-signed orp-pending tests-approved
Projects
None yet
Milestone
CMSSW_12_4_X
Development

Successfully merging this pull request may close these issues.
4 participants
@makortel @cmsbuild @Martin-Grunewald @perrotta