Add support for wildcard in EDAlias type field #31269

makortel · 2020-08-27T19:18:44Z

PR description:

This PR adds support for wildcard in EDAlias type field, i.e.

alias = cms.EDAlias(
    module = cms.VPSet(
        cms.PSet(
            type = cms.string("*")
        )
    )
)

For the case that the EDAlias should pick all products of a module, the following shorthand is included

alias = cms.EDAlias(
    module = cms.EDAlias.allProducts()
)

Resolves #29847

PR validation:

Framework unit tests pass

Also add a shorthand EDAlias.allProducts()

cmsbuild · 2020-08-27T19:19:07Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-08-27T19:25:55Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31269/17950

This PR adds an extra 52KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Framework/src/Schedule.cc modified in PR(s): Use TransitionInfo in more interfaces, part 2 #31136, Delete unscheduled modules that are not consumed by any scheduled module #29553, Use TransitionInfo in more interfaces #30969
- File FWCore/Integration/test/BuildFile.xml modified in PR(s): Extend thinning to support slimming #31100

cmsbuild · 2020-08-27T19:26:21Z

A new Pull Request was created by @makortel (Matti Kortelainen) for master.

It involves the following packages:

FWCore/Framework
FWCore/Integration
FWCore/ParameterSet

@makortel, @smuzaffar, @cmsbuild, @Dr15Jones can you please review it and eventually sign? Thanks.
@wddgit this is something you requested to watch as well.
@silviodonato, @dpiparo, @qliphy you are the release manager for this.

cms-bot commands are listed here

makortel · 2020-08-27T19:33:29Z

@cmsbuild, please test

cmsbuild · 2020-08-27T19:33:51Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-08-27-1100/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/8967/console Started: 2020/08/27 21:34

cmsbuild · 2020-08-27T21:27:22Z

+1
Tested at: c0144f2
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d44067/8967/summary.html
CMSSW: CMSSW_11_2_X_2020-08-27-1100
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-08-27T21:27:25Z

Comparison job queued.

cmsbuild · 2020-08-27T22:55:37Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d44067/8967/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2609667
DQMHistoTests: Total failures: 1
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 2609644
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 34 files compared)
Checked 149 log files, 22 edm output root files, 35 DQM output files

fwyzard · 2020-08-28T08:37:51Z

FWCore/Integration/test/EDAlias_t.cpp

+process.intalias = cms.EDAlias(
+    intprod = cms.EDAlias.allProducts(),
+    # can't use allProducts() because the product instance '' would lead to duplicate brances to be aliased
+    intprod2 = cms.VPSet(
+        cms.PSet(type = cms.string('*'), fromProductInstance = cms.string('foo2')),
+        cms.PSet(type = cms.string('*'), fromProductInstance = cms.string('another2')),
+    )
+)


does this mean that

alias = cms.EDAlias( module = cms.VPSet( cms.PSet( type = cms.string("*") ) ) )

and the allProducts() shorthand work only if a module does not produce multiple collections with the same type ?

does this mean that ... and the allProducts() shorthand work only if a module does not produce multiple collections with the same type ?

It does not. The first added test

cmssw/FWCore/Integration/test/EDAlias_t.cpp

Line 136 in c0144f2

process.intalias = cms.EDAlias(intprod = cms.EDAlias.allProducts())

tests exactly that, creating an EDAlias for a module that produces three edmtest::IntProduct collections.

This test tests a case where products from two modules are aliased-for in a single EDAlias, and that type type wildcard works there as well. Because of my laziness of constructing the test (essentially the intprod and intprod2 producing products of the same type, and both producing a product without product instance name) the simpler

process.intalias = cms.EDAlias( intprod = cms.EDAlias.allProducts(), intprod2 = cms.EDAlias.allProducts(), )

would lead to two products of (edmtest::IntProduct, "intalias", "", "TEST"), which is not allowed.

Ah, I see, the conflict would be between two modules producing a collection with the same type and label.

makortel · 2020-08-28T15:13:09Z

@wddgit Could you review the changes? Thanks.

wddgit · 2020-08-28T15:17:26Z

Yes. I will review it today.

FWCore/Framework/src/Schedule.cc

cmsbuild · 2020-09-02T18:51:00Z

The code-checks are being triggered in jenkins.

cmsbuild · 2020-09-02T18:58:48Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-31269/18104

This PR adds an extra 40KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Framework/src/Schedule.cc modified in PR(s): Use TransitionInfo in more interfaces, part 2 #31136, Delete unscheduled modules that are not consumed by any scheduled module #29553, Use TransitionInfo in more interfaces #30969

cmsbuild · 2020-09-02T18:59:15Z

Pull request #31269 was updated. @makortel, @smuzaffar, @cmsbuild, @Dr15Jones can you please check and sign again.

makortel · 2020-09-02T19:01:14Z

@cmsbuild, please test

cmsbuild · 2020-09-02T19:01:38Z

The tests are being triggered in jenkins.

CMSSW_11_2_X_2020-09-01-2300/slc7_amd64_gcc820: https://cmssdt.cern.ch/jenkins/job/ib-run-pr-tests/9079/console Started: 2020/09/02 21:02

cmsbuild · 2020-09-02T20:58:36Z

+1
Tested at: 5340da8
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d44067/9079/summary.html
CMSSW: CMSSW_11_2_X_2020-09-01-2300
SCRAM_ARCH: slc7_amd64_gcc820

cmsbuild · 2020-09-02T20:58:38Z

Comparison job queued.

cmsbuild · 2020-09-02T22:23:14Z

Comparison is ready
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d44067/9079/summary.html

Comparison Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 35
DQMHistoTests: Total histograms compared: 2609667
DQMHistoTests: Total failures: 5
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 2609639
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 34 files compared)
DQMHistoSizes: changed ( 10224.0 ): 0.004 KiB MessageLogger/Warnings
Checked 149 log files, 22 edm output root files, 35 DQM output files

makortel · 2020-09-02T23:41:42Z

+1

cmsbuild · 2020-09-02T23:42:04Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @silviodonato, @dpiparo, @qliphy (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2020-09-03T01:14:58Z

+1

Add support for wildcards in EDAlias type field

c0144f2

Also add a shorthand EDAlias.allProducts()

cmsbuild added this to the CMSSW_11_2_X milestone Aug 27, 2020

cmsbuild added code-checks-pending comparison-pending core-pending orp-pending pending-signatures tests-pending labels Aug 27, 2020

makortel mentioned this pull request Aug 27, 2020

implement more firendly syntax for using EDAlias #29847

Closed

cmsbuild added code-checks-approved and removed code-checks-pending labels Aug 27, 2020

cmsbuild added tests-started and removed tests-pending labels Aug 27, 2020

cmsbuild added tests-approved and removed tests-started labels Aug 27, 2020

cmsbuild added comparison-available and removed comparison-pending labels Aug 27, 2020

cmsbuild mentioned this pull request Aug 28, 2020

Extend thinning to support slimming #31100

Merged

fwyzard reviewed Aug 28, 2020

View reviewed changes

wddgit reviewed Aug 28, 2020

View reviewed changes

FWCore/Framework/src/Schedule.cc Outdated Show resolved Hide resolved

cmsbuild added code-checks-pending tests-pending labels Sep 2, 2020

cmsbuild added code-checks-approved and removed code-checks-pending labels Sep 2, 2020

cmsbuild added tests-started and removed tests-pending labels Sep 2, 2020

cmsbuild added tests-approved and removed tests-started labels Sep 2, 2020

cmsbuild added comparison-available and removed comparison-pending labels Sep 2, 2020

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels Sep 2, 2020

cmsbuild added orp-approved and removed orp-pending labels Sep 3, 2020

cmsbuild merged commit f2d3972 into cms-sw:master Sep 3, 2020

cmsbuild mentioned this pull request Sep 3, 2020

DQM: Remove concurrent-lumi blocking DQM Modules. #30111

Closed

makortel deleted the edaliasAllProducts branch September 4, 2020 01:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for wildcard in EDAlias type field #31269

Add support for wildcard in EDAlias type field #31269

makortel commented Aug 27, 2020

cmsbuild commented Aug 27, 2020

cmsbuild commented Aug 27, 2020

cmsbuild commented Aug 27, 2020

makortel commented Aug 27, 2020

cmsbuild commented Aug 27, 2020 •

edited

Loading

cmsbuild commented Aug 27, 2020

cmsbuild commented Aug 27, 2020

cmsbuild commented Aug 27, 2020

fwyzard Aug 28, 2020

makortel Aug 28, 2020 •

edited

Loading

fwyzard Aug 28, 2020

makortel commented Aug 28, 2020

wddgit commented Aug 28, 2020

cmsbuild commented Sep 2, 2020

cmsbuild commented Sep 2, 2020

cmsbuild commented Sep 2, 2020

makortel commented Sep 2, 2020

cmsbuild commented Sep 2, 2020 •

edited

Loading

cmsbuild commented Sep 2, 2020

cmsbuild commented Sep 2, 2020

cmsbuild commented Sep 2, 2020

makortel commented Sep 2, 2020

cmsbuild commented Sep 2, 2020

qliphy commented Sep 3, 2020

Add support for wildcard in EDAlias type field #31269

Add support for wildcard in EDAlias type field #31269

Conversation

makortel commented Aug 27, 2020

PR description:

PR validation:

cmsbuild commented Aug 27, 2020

cmsbuild commented Aug 27, 2020

cmsbuild commented Aug 27, 2020

makortel commented Aug 27, 2020

cmsbuild commented Aug 27, 2020 • edited Loading

cmsbuild commented Aug 27, 2020

cmsbuild commented Aug 27, 2020

cmsbuild commented Aug 27, 2020

fwyzard Aug 28, 2020

Choose a reason for hiding this comment

makortel Aug 28, 2020 • edited Loading

Choose a reason for hiding this comment

fwyzard Aug 28, 2020

Choose a reason for hiding this comment

makortel commented Aug 28, 2020

wddgit commented Aug 28, 2020

cmsbuild commented Sep 2, 2020

cmsbuild commented Sep 2, 2020

cmsbuild commented Sep 2, 2020

makortel commented Sep 2, 2020

cmsbuild commented Sep 2, 2020 • edited Loading

cmsbuild commented Sep 2, 2020

cmsbuild commented Sep 2, 2020

cmsbuild commented Sep 2, 2020

makortel commented Sep 2, 2020

cmsbuild commented Sep 2, 2020

qliphy commented Sep 3, 2020

cmsbuild commented Aug 27, 2020 •

edited

Loading

makortel Aug 28, 2020 •

edited

Loading

cmsbuild commented Sep 2, 2020 •

edited

Loading