Add prefetching ActivityRegistry signals for more transitions #38708

Dr15Jones · 2022-07-12T15:51:41Z

PR description:

All non-Event transitions now also have prefetching related signals.

PR validation:

Code compiles and framework test pass.

fixes #38634

cmsbuild · 2022-07-12T16:00:00Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38708/31005

This PR adds an extra 96KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Framework/interface/maker/Worker.h modified in PR(s): Have WaitingTask hold an exception_ptr directly #38621, Add Transformer ability to modules #38454
- File FWCore/Framework/test/global_filter_t.cppunit.cc modified in PR(s): Have WaitingTask hold an exception_ptr directly #38621, Add Transformer ability to modules #38454
- File FWCore/Framework/test/global_outputmodule_t.cppunit.cc modified in PR(s): Have WaitingTask hold an exception_ptr directly #38621
- File FWCore/Framework/test/global_producer_t.cppunit.cc modified in PR(s): Have WaitingTask hold an exception_ptr directly #38621, Add Transformer ability to modules #38454
- File FWCore/Framework/test/limited_filter_t.cppunit.cc modified in PR(s): Have WaitingTask hold an exception_ptr directly #38621, Add Transformer ability to modules #38454
- File FWCore/Framework/test/limited_outputmodule_t.cppunit.cc modified in PR(s): Have WaitingTask hold an exception_ptr directly #38621
- File FWCore/Framework/test/limited_producer_t.cppunit.cc modified in PR(s): Have WaitingTask hold an exception_ptr directly #38621, Add Transformer ability to modules #38454
- File FWCore/Framework/test/one_outputmodule_t.cppunit.cc modified in PR(s): Have WaitingTask hold an exception_ptr directly #38621
- File FWCore/Framework/test/stream_filter_t.cppunit.cc modified in PR(s): Have WaitingTask hold an exception_ptr directly #38621
- File FWCore/Framework/test/stream_producer_t.cppunit.cc modified in PR(s): Have WaitingTask hold an exception_ptr directly #38621

cmsbuild · 2022-07-12T16:00:27Z

A new Pull Request was created by @Dr15Jones (Chris Jones) for master.

It involves the following packages:

FWCore/Framework (core)
FWCore/Integration (core)
FWCore/ServiceRegistry (core)
FWCore/Services (core)

@cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please review it and eventually sign? Thanks.
@makortel, @wddgit, @fwyzard this is something you requested to watch as well.
@perrotta, @dpiparo, @qliphy, @rappoccio you are the release manager for this.

cms-bot commands are listed here

Dr15Jones · 2022-07-12T16:03:35Z

please test

FWCore/Framework/test/global_filter_t.cppunit.cc

FWCore/Framework/test/global_outputmodule_t.cppunit.cc

FWCore/Framework/test/global_producer_t.cppunit.cc

FWCore/Framework/test/limited_filter_t.cppunit.cc

FWCore/Framework/test/limited_outputmodule_t.cppunit.cc

FWCore/Framework/test/limited_producer_t.cppunit.cc

FWCore/Framework/test/one_outputmodule_t.cppunit.cc

FWCore/Framework/test/stream_filter_t.cppunit.cc

FWCore/Framework/test/stream_producer_t.cppunit.cc

FWCore/Integration/test/unit_test_outputs/testSubProcess.grep2.txt

cmsbuild · 2022-07-12T22:51:12Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-128fb8/26174/summary.html
COMMIT: 25df583
CMSSW: CMSSW_12_5_X_2022-07-12-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38708/26174/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 2 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3653734
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3653704
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 49 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

All non-Event transitions now also have prefetching related signals.

cmsbuild · 2022-07-13T15:09:40Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-38708/31028

This PR adds an extra 84KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File FWCore/Framework/interface/maker/Worker.h modified in PR(s): Use tbb::task_group::defer in FinalWaitingTask #38715, Add Transformer ability to modules #38454
- File FWCore/Framework/test/global_filter_t.cppunit.cc modified in PR(s): Use tbb::task_group::defer in FinalWaitingTask #38715, Add Transformer ability to modules #38454
- File FWCore/Framework/test/global_outputmodule_t.cppunit.cc modified in PR(s): Use tbb::task_group::defer in FinalWaitingTask #38715
- File FWCore/Framework/test/global_producer_t.cppunit.cc modified in PR(s): Use tbb::task_group::defer in FinalWaitingTask #38715, Add Transformer ability to modules #38454
- File FWCore/Framework/test/limited_filter_t.cppunit.cc modified in PR(s): Use tbb::task_group::defer in FinalWaitingTask #38715, Add Transformer ability to modules #38454
- File FWCore/Framework/test/limited_outputmodule_t.cppunit.cc modified in PR(s): Use tbb::task_group::defer in FinalWaitingTask #38715
- File FWCore/Framework/test/limited_producer_t.cppunit.cc modified in PR(s): Use tbb::task_group::defer in FinalWaitingTask #38715, Add Transformer ability to modules #38454
- File FWCore/Framework/test/one_outputmodule_t.cppunit.cc modified in PR(s): Use tbb::task_group::defer in FinalWaitingTask #38715
- File FWCore/Framework/test/stream_filter_t.cppunit.cc modified in PR(s): Use tbb::task_group::defer in FinalWaitingTask #38715
- File FWCore/Framework/test/stream_producer_t.cppunit.cc modified in PR(s): Use tbb::task_group::defer in FinalWaitingTask #38715

cmsbuild · 2022-07-13T15:10:07Z

Pull request #38708 was updated. @cmsbuild, @smuzaffar, @Dr15Jones, @makortel can you please check and sign again.

Dr15Jones · 2022-07-13T15:11:33Z

please test

cmsbuild · 2022-07-13T21:54:02Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-128fb8/26210/summary.html
COMMIT: c101a78
CMSSW: CMSSW_12_5_X_2022-07-13-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38708/26210/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test condTestRegression had ERRORS
---> test SiStripDAQ_O2O_test had ERRORS

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3653966
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3653930
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 49 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

makortel · 2022-07-13T22:06:17Z

+1

Unit test failures look unrelated.

cmsbuild · 2022-07-13T22:06:40Z

This pull request is fully signed and it will be integrated in one of the next master IBs (but tests are reportedly failing). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @qliphy, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

qliphy · 2022-07-14T00:13:04Z

please test

cmsbuild · 2022-07-14T08:22:43Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-128fb8/26221/summary.html
COMMIT: c101a78
CMSSW: CMSSW_12_5_X_2022-07-13-1100/el8_amd64_gcc10
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/38708/26221/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 50
DQMHistoTests: Total histograms compared: 3653966
DQMHistoTests: Total failures: 13
DQMHistoTests: Total nulls: 1
DQMHistoTests: Total successes: 3653930
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.004 KiB( 49 files compared)
DQMHistoSizes: changed ( 312.0 ): 0.004 KiB MessageLogger/Warnings
Checked 208 log files, 45 edm output root files, 50 DQM output files
TriggerResults: no differences found

qliphy · 2022-07-14T08:25:12Z

+1

cmsbuild added this to the CMSSW_12_5_X milestone Jul 12, 2022

cmsbuild added code-checks-pending core-pending orp-pending pending-signatures tests-pending labels Jul 12, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 12, 2022

cmsbuild added tests-started and removed tests-pending labels Jul 12, 2022

makortel mentioned this pull request Jul 12, 2022

Add Run and LuminosityBlock prefetching signals to ActivityRegistry cms-sw/framework-team#411

Closed

makortel reviewed Jul 12, 2022

View reviewed changes

cmsbuild mentioned this pull request Jul 12, 2022

Use tbb::task_group::defer in FinalWaitingTask #38715

Merged

cmsbuild added tests-approved and removed tests-started labels Jul 12, 2022

Add prefetching ActivityRegistry signals for more transitions

c101a78

All non-Event transitions now also have prefetching related signals.

Dr15Jones force-pushed the morePrefetchSignals branch from 25df583 to c101a78 Compare July 13, 2022 15:01

cmsbuild added code-checks-pending tests-pending and removed tests-approved code-checks-approved labels Jul 13, 2022

cmsbuild added code-checks-approved and removed code-checks-pending labels Jul 13, 2022

cmsbuild added tests-started and removed tests-pending labels Jul 13, 2022

cmsbuild added tests-rejected and removed tests-started labels Jul 13, 2022

cmsbuild added core-approved fully-signed and removed core-pending pending-signatures labels Jul 13, 2022

cmsbuild added tests-started and removed tests-rejected labels Jul 14, 2022

cmsbuild added tests-approved and removed tests-started labels Jul 14, 2022

cmsbuild added orp-approved and removed orp-pending labels Jul 14, 2022

cmsbuild merged commit 9e6c192 into cms-sw:master Jul 14, 2022

Dr15Jones deleted the morePrefetchSignals branch July 18, 2022 14:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add prefetching ActivityRegistry signals for more transitions #38708

Add prefetching ActivityRegistry signals for more transitions #38708

Dr15Jones commented Jul 12, 2022 •

edited

Loading

cmsbuild commented Jul 12, 2022

cmsbuild commented Jul 12, 2022

Dr15Jones commented Jul 12, 2022

cmsbuild commented Jul 12, 2022

cmsbuild commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

Dr15Jones commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

makortel commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

qliphy commented Jul 14, 2022

cmsbuild commented Jul 14, 2022

qliphy commented Jul 14, 2022

Add prefetching ActivityRegistry signals for more transitions #38708

Add prefetching ActivityRegistry signals for more transitions #38708

Conversation

Dr15Jones commented Jul 12, 2022 • edited Loading

PR description:

PR validation:

cmsbuild commented Jul 12, 2022

cmsbuild commented Jul 12, 2022

Dr15Jones commented Jul 12, 2022

cmsbuild commented Jul 12, 2022

Comparison Summary

cmsbuild commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

Dr15Jones commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

Unit Tests

Comparison Summary

makortel commented Jul 13, 2022

cmsbuild commented Jul 13, 2022

qliphy commented Jul 14, 2022

cmsbuild commented Jul 14, 2022

Comparison Summary

qliphy commented Jul 14, 2022

Dr15Jones commented Jul 12, 2022 •

edited

Loading