Improve behavior after exception in begin/end run transitions #45017
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
cms-bot internal usage |
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45017/40305
|
|
A new Pull Request was created by @wddgit for master. It involves the following packages:
@cmsbuild, @Dr15Jones, @makortel, @smuzaffar can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
|
please test |
|
-1 Failed Tests: Build BuildI found compilation error when building: >> Compiling src/Mixing/Base/src/PileupRandomNumberGenerator.cc
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/c++ -c -DGNU_GCC -D_GNU_SOURCE -DTBB_USE_GLIBCXX_VERSION=120301 -DTBB_SUPPRESS_DEPRECATED_MESSAGES -DTBB_PREVIEW_RESUMABLE_TASKS=1 -DTBB_PREVIEW_TASK_GROUP_EXTENSIONS=1 -DBOOST_SPIRIT_THREADSAFE -DPHOENIX_THREADSAFE -DBOOST_MATH_DISABLE_STD_FPCLASSIFY -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DCMSSW_GIT_HASH='CMSSW_14_1_X_2024-05-22-1100' -DPROJECT_NAME='CMSSW' -DPROJECT_VERSION='CMSSW_14_1_X_2024-05-22-1100' -Isrc -Ipoison -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-05-22-1100/src -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/cms/coral/CORAL_2_3_21-27ab7e52f21297bcbeaa636ca097acc7/include/LCG -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/pcre/8.43-e34796d17981e9b6d174328c69446455/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/boost/1.80.0-941b136a4a3be6f8bc1e903d36ddc172/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/bz2lib/1.0.6-d065ccd79984efc6d4660f410e4c81de/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/clhep/2.4.7.1-8e40efd27b7394c1fa4e9c7e432d85cd/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/curl/7.79.0-e9aea8dd47e409f0dcfd76a7b3220112/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/gsl/2.6-5e2ce72ea2977ff21a2344bbb52daf5c/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/libuuid/2.34-27ce4c3579b5b1de2808ea9c4cd8ed29/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/lcg/root/6.30.07-f3322c77db1c59847b28fde88ff7218c/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/tbb/v2021.9.0-a7089dd5ec356e9a0bc222e109b15cef/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/xerces-c/3.1.3-c7b88eaa36d0408120f3c29826a04bf6/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/xz/5.2.5-6f3f49b07db84e10c9be594a1176c114/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/zlib/1.2.11-1a082fc322b0051b504cc023f21df178/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/fmt/8.0.1-258b4791803c34b7e98cf43693e54d87/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/md5/1.0.0-5b594b264e04ae51e893b1d69a797ec6/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/OpenBLAS/0.3.15-c877ab57fa7b04ce290093588c6c5717/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/tinyxml2/6.2.0-88fe0ec301baf763fa3c485e5b67ed91/include -O2 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++17 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v2 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=missing-braces -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-error=unused-variable -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -fPIC -MMD -MF tmp/el8_amd64_gcc12/src/Mixing/Base/src/MixingBase/PileupRandomNumberGenerator.cc.d src/Mixing/Base/src/PileupRandomNumberGenerator.cc -o tmp/el8_amd64_gcc12/src/Mixing/Base/src/MixingBase/PileupRandomNumberGenerator.cc.o
>> Compiling src/Mixing/Base/src/SecondaryEventProvider.cc
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/gcc/12.3.1-40d504be6370b5a30e3947a6e575ca28/bin/c++ -c -DGNU_GCC -D_GNU_SOURCE -DTBB_USE_GLIBCXX_VERSION=120301 -DTBB_SUPPRESS_DEPRECATED_MESSAGES -DTBB_PREVIEW_RESUMABLE_TASKS=1 -DTBB_PREVIEW_TASK_GROUP_EXTENSIONS=1 -DBOOST_SPIRIT_THREADSAFE -DPHOENIX_THREADSAFE -DBOOST_MATH_DISABLE_STD_FPCLASSIFY -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DCMSSW_GIT_HASH='CMSSW_14_1_X_2024-05-22-1100' -DPROJECT_NAME='CMSSW' -DPROJECT_VERSION='CMSSW_14_1_X_2024-05-22-1100' -Isrc -Ipoison -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-05-22-1100/src -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/cms/coral/CORAL_2_3_21-27ab7e52f21297bcbeaa636ca097acc7/include/LCG -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/pcre/8.43-e34796d17981e9b6d174328c69446455/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/boost/1.80.0-941b136a4a3be6f8bc1e903d36ddc172/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/bz2lib/1.0.6-d065ccd79984efc6d4660f410e4c81de/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/clhep/2.4.7.1-8e40efd27b7394c1fa4e9c7e432d85cd/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/curl/7.79.0-e9aea8dd47e409f0dcfd76a7b3220112/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/gsl/2.6-5e2ce72ea2977ff21a2344bbb52daf5c/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/libuuid/2.34-27ce4c3579b5b1de2808ea9c4cd8ed29/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/lcg/root/6.30.07-f3322c77db1c59847b28fde88ff7218c/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/tbb/v2021.9.0-a7089dd5ec356e9a0bc222e109b15cef/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/xerces-c/3.1.3-c7b88eaa36d0408120f3c29826a04bf6/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/xz/5.2.5-6f3f49b07db84e10c9be594a1176c114/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/zlib/1.2.11-1a082fc322b0051b504cc023f21df178/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/fmt/8.0.1-258b4791803c34b7e98cf43693e54d87/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/md5/1.0.0-5b594b264e04ae51e893b1d69a797ec6/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/OpenBLAS/0.3.15-c877ab57fa7b04ce290093588c6c5717/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02838/el8_amd64_gcc12/external/tinyxml2/6.2.0-88fe0ec301baf763fa3c485e5b67ed91/include -O2 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++17 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v2 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=missing-braces -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-error=unused-variable -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -fPIC -MMD -MF tmp/el8_amd64_gcc12/src/Mixing/Base/src/MixingBase/SecondaryEventProvider.cc.d src/Mixing/Base/src/SecondaryEventProvider.cc -o tmp/el8_amd64_gcc12/src/Mixing/Base/src/MixingBase/SecondaryEventProvider.cc.o
src/Mixing/Base/src/SecondaryEventProvider.cc: In function 'void {anonymous}::processOneOccurrence(edm::WorkerManager&, typename T::TransitionInfoType&, edm::StreamID, const typename T::Context*, const U*, bool)':
src/Mixing/Base/src/SecondaryEventProvider.cc:40:16: error: 'addContextAndPrintException' is not a member of 'edm'
40 | edm::addContextAndPrintException("Calling SecondaryEventProvider", ex, cleaningUpAfterException);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
src/Mixing/Base/src/SecondaryEventProvider.cc:42:16: error: 'addContextAndPrintException' is not a member of 'edm'
42 | edm::addContextAndPrintException("", ex, cleaningUpAfterException);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
|
wddgit
force-pushed
the
exceptionBehaviorBeginEndRun
branch
from
44b20e0 to
66d135d
Compare
cmsbuild
added
simulation-pending
tests-pending
code-checks-pending
and removed
tests-rejected
code-checks-approved
labels
|
enable threading |
|
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-45017/40308
|
|
Pull request #45017 was updated. @makortel, @cmsbuild, @smuzaffar, @civanch, @mdhildreth, @Dr15Jones can you please check and sign again. |
|
please test |
|
Pull request #45017 was updated. @smuzaffar, @makortel, @cmsbuild, @civanch, @Dr15Jones, @mdhildreth can you please check and sign again. |
|
please test Includes the added comment that was requested. Squashed commits. This should resolve all the comments and questions received so far. |
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-2909e4/39609/summary.html Comparison SummarySummary:
|
|
+1 |
|
+core |
cmsbuild
added
core-approved
fully-signed
and removed
core-pending
pending-signatures
labels
|
This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @sextonkennedy, @antoniovilela, @rappoccio (and backports should be raised in the release meeting by the corresponding L2) |
|
+1 |
This was referenced Jun 4, 2024
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR description:
Improve the behavior of the Framework after exceptions in Run begin/end stream/global transitions. This is the third in a series of PRs where we plan to make the behavior after exceptions more consistent in all the begin/end transitions. The first PR handled stream begin/end lumi exceptions (PR #44624). The second PR handled global begin/end lumi exceptions (PR #44840). The comments at the head of the first PR state the design for this behavior we are implementing.
The intent is that nothing in the output will change if there are not any exceptions. The order of modules in begin/end stream transitions may change, although in existing releases these functions are run asynchronously and the order can vary if an identical configuration is repeated in multi-threaded jobs. It is a problem already if something depends on that order.
Another minor change in behavior is that signals pre/post for beginStream and endStream will no longer be issued for trigger results inserter, path status inserters, and end path status inserters. These modules don't do anything in those transitions. I examined the services watching those signals and cannot see any reason for those signals to be emitted.
This work was motivated by discussions related to Issues #43831 and #42501.
The most complicated detail in this PR is that instead of each StreamSchedule having one WorkerManager, each StreamSchedule will have 3, one for lumis/events, one for runs, and one for beginStream/endStream. Although I intend
to address the beginStream/endStream transitions in the next PR, I went ahead and converted them to use a different WorkerManager because I didn't want to have to modify that complex part of the code twice.
PR validation:
An existing unit test covering exceptions in different transitions is extended to cover the most salient cases. Additional manual testing of many various cases was also done. Existing unit tests pass.