Removes duplicate/unneeded entries in serialization XML of test objects. #44219

ericcano · 2024-02-27T12:28:47Z

PR description:

This PR simplifies the XML for tests objects.

PR validation:

Unit tests plus writer.pl and reader.pl from HeterogeneousCore/AlpakaTest/test/ ran successfully.

cmsbuild · 2024-02-27T12:29:09Z

cms-bot internal usage

cmsbuild · 2024-02-27T13:59:19Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44219/39204

This PR adds an extra 12KB to repository

cmsbuild · 2024-02-27T13:59:41Z

A new Pull Request was created by @ericcano for master.

It involves the following packages:

DataFormats/PortableTestObjects (heterogeneous)

@cmsbuild, @makortel, @fwyzard can you please review it and eventually sign? Thanks.
@rovere, @missirol this is something you requested to watch as well.
@antoniovilela, @rappoccio, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

makortel · 2024-02-27T17:33:12Z

DataFormats/PortableTestObjects/src/classes_def.xml

@@ -1,24 +1,16 @@
 <lcgdict>
  <class name="portabletest::TestSoA"/>


Isn't TestSoA a duplicate too (being an alias to portabletest::TestSoALayout<128,false>)?

Maybe move all the TestSoALayout<...> at the beginning of the file?

Actually I would prefer to keep the aliases and remove the template specialisations.

I agree using the alias would be fine. Unfortunately, I re-tested this and it only works up to 2 layouts, but with the 3 layouts version, the dictionary generation crashes:

/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/lcg/root/6.30.05-00c2425f5a7c62d1e0d7747537277755/bin/rootcling -reflex -f tmp/el8_amd64_gcc12/src/DataFormats/PortableTestObjects/src/DataFormatsPortableTestObjects/a/DataFormatsPortableTestObjects_xr.cc -inlineInputHeader -failOnWarnings -rmf tmp/el8_amd64_gcc12/src/DataFormats/PortableTestObjects/src/DataFormatsPortableTestObjects/a/DataFormatsPortableTestObjects_xr.rootmap -rml libDataFormatsPortableTestObjects.so -DALPAKA_DEFAULT_HOST_MEMORY_ALIGNMENT=128 -DALPAKA_DISABLE_VENDOR_RNG -DCMS_DICT_IMPL -D_REENTRANT -DGNUSOURCE -D__STRICT_ANSI__ -DGNU_GCC -D_GNU_SOURCE -DEIGEN_DONT_PARALLELIZE -DEIGEN_MAX_ALIGN_BYTES=64 -DTBB_USE_GLIBCXX_VERSION=120301 -DTBB_SUPPRESS_DEPRECATED_MESSAGES -DTBB_PREVIEW_RESUMABLE_TASKS=1 -DTBB_PREVIEW_TASK_GROUP_EXTENSIONS=1 -DBOOST_SPIRIT_THREADSAFE -DPHOENIX_THREADSAFE -DBOOST_MATH_DISABLE_STD_FPCLASSIFY -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DCMSSW_GIT_HASH="CMSSW_14_1_X_2024-03-11-1100" -DPROJECT_NAME="CMSSW" -DPROJECT_VERSION="CMSSW_14_1_X_2024-03-11-1100" -I/data/user/cano/ZVertexMultiLayout/CMSSW_14_1_X_2024-03-11-1100/src -I/data/user/cano/ZVertexMultiLayout/CMSSW_14_1_X_2024-03-11-1100/poison -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/cms/cmssw-patch/CMSSW_14_1_X_2024-03-11-1100/src -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/external/alpaka/1.1.0-c6af69ddd6f2ee5be4f2b069590bae19/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/external/pcre/8.43-37eb2e8b73bab83d6645ecfd5d73dcaa/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/external/boost/1.80.0-fe2842446e7b8c2042e86925ee67be29/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/external/bz2lib/1.0.6-d065ccd79984efc6d4660f410e4c81de/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/external/libuuid/2.34-27ce4c3579b5b1de2808ea9c4cd8ed29/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/lcg/root/6.30.05-00c2425f5a7c62d1e0d7747537277755/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/external/tbb/v2021.9.0-c3903c50b52342174dbd3a52854a6e6d/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/external/xz/5.2.5-6f3f49b07db84e10c9be594a1176c114/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/external/zlib/1.2.11-51072030b7f93c3ac6c4235f21e413cb/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/external/eigen/3bb6a48d8c171cf20b5f8e48bfb4e424fbd4f79e-cf66836cbd6ad8d696d804181e36d61c/include/eigen3 -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/external/fmt/8.0.1-54e94b39f5cf29341bb9c4765764e1ca/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/external/md5/1.0.0-5b594b264e04ae51e893b1d69a797ec6/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/external/tinyxml2/6.2.0-d17873b4d6a42a43226cf689f82ec1ef/include -DCMSSW_REFLEX_DICT src/DataFormats/PortableTestObjects/src/classes.h src/DataFormats/PortableTestObjects/src/classes_def.xml #0 0x00007f21f0f7dc44 PrintStackTraceSignalHandler(void*) Signals.cpp:0:0 #1 0x00007f21f0f7b45b SignalHandler(int) Signals.cpp:0:0 #2 0x00007f21ec94ccf0 __restore_rt (/lib64/libpthread.so.0+0x12cf0) #3 0x00007f21ec7baaec TProtoClass::Delete(char const*) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/lcg/root/6.30.05-00c2425f5a7c62d1e0d7747537277755/bin/../lib/libCore.so+0x32caec) #4 0x00007f21ec7babf7 TProtoClass::~TProtoClass() (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/lcg/root/6.30.05-00c2425f5a7c62d1e0d7747537277755/bin/../lib/libCore.so+0x32cbf7) #5 0x00007f21ec7bacc9 TProtoClass::~TProtoClass() (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/lcg/root/6.30.05-00c2425f5a7c62d1e0d7747537277755/bin/../lib/libCore.so+0x32ccc9) #6 0x00007f21ec75129c TObjArray::Delete(char const*) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/lcg/root/6.30.05-00c2425f5a7c62d1e0d7747537277755/bin/../lib/libCore.so+0x2c329c) #7 0x00007f21f243074e CloseStreamerInfoROOTFile (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/lcg/root/6.30.05-00c2425f5a7c62d1e0d7747537277755/bin/../lib/libRIO.so+0xca74e) #8 0x00007f21ed594344 FinalizeStreamerInfoWriting(cling::Interpreter&, bool) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/lcg/root/6.30.05-00c2425f5a7c62d1e0d7747537277755/bin/../lib/libCling.so+0xa3a344) #9 0x00007f21ed5a7f24 GenerateFullDict(std::ostream&, cling::Interpreter&, RScanner&, std::__cxx11::list<ROOT::TMetaUtils::RConstructorType, std::allocator<ROOT::TMetaUtils::RConstructorType>> const&, bool, bool, bool, bool) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/lcg/root/6.30.05-00c2425f5a7c62d1e0d7747537277755/bin/../lib/libCling.so+0xa4df24) #10 0x00007f21ed5b0785 RootClingMain(int, char**, bool) (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/lcg/root/6.30.05-00c2425f5a7c62d1e0d7747537277755/bin/../lib/libCling.so+0xa56785) #11 0x00007f21ed5b407c ROOT_rootcling_Driver (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/lcg/root/6.30.05-00c2425f5a7c62d1e0d7747537277755/bin/../lib/libCling.so+0xa5a07c) #12 0x000000000040115e main (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/lcg/root/6.30.05-00c2425f5a7c62d1e0d7747537277755/bin/rootcling+0x40115e) #13 0x00007f21ebb5fd85 __libc_start_main (/lib64/libc.so.6+0x3ad85) #14 0x00000000004011ae _start (/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02828/el8_amd64_gcc12/lcg/root/6.30.05-00c2425f5a7c62d1e0d7747537277755/bin/rootcling+0x4011ae) PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace.

Mixing the alias and developed versions of the template works fine (i.e. with writer.pl - reader.pl), but I prefer using just the developed version to avoid confusion (this is not too cumbersome when using the helper script DataFormats/Portable/scripts/portableHostCollectionHints to generate the XML).

@makortel: indeed this is duplicate. All is fine without this line (pushed).

Unfortunately, I re-tested this and it only works up to 2 layouts, but with the 3 layouts version, the dictionary generation crashes:

This crash might be worth of further investigation (in a form of a new CMSSW issue with instructions how to reproduce, where we'd involve ROOT folks).

Sure, done in #44386.

cmsbuild · 2024-03-12T10:13:34Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-44219/39430

This PR adds an extra 16KB to repository

cmsbuild · 2024-03-12T10:13:52Z

Pull request #44219 was updated. @makortel, @cmsbuild, @fwyzard can you please check and sign again.

fwyzard · 2024-03-21T22:49:57Z

please test

cmsbuild · 2024-03-22T01:46:40Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8ac2b0/38336/summary.html
COMMIT: 4f779de
CMSSW: CMSSW_14_1_X_2024-03-21-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44219/38336/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

The workflows 1001.0, 1000.0 have different files in step1_dasquery.log than the ones found in the baseline. You may want to check and retrigger the tests if necessary. You can check it in the "files" directory in the results of the comparisons

Summary:

You potentially removed 9 lines from the logs
Reco comparison results: 50 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3297369
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3297343
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 202 log files, 165 edm output root files, 48 DQM output files
TriggerResults: no differences found

fwyzard · 2024-06-20T07:45:17Z

please test

fwyzard · 2024-06-20T07:45:21Z

+heterogeneous

cmsbuild · 2024-06-20T07:45:42Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @antoniovilela, @rappoccio, @sextonkennedy (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild · 2024-06-20T10:48:42Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-8ac2b0/39984/summary.html
COMMIT: 4f779de
CMSSW: CMSSW_14_1_X_2024-06-19-2300/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/44219/39984/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 1 lines to the logs
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 48
DQMHistoTests: Total histograms compared: 3345018
DQMHistoTests: Total failures: 0
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3344998
DQMHistoTests: Total skipped: 20
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 47 files compared)
Checked 202 log files, 165 edm output root files, 48 DQM output files
TriggerResults: no differences found

antoniovilela · 2024-06-21T12:37:13Z

+1

cmsbuild added this to the CMSSW_14_1_X milestone Feb 27, 2024

cmsbuild added pending-signatures tests-pending orp-pending code-checks-pending heterogeneous-pending labels Feb 27, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Feb 27, 2024

makortel reviewed Feb 27, 2024

View reviewed changes

cmsbuild mentioned this pull request Feb 27, 2024

Add missing newlines in DataFormats modules #44230

Merged

ericcano added 2 commits March 11, 2024 12:36

Removes duplicate/unneeded entries in serialization XML of test objects.

cacab0a

Removes redundant class declration for dictionary generation.

4f779de

ericcano force-pushed the Serialization_XML_simplification branch from 3117516 to 4f779de Compare March 12, 2024 10:08

cmsbuild added code-checks-pending and removed code-checks-approved labels Mar 12, 2024

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 12, 2024

cmsbuild added tests-started and removed tests-pending labels Mar 21, 2024

cmsbuild added tests-approved and removed tests-started labels Mar 22, 2024

cmsbuild added fully-signed heterogeneous-approved and removed pending-signatures heterogeneous-pending labels Jun 20, 2024

cmsbuild added tests-started and removed tests-approved labels Jun 20, 2024

cmsbuild added tests-approved and removed tests-started labels Jun 20, 2024

cmsbuild added orp-approved and removed orp-pending labels Jun 21, 2024

cmsbuild merged commit c5431dc into cms-sw:master Jun 21, 2024
11 checks passed

cmsbuild mentioned this pull request Jun 21, 2024

Heterogeneous Workflow (CLUE) in Phase2 HLT Simplified Menu #45178

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Removes duplicate/unneeded entries in serialization XML of test objects. #44219

Removes duplicate/unneeded entries in serialization XML of test objects. #44219

ericcano commented Feb 27, 2024

cmsbuild commented Feb 27, 2024 •

edited

Loading

cmsbuild commented Feb 27, 2024

cmsbuild commented Feb 27, 2024

makortel Feb 27, 2024

fwyzard Feb 27, 2024

ericcano Mar 12, 2024

ericcano Mar 12, 2024

makortel Mar 12, 2024

ericcano Mar 13, 2024

cmsbuild commented Mar 12, 2024

cmsbuild commented Mar 12, 2024

fwyzard commented Mar 21, 2024

cmsbuild commented Mar 22, 2024

fwyzard commented Jun 20, 2024

fwyzard commented Jun 20, 2024

cmsbuild commented Jun 20, 2024

cmsbuild commented Jun 20, 2024

antoniovilela commented Jun 21, 2024

		@@ -1,24 +1,16 @@
		<lcgdict>
		<class name="portabletest::TestSoA"/>

Removes duplicate/unneeded entries in serialization XML of test objects. #44219

Removes duplicate/unneeded entries in serialization XML of test objects. #44219

Conversation

ericcano commented Feb 27, 2024

PR description:

PR validation:

cmsbuild commented Feb 27, 2024 • edited Loading

cmsbuild commented Feb 27, 2024

cmsbuild commented Feb 27, 2024

makortel Feb 27, 2024

Choose a reason for hiding this comment

fwyzard Feb 27, 2024

Choose a reason for hiding this comment

ericcano Mar 12, 2024

Choose a reason for hiding this comment

ericcano Mar 12, 2024

Choose a reason for hiding this comment

makortel Mar 12, 2024

Choose a reason for hiding this comment

ericcano Mar 13, 2024

Choose a reason for hiding this comment

cmsbuild commented Mar 12, 2024

cmsbuild commented Mar 12, 2024

fwyzard commented Mar 21, 2024

cmsbuild commented Mar 22, 2024

Comparison Summary

fwyzard commented Jun 20, 2024

fwyzard commented Jun 20, 2024

cmsbuild commented Jun 20, 2024

cmsbuild commented Jun 20, 2024

Comparison Summary

antoniovilela commented Jun 21, 2024

cmsbuild commented Feb 27, 2024 •

edited

Loading