move sanitizer flags in to toolfile; drop sanitizer deps for rocm prod

[smuzaffar]

Move ASAN and UBSAN sanitizer flags in to toolfiles which are added in to dependency chain via cmssw/config/BuildFile.xml. Drop sanitizer tool deps from rocm products. Hopefully this can fix the ASAN/UBSAN build issues

[smuzaffar]

test parameters:

• full_cmssw = true

[cmsbuild]

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for branch IB/CMSSW_13_0_X/master.

@cmsbuild, @smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.
cms-bot commands are listed here

[smuzaffar]

please test for CMSSW_13_0_ASAN_X

[cmsbuild]

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cbfc28/30587/summary.html
COMMIT: ce2102e
CMSSW: CMSSW_13_0_ASAN_X_2023-02-08-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/8311/30587/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

• @makortel Allow ProcessAccelerators to delete Services cmssw#40736
• @bsunanda Run3-sim140 Add filldescription to one class in SimG4CMS/ShowerLibraryProducer cmssw#40719
• @bsunanda Phase2-gex149 Correct the order of Tracker files in some of the scenarios which is relevant for DD4hep application cmssw#40718
• @casarsa BTL geometry: fixed overlaps cmssw#40715
• @francescobrivio Add try/catch in OnlineBeamMonitor plugin to resolve runtime error cmssw#40716
• @nurfikri89 [QGL, JMENanoAOD] PUPPI-compatible QGTagger producer cmssw#40667
• @rovere Fix runTheMatrix interactive and add tests. cmssw#40596
• @CMSTrackerDPG The Lorentz Angle Prompt Calibration Loop for Pixel Forward Phase 1 detector  cmssw#40664
• @bundocka L1T: Add jet quality and eg/tau isolation data vs. emu comparisons and ignore object index cmssw#40511
• @aandvalenzuela [DQM][CLANG] Fix warnings reported by clang 14 cmssw#40721
• @bsunanda Run3-sim141 Update the scripts in SimG4Core/PrintGeomInfo/test/python cmssw#40726
• @fabferro PPS pixel track reco in bad pots cmssw#40683
• @trackreco [MkFit] Consolidate nan checks and filtering, expose propagation fail flag to steering code cmssw#40681

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cbfc28/30587/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cbfc28/30587/git-merge-result

Build

I found compilation error when building:

>> Leaving Package DataFormats/SoATemplate
>> Package DataFormats/SoATemplate built
>> Compiling  /data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_0_ASAN_X_2023-02-08-2300/src/DataFormats/SoATemplate/test/SoALayoutAndView_t.cu
>> Cuda Device Link tmp/el8_amd64_gcc11/src/DataFormats/SoATemplate/test/testCudaSoALayoutAndView_t/testCudaSoALayoutAndView_t_cudadlink.o 
>> Building binary testCudaSoALayoutAndView_t
ld.lld: error: undefined symbol: std::__cxx11::basic_ostringstream, std::allocator>::basic_ostringstream()
>>> referenced by tmpxft_000037ab_00000000-6_SoALayoutAndView_t.compute_75.cudafe1.cpp
>>>               tmp/el8_amd64_gcc11/src/DataFormats/SoATemplate/test/testCudaSoALayoutAndView_t/SoALayoutAndView_t.cu.o:(cms::cuda::abortOnCudaError(char const*, int, char const*, char const*, char const*, std::basic_string_view>))
>>> did you mean: std::__cxx11::basic_ostringstream, std::allocator>::~basic_ostringstream()
>>> defined in: /usr/lib/gcc/x86_64-redhat-linux/8/libstdc++.so



[smuzaffar]

please test for CMSSW_13_0_ASAN_X

[cmsbuild]

Pull request #8311 was updated.

[smuzaffar]

please test for CMSSW_13_1_ASAN_X

[cmsbuild]

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cbfc28/30592/summary.html
COMMIT: 118e4c6
CMSSW: CMSSW_13_1_ASAN_X_2023-02-11-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/8311/30592/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestFWCoreSkeletons had ERRORS
---> test testTriggerEventAnalyzers had ERRORS
---> test TestIOPoolInputNoParentDictionary had ERRORS
---> test test_MpsWorkFlow had ERRORS
and more ...

[smuzaffar]

please test

[smuzaffar]

please test for CMSSW_13_1_ASAN_X

[smuzaffar]

please test for CMSSW_13_1_UBSAN_X

[cmsbuild]

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cbfc28/30593/summary.html
COMMIT: 118e4c6
CMSSW: CMSSW_13_0_X_2023-02-11-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/8311/30593/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test GeometryTrackerGeometryBuilderTestDriver had ERRORS

Comparison Summary

Summary:

• You potentially removed 159 lines from the logs
• Reco comparison results: 4 differences found in the comparisons
• DQMHistoTests: Total files compared: 49
• DQMHistoTests: Total histograms compared: 3555972
• DQMHistoTests: Total failures: 0
• DQMHistoTests: Total nulls: 0
• DQMHistoTests: Total successes: 3555950
• DQMHistoTests: Total skipped: 22
• DQMHistoTests: Total Missing objects: 0
• DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
• Checked 213 log files, 164 edm output root files, 49 DQM output files
• TriggerResults: no differences found

[cmsbuild]

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cbfc28/30595/summary.html
COMMIT: 118e4c6
CMSSW: CMSSW_13_1_UBSAN_X_2023-02-11-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/8311/30595/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestSubProcess had ERRORS
---> test unitTestsGroup_1 had ERRORS
---> test unitTestsGroup_4 had ERRORS
---> test GeometryTrackerGeometryBuilderTestDriver had ERRORS
and more ...

[cmsbuild]

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-cbfc28/30594/summary.html
COMMIT: 118e4c6
CMSSW: CMSSW_13_1_ASAN_X_2023-02-11-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/8311/30594/install.sh to create a dev area with all the needed externals and cmssw changes.

Unit Tests

I found errors in the following unit tests:

---> test TestFWCoreSkeletons had ERRORS
---> test testTriggerEventAnalyzers had ERRORS
---> test TestIOPoolInputNoParentDictionary had ERRORS
---> test GCPall had ERRORS
and more ...

[fwyzard]

@smuzaffar I saw that these changes are only in 13.1.x and not in 13.0.x.
Is it OK to make a backport to 13.0.x, so we can later enable ROCm there as well ?

[cmsbuild]

backport done
Successfully backported PR #8311 as #8317 for branch IB/CMSSW_13_0_X/master

[smuzaffar]

@fwyzard , I have opened #8317 for 13.0.X

[fwyzard]

thank you !