[BuildRules] First implementation of RocM build rules #8268

smuzaffar · 2023-01-27T14:31:21Z

No description provided.

cmsbuild · 2023-01-27T14:31:46Z

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for branch IB/CMSSW_13_0_X/master.

@cmsbuild, @smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.
cms-bot commands are listed here

smuzaffar · 2023-01-27T14:32:04Z

please test

cmsbuild · 2023-01-27T20:24:52Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3c42a7/30205/summary.html
COMMIT: c5e3b19
CMSSW: CMSSW_13_0_X_2023-01-27-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/8268/30205/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@tahuang1991 Add new ALCT-CLCT match algorithm for high pt muon showers cmssw#40572

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3c42a7/30205/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3c42a7/30205/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test test-das-selected-lumis had ERRORS
---> test test_edmPickEvents had ERRORS

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3555495
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3555470
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 211 log files, 162 edm output root files, 49 DQM output files
TriggerResults: no differences found

smuzaffar · 2023-01-27T22:15:41Z

please test

cmsbuild · 2023-01-27T22:15:49Z

Pull request #8268 was updated.

cmsbuild · 2023-01-28T04:10:07Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3c42a7/30212/summary.html
COMMIT: 5be4026
CMSSW: CMSSW_13_0_X_2023-01-27-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/8268/30212/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3c42a7/30212/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3c42a7/30212/git-merge-result

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3555495
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3555470
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 211 log files, 162 edm output root files, 49 DQM output files
TriggerResults: no differences found

cmsbuild · 2023-01-28T09:42:35Z

Pull request #8268 was updated.

smuzaffar · 2023-01-28T09:42:42Z

please test with cms-sw/cmssw#40619

cmsbuild · 2023-01-28T10:05:10Z

Pull request #8268 was updated.

cmsbuild · 2023-01-28T16:30:03Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3c42a7/30221/summary.html
COMMIT: 6dbb4e1
CMSSW: CMSSW_13_0_X_2023-01-27-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/8268/30221/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 4 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3555495
DQMHistoTests: Total failures: 4
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3555469
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 211 log files, 162 edm output root files, 49 DQM output files
TriggerResults: no differences found

fwyzard · 2023-01-28T16:53:39Z

👍🏻

the ROCm tests compiled and ran, as much as they can without a GPU:

===== Test "testHipCheck" ====
===============================================================================
All tests passed (2 assertions in 1 test case)

---> test testHipCheck succeeded
TestTime:4
^^^^ End Test testHipCheck ^^^^

and

===== Test "testRequireROCmDevices" ====
Failed to initialise the ROCm runtime, the test will be skipped.

---> test testRequireROCmDevices succeeded
TestTime:4
^^^^ End Test testRequireROCmDevices ^^^^

cmsbuild · 2023-01-28T21:47:40Z

Pull request #8268 was updated.

smuzaffar · 2023-01-28T21:48:00Z

please test with cms-sw/cmssw#40619

cmsbuild · 2023-01-29T04:11:58Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-3c42a7/30228/summary.html
COMMIT: 41ddb86
CMSSW: CMSSW_13_0_X_2023-01-28-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmsdist/8268/30228/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 13 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3555495
DQMHistoTests: Total failures: 9
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3555464
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 211 log files, 162 edm output root files, 49 DQM output files
TriggerResults: no differences found

smuzaffar · 2023-01-29T10:37:49Z

+externals
build rules looks good

cmsbuild · 2023-01-29T10:38:11Z

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_13_0_X/master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

fwyzard · 2023-01-29T19:53:18Z

@smuzaffar I'm trying to build some simple HIP/ROCm device code, and I noticed that when hipcc compiled the *.hip.cc files, the ROCM_FLAGS are missing:

$ scram tool info rocm | grep ROCM_FLAGS
ROCM_FLAGS+=-fgpu-rdc --offload-arch=gfx900 --target=x86_64-redhat-linux-gnu --gcc-toolchain=$(COMPILER_PATH)

Do we need to add ROCM_FLAGS (and maybe ROCM_LDFLAGS) to .SCRAM/el8_amd64_gcc11/MakeData/Tools/self.mk and .SCRAM/el8_amd64_gcc11/MakeData/variables.mk ?

smuzaffar · 2023-01-29T20:38:40Z

@fwyzard , #8271 should fix this issue. We just need to tell scram to treat these ROCM flags as valid compiler flags

[BuildRules] First implementation of RocM build rules

c5e3b19

cmsbuild added externals-pending orp-pending pending-signatures tests-pending labels Jan 27, 2023

cmsbuild added tests-started and removed tests-pending labels Jan 27, 2023

cmsbuild added tests-rejected and removed tests-started labels Jan 27, 2023

Update scram-project-build.file

5be4026

cmsbuild added tests-started and removed tests-rejected labels Jan 27, 2023

cmsbuild added tests-approved and removed tests-started labels Jan 28, 2023

use llvm compiler flags for for rocm

4242bb0

cmsbuild added tests-pending and removed tests-approved labels Jan 28, 2023

cmsbuild added requires-external tests-started and removed tests-pending labels Jan 28, 2023

rocm rules use llvm flags so no need to duplicate these REM_CXXFLAGS

182825d

cmsbuild removed the tests-started label Jan 28, 2023

cmsbuild added tests-approved and removed tests-started labels Jan 28, 2023

hip/hsa includes are also available via main include path

41ddb86

cmsbuild added tests-pending and removed tests-approved requires-external labels Jan 28, 2023

cmsbuild added requires-external tests-started and removed tests-pending labels Jan 28, 2023

cmsbuild added tests-approved and removed tests-started labels Jan 29, 2023

cmsbuild added externals-approved fully-signed and removed externals-pending pending-signatures labels Jan 29, 2023

smuzaffar merged commit c0f5018 into IB/CMSSW_13_0_X/master Jan 29, 2023

smuzaffar mentioned this pull request Jan 29, 2023

Add ROCm utilities and tests, and CUDA fallbacks cms-sw/cmssw#40619

Merged

fwyzard mentioned this pull request Jan 29, 2023

Implement HIP/ROCm build rules and test packages cms-sw/cmssw#40627

Closed

cmsbuild mentioned this pull request Jan 31, 2023

[ROOT626] Updated root to tip of branch v6-26-00-patches #8277

Merged

smuzaffar deleted the smuzaffar-patch-8 branch February 15, 2023 14:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BuildRules] First implementation of RocM build rules #8268

[BuildRules] First implementation of RocM build rules #8268

smuzaffar commented Jan 27, 2023

cmsbuild commented Jan 27, 2023

smuzaffar commented Jan 27, 2023

cmsbuild commented Jan 27, 2023

smuzaffar commented Jan 27, 2023

cmsbuild commented Jan 27, 2023

cmsbuild commented Jan 28, 2023

cmsbuild commented Jan 28, 2023

smuzaffar commented Jan 28, 2023

cmsbuild commented Jan 28, 2023

cmsbuild commented Jan 28, 2023

fwyzard commented Jan 28, 2023

cmsbuild commented Jan 28, 2023

smuzaffar commented Jan 28, 2023

cmsbuild commented Jan 29, 2023

smuzaffar commented Jan 29, 2023

cmsbuild commented Jan 29, 2023

fwyzard commented Jan 29, 2023

smuzaffar commented Jan 29, 2023

[BuildRules] First implementation of RocM build rules #8268

[BuildRules] First implementation of RocM build rules #8268

Conversation

smuzaffar commented Jan 27, 2023

cmsbuild commented Jan 27, 2023

smuzaffar commented Jan 27, 2023

cmsbuild commented Jan 27, 2023

Unit Tests

Comparison Summary

smuzaffar commented Jan 27, 2023

cmsbuild commented Jan 27, 2023

cmsbuild commented Jan 28, 2023

Comparison Summary

cmsbuild commented Jan 28, 2023

smuzaffar commented Jan 28, 2023

cmsbuild commented Jan 28, 2023

cmsbuild commented Jan 28, 2023

Comparison Summary

fwyzard commented Jan 28, 2023

cmsbuild commented Jan 28, 2023

smuzaffar commented Jan 28, 2023

cmsbuild commented Jan 29, 2023

Comparison Summary

smuzaffar commented Jan 29, 2023

cmsbuild commented Jan 29, 2023

fwyzard commented Jan 29, 2023

smuzaffar commented Jan 29, 2023