Add a test for Alpaka libraries and build rules [14.0.x] #44635

fwyzard · 2024-04-05T14:30:25Z

PR description:

The package HeterogeneousTest/AlpakaDevice implements a header only library that defines Alpaka device-only functions, and a plugin and test that use them.

The package HeterogeneousTest/AlpakaKernel implements a header only library that imports device functions from HeterogeneousTest/AlpakaDevice to define Alpaka kernels, and a plugin and test that use them.

The package HeterogeneousTest/AlpakaWrapper implements a library that imports kernels from HeterogeneousTest/AlpakaKernel to define and export host-only wrappers around them, usable by non-Alpaka libraries, plugins and applications, and implements a plugin and test that use them.

The package HeterogeneousTest/AlpakaOpaque implements a library that imports kernels from HeterogeneousTest/AlpakaKernel to define and export host-only wrappers around the whole Alpaka section, usable by libraries, plugins and applications that are not explicitly Alpaka-aware, and implements a plugin and test that use them.

In addition, fix the CUDA and ROCm documentation, and various typos in the ROCm tests.

PR validation:

The new unit tests build and pass (on a machine with NVIDIA A10 GPU and an AMD Radeon Pro W7600):

Creating test log file logs/el8_amd64_gcc12/testing.log
Pass    0s ... HeterogeneousTest/AlpakaDevice/testAlpakaDeviceAdditionSerialSync
Pass    0s ... HeterogeneousTest/AlpakaOpaque/testAlpakaDeviceAdditionOpaqueSerialSync
Pass    0s ... HeterogeneousTest/AlpakaKernel/testAlpakaDeviceAdditionKernelSerialSync
Pass    0s ... HeterogeneousCore/ROCmUtilities/testHipCheck
Pass    0s ... HeterogeneousTest/AlpakaWrapper/testAlpakaDeviceAdditionWrapperSerialSync
Pass    0s ... HeterogeneousCore/ROCmUtilities/testRequireROCmDevices
Pass    0s ... HeterogeneousTest/ROCmOpaque/testRocmDeviceAdditionOpaque
Pass    0s ... HeterogeneousTest/AlpakaOpaque/testAlpakaDeviceAdditionOpaqueROCmAsync
Pass    0s ... HeterogeneousTest/ROCmWrapper/testRocmDeviceAdditionWrapper
Pass    0s ... HeterogeneousTest/AlpakaWrapper/testAlpakaDeviceAdditionWrapperROCmAsync
Pass    0s ... HeterogeneousTest/ROCmDevice/testRocmDeviceAddition
Pass    0s ... HeterogeneousTest/AlpakaKernel/testAlpakaDeviceAdditionKernelROCmAsync
Pass    0s ... HeterogeneousTest/AlpakaDevice/testAlpakaDeviceAdditionROCmAsync
Pass    0s ... HeterogeneousTest/ROCmKernel/testRocmDeviceAdditionKernel
Pass    1s ... HeterogeneousTest/ROCmOpaque/testROCmTestOpaqueAdditionModule
Pass    1s ... HeterogeneousTest/ROCmDevice/testROCmTestDeviceAdditionModule
Pass    1s ... HeterogeneousTest/ROCmOpaque/testROCmTestAdditionModules
Pass    1s ... HeterogeneousTest/ROCmKernel/testROCmTestKernelAdditionModule
Pass    1s ... HeterogeneousTest/ROCmWrapper/testROCmTestWrapperAdditionModule
Pass    4s ... HeterogeneousTest/AlpakaOpaque/testAlpakaDeviceAdditionOpaqueCudaAsync
Pass    4s ... HeterogeneousTest/CUDAWrapper/testCudaDeviceAdditionWrapper
Pass    4s ... HeterogeneousTest/CUDAKernel/testCudaDeviceAdditionKernel
Pass    4s ... HeterogeneousTest/AlpakaDevice/testAlpakaDeviceAdditionCudaAsync
Pass    5s ... HeterogeneousTest/AlpakaKernel/testAlpakaDeviceAdditionKernelCudaAsync
Pass    5s ... HeterogeneousTest/AlpakaWrapper/testAlpakaDeviceAdditionWrapperCudaAsync
Pass    5s ... HeterogeneousTest/CUDAOpaque/testCudaDeviceAdditionOpaque
Pass    5s ... HeterogeneousTest/CUDADevice/testCudaDeviceAddition
Pass    9s ... HeterogeneousTest/CUDAOpaque/testCUDATestOpaqueAdditionModule
Pass    9s ... HeterogeneousTest/CUDAWrapper/testCUDATestWrapperAdditionModule
Pass    9s ... HeterogeneousTest/CUDAKernel/testCUDATestKernelAdditionModule
Pass    9s ... HeterogeneousTest/CUDADevice/testCUDATestDeviceAdditionModule
Pass    9s ... HeterogeneousTest/AlpakaWrapper/testAlpakaTestWrapperAdditionModule
Pass    9s ... HeterogeneousTest/CUDAOpaque/testCUDATestAdditionModules
Pass    9s ... HeterogeneousTest/AlpakaKernel/testAlpakaTestKernelAdditionModule
Pass    9s ... HeterogeneousTest/AlpakaDevice/testAlpakaTestDeviceAdditionModule
Pass    9s ... HeterogeneousTest/AlpakaOpaque/testAlpakaTestOpaqueAdditionModule
Pass   10s ... HeterogeneousTest/AlpakaOpaque/testAlpakaTestAdditionModules
>> Test sequence completed for CMSSW CMSSW_14_0_4

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Backport of #44622 to CMSSW_14_0_X to ensure the Alpaka build rules are correct.

fwyzard · 2024-04-05T14:30:31Z

enable gpu

fwyzard · 2024-04-05T14:30:34Z

please test

fwyzard · 2024-04-05T14:30:38Z

backport #44622

cmsbuild · 2024-04-05T14:30:52Z

A new Pull Request was created by @fwyzard for CMSSW_14_0_X.

It involves the following packages:

HeterogeneousTest/AlpakaDevice (heterogeneous)
HeterogeneousTest/AlpakaKernel (heterogeneous)
HeterogeneousTest/AlpakaOpaque (heterogeneous)
HeterogeneousTest/AlpakaWrapper (heterogeneous)
HeterogeneousTest/CUDADevice (heterogeneous)
HeterogeneousTest/CUDAKernel (heterogeneous)
HeterogeneousTest/ROCmDevice (heterogeneous)
HeterogeneousTest/ROCmKernel (heterogeneous)
HeterogeneousTest/ROCmOpaque (heterogeneous)
HeterogeneousTest/ROCmWrapper (heterogeneous)

@makortel, @fwyzard can you please review it and eventually sign? Thanks.
@missirol this is something you requested to watch as well.
@sextonkennedy, @rappoccio, @antoniovilela you are the release manager for this.

cms-bot commands are listed here

Backported from Add a test for Alpaka libraries and build rules #44622

cmsbuild · 2024-04-05T14:30:53Z

cms-bot internal usage

cmsbuild · 2024-04-05T17:17:57Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-09fbd0/38632/summary.html
COMMIT: dc4faa2
CMSSW: CMSSW_14_0_X_2024-04-05-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44635/38632/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 86 lines to the logs
Reco comparison results: 47 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3346180
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3346155
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 205 log files, 166 edm output root files, 49 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 22 differences found in the comparisons
DQMHistoTests: Total files compared: 3
DQMHistoTests: Total histograms compared: 39740
DQMHistoTests: Total failures: 974
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 38766
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 2 files compared)
Checked 8 log files, 10 edm output root files, 3 DQM output files
TriggerResults: no differences found

The package HeterogeneousTest/AlpakaDevice implements a header only library that defines Alpaka device-only functions, and a plugin and test that use them. The package HeterogeneousTest/AlpakaKernel implements a header only library that imports device functions from HeterogeneousTest/AlpakaDevice to define Alpaka kernels, and a plugin and test that use them. The package HeterogeneousTest/AlpakaWrapper implements a library that imports kernels from HeterogeneousTest/AlpakaKernel to define and export host-only wrappers around them, usable by non-Alpaka libraries, plugins and applications, and implements a plugin and test that use them. The package HeterogeneousTest/AlpakaOpaque implements a library that imports kernels from HeterogeneousTest/AlpakaKernel to define and export host-only wrappers around the whole Alpaka section, usable by libraries, plugins and applications that are not explicitly Alpaka-aware, and implements a plugin and test that use them.

cmsbuild · 2024-04-05T20:37:06Z

Pull request #44635 was updated. @fwyzard, @makortel, @cmsbuild can you please check and sign again.

fwyzard · 2024-04-05T20:37:21Z

please test

fwyzard · 2024-04-05T20:45:04Z

+heterogeneous

cmsbuild · 2024-04-05T20:45:32Z

This pull request is fully signed and it will be integrated in one of the next CMSSW_14_0_X IBs after it passes the integration tests and once validation in the development release cycle CMSSW_14_1_X is complete. This pull request will now be reviewed by the release team before it's merged. @rappoccio, @sextonkennedy, @antoniovilela (and backports should be raised in the release meeting by the corresponding L2)

cmsbuild · 2024-04-05T23:25:04Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-09fbd0/38638/summary.html
COMMIT: ceaac24
CMSSW: CMSSW_14_0_X_2024-04-05-1100/el8_amd64_gcc12
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/44635/38638/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 135 lines to the logs
Reco comparison results: 49 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3346180
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3346155
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 205 log files, 166 edm output root files, 49 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 24 differences found in the comparisons
DQMHistoTests: Total files compared: 3
DQMHistoTests: Total histograms compared: 39740
DQMHistoTests: Total failures: 1267
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 38473
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 2 files compared)
Checked 8 log files, 10 edm output root files, 3 DQM output files
TriggerResults: no differences found

antoniovilela · 2024-04-09T14:58:02Z

+1

cmsbuild added this to the CMSSW_14_0_X milestone Apr 5, 2024

cmsbuild added pending-signatures orp-pending tests-started backport heterogeneous-pending labels Apr 5, 2024

fwyzard mentioned this pull request Apr 5, 2024

Add a test for Alpaka libraries and build rules #44622

Merged

cmsbuild added tests-approved and removed tests-started labels Apr 5, 2024

fwyzard added 5 commits April 5, 2024 22:34

Fix names in the ROCm packages documentation

05f15bf

Fix typos in the ROCm tests

15e21be

Fix line wrap

c8ec63c

Update "private" namespace names to match package names

f080146

fwyzard force-pushed the implement_HeterogeneousTest_Alpaka_140x branch from dc4faa2 to ceaac24 Compare April 5, 2024 20:36

cmsbuild added tests-pending and removed tests-approved labels Apr 5, 2024

cmsbuild added tests-started and removed tests-pending labels Apr 5, 2024

cmsbuild removed pending-signatures heterogeneous-pending labels Apr 5, 2024

cmsbuild added fully-signed heterogeneous-approved labels Apr 5, 2024

cmsbuild added tests-approved and removed tests-started labels Apr 5, 2024

fwyzard changed the title ~~Add a test for Alpaka libraries and build rules (14.0.x)~~ Add a test for Alpaka libraries and build rules [14.0.x] Apr 7, 2024

cmsbuild added backport-ok and removed backport labels Apr 8, 2024

cmsbuild added orp-approved and removed orp-pending labels Apr 9, 2024

cmsbuild merged commit f173701 into cms-sw:CMSSW_14_0_X Apr 9, 2024
13 checks passed

fwyzard deleted the implement_HeterogeneousTest_Alpaka_140x branch April 9, 2024 14:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a test for Alpaka libraries and build rules [14.0.x] #44635

Add a test for Alpaka libraries and build rules [14.0.x] #44635

fwyzard commented Apr 5, 2024 •

edited

Loading

fwyzard commented Apr 5, 2024

fwyzard commented Apr 5, 2024

fwyzard commented Apr 5, 2024

cmsbuild commented Apr 5, 2024 •

edited

Loading

cmsbuild commented Apr 5, 2024 •

edited

Loading

cmsbuild commented Apr 5, 2024

cmsbuild commented Apr 5, 2024

fwyzard commented Apr 5, 2024

fwyzard commented Apr 5, 2024

cmsbuild commented Apr 5, 2024

cmsbuild commented Apr 5, 2024

antoniovilela commented Apr 9, 2024

Add a test for Alpaka libraries and build rules [14.0.x] #44635

Add a test for Alpaka libraries and build rules [14.0.x] #44635

Conversation

fwyzard commented Apr 5, 2024 • edited Loading

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

fwyzard commented Apr 5, 2024

fwyzard commented Apr 5, 2024

fwyzard commented Apr 5, 2024

cmsbuild commented Apr 5, 2024 • edited Loading

cmsbuild commented Apr 5, 2024 • edited Loading

cmsbuild commented Apr 5, 2024

Comparison Summary

GPU Comparison Summary

cmsbuild commented Apr 5, 2024

fwyzard commented Apr 5, 2024

fwyzard commented Apr 5, 2024

cmsbuild commented Apr 5, 2024

cmsbuild commented Apr 5, 2024

Comparison Summary

GPU Comparison Summary

antoniovilela commented Apr 9, 2024

fwyzard commented Apr 5, 2024 •

edited

Loading

cmsbuild commented Apr 5, 2024 •

edited

Loading

cmsbuild commented Apr 5, 2024 •

edited

Loading