Pixel Alpaka Migration: `AlpakaInterface` Updates [II] #41284

AdrianoDee · 2023-04-05T15:48:09Z

PR description:

This PR stems from #41117 and it's the 2nd of a series of smaller PRs.

This is It includes all the needed additions to HeterogenousCore. It creates a new sub-package HeterogeneousCore/AlpakaUtilities that includes all the needed utilities (such as histograms and vectors) for Alpaka mimicking what's done in HeterogenousCore/CUDAUtilities. It also adds some helpers in HeterogeneousCore/AlpakaInterface/interface/workdivision.h for range computation and looping.

This includes #40932 and supersedes it with the latest comments received addressed.

Using PixelTracksAlpaka#15 as a log report to keep track of all the comments.

This PR and #41282, once reviewed, may be merged as they are. They do not depend on the other PRs.

PR validation:

It compiles. Running base tests. No regression expected.

cmsbuild · 2023-04-05T15:53:03Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41284/35059

This PR adds an extra 24KB to repository
There are other open Pull requests which might conflict with changes you have proposed:
- File HeterogeneousCore/AlpakaInterface/interface/workdivision.h modified in PR(s): Adding HeterogeneousCore/AlpakaUtilities and changes into HeterogeneousCore/AlpakaInterface #40932, Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaInterface/test/alpaka/testVec.cc modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaUtilities/BuildFile.xml modified in PR(s): Adding HeterogeneousCore/AlpakaUtilities and changes into HeterogeneousCore/AlpakaInterface #40932, Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaUtilities/interface/AtomicPairCounter.h modified in PR(s): Adding HeterogeneousCore/AlpakaUtilities and changes into HeterogeneousCore/AlpakaInterface #40932, Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaUtilities/interface/HistoContainer.h modified in PR(s): Adding HeterogeneousCore/AlpakaUtilities and changes into HeterogeneousCore/AlpakaInterface #40932, Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaUtilities/interface/SimpleVector.h modified in PR(s): Adding HeterogeneousCore/AlpakaUtilities and changes into HeterogeneousCore/AlpakaInterface #40932, Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaUtilities/interface/VecArray.h modified in PR(s): Adding HeterogeneousCore/AlpakaUtilities and changes into HeterogeneousCore/AlpakaInterface #40932, Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaUtilities/interface/alpakastdAlgorithm.h modified in PR(s): Adding HeterogeneousCore/AlpakaUtilities and changes into HeterogeneousCore/AlpakaInterface #40932, Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaUtilities/interface/demangle.h modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaUtilities/interface/initialise.h modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaUtilities/interface/prefixScan.h modified in PR(s): Adding HeterogeneousCore/AlpakaUtilities and changes into HeterogeneousCore/AlpakaInterface #40932, Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaUtilities/interface/radixSort.h modified in PR(s): Adding HeterogeneousCore/AlpakaUtilities and changes into HeterogeneousCore/AlpakaInterface #40932, Porting Pixel Tracks to Alpaka [Not to Merge] #41117

cmsbuild · 2023-04-05T15:53:27Z

A new Pull Request was created by @AdrianoDee (Adriano Di Florio) for master.

It involves the following packages:

HeterogeneousCore/AlpakaInterface (heterogeneous)
HeterogeneousCore/AlpakaUtilities (****)

The following packages do not have a category, yet:

HeterogeneousCore/AlpakaUtilities
Please create a PR for https://github.com/cms-sw/cms-bot/blob/master/categories_map.py to assign category

@cmsbuild, @makortel, @fwyzard can you please review it and eventually sign? Thanks.
@makortel, @missirol, @rovere this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

AdrianoDee · 2023-04-05T16:31:49Z

please test

makortel

What about the unit tests (either ported from CUDAUtilities or from the pixeltrack-standalone)?

HeterogeneousCore/AlpakaInterface/interface/workdivision.h

HeterogeneousCore/AlpakaInterface/test/alpaka/testVec.cc

makortel · 2023-04-05T15:58:18Z

HeterogeneousCore/AlpakaUtilities/interface/AtomicPairCounter.h

In the hackathon with @fwyzard and @rovere we concluded that the kind of classes in AlpakaUtilities here would be ok to be plated in AlpakaInterface (to avoid adding new package), for now at least.

Moved everything under AlpakaInterface.

HeterogeneousCore/AlpakaUtilities/interface/AtomicPairCounter.h

makortel · 2023-04-05T16:03:43Z

HeterogeneousCore/AlpakaUtilities/interface/radixSort.h

+    radixSortImpl<TAcc, I, NS>(acc, (I const*)(a), ind, ind2, size, reorderFloat<TAcc, I>);
+  }
+
+  /* Not needed


Can it be removed?

HeterogeneousCore/AlpakaUtilities/interface/demangle.h

HeterogeneousCore/AlpakaUtilities/interface/initialise.h

cmsbuild · 2023-04-05T18:53:11Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-96c764/31848/summary.html
COMMIT: 0a63573
CMSSW: CMSSW_13_1_X_2023-04-05-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41284/31848/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 7 lines to the logs
Reco comparison results: 7 differences found in the comparisons
DQMHistoTests: Total files compared: 47
DQMHistoTests: Total histograms compared: 3365163
DQMHistoTests: Total failures: 3
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3365138
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 46 files compared)
Checked 202 log files, 155 edm output root files, 47 DQM output files
TriggerResults: no differences found

cmsbuild · 2023-04-12T09:32:42Z

-code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41284/35136

This PR adds an extra 40KB to repository
Found files with invalid states:
- HeterogeneousCore/AlpakaUtilities/interface/initialise.h:
  - Added: 0a63573
  - Deleted: bb7e8bf
- HeterogeneousCore/AlpakaUtilities/interface/radixSort.h:
  - Added: 0a63573
  - Deleted: bb7e8bf
- HeterogeneousCore/AlpakaInterface/interface/demangle.h:
  - Added: bb7e8bf
  - Deleted: e0d0108
- HeterogeneousCore/AlpakaUtilities/interface/HistoContainer.h:
  - Added: 0a63573
  - Deleted: bb7e8bf
- HeterogeneousCore/AlpakaInterface/interface/initialise.h:
  - Added: bb7e8bf
  - Deleted: ec38106
- HeterogeneousCore/AlpakaUtilities/interface/alpakastdAlgorithm.h:
  - Added: 0a63573
  - Deleted: bb7e8bf
- HeterogeneousCore/AlpakaUtilities/BuildFile.xml:
  - Added: 0a63573
  - Deleted: bb7e8bf
- HeterogeneousCore/AlpakaUtilities/interface/AtomicPairCounter.h:
  - Added: 0a63573
  - Deleted: bb7e8bf
- HeterogeneousCore/AlpakaUtilities/interface/SimpleVector.h:
  - Added: 0a63573
  - Deleted: bb7e8bf
- HeterogeneousCore/AlpakaUtilities/interface/VecArray.h:
  - Added: 0a63573
  - Deleted: bb7e8bf
- HeterogeneousCore/AlpakaUtilities/interface/prefixScan.h:
  - Added: 0a63573
  - Deleted: bb7e8bf
- HeterogeneousCore/AlpakaUtilities/interface/demangle.h:
  - Added: 0a63573
  - Deleted: bb7e8bf
There are other open Pull requests which might conflict with changes you have proposed:
- File HeterogeneousCore/AlpakaInterface/interface/workdivision.h modified in PR(s): Adding HeterogeneousCore/AlpakaUtilities and changes into HeterogeneousCore/AlpakaInterface #40932, Porting Pixel Tracks to Alpaka [Not to Merge] #41117
- File HeterogeneousCore/AlpakaInterface/test/alpaka/testVec.cc modified in PR(s): Porting Pixel Tracks to Alpaka [Not to Merge] #41117

Code check has found code style and quality issues which could be resolved by applying following patch(s)

code-format:
https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41284/35136/code-format.patch
e.g. curl -k https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41284/35136/code-format.patch | patch -p1
You can also run scram build code-format to apply code format directly

slava77 · 2023-04-28T18:50:19Z

@AdrianoDee @cms-sw/heterogeneous-l2
can this be integrated by itself or is this waiting for review of other "pixel alpaka migration" parts >=3 or yet something else?

slava77 · 2023-05-04T16:49:03Z

@AdrianoDee @cms-sw/heterogeneous-l2 can this be integrated by itself or is this waiting for review of other "pixel alpaka migration" parts >=3 or yet something else?

now that we moved to 13_2_X, can the integration proceed or is it blocked by something?

fwyzard · 2023-05-05T16:17:58Z

PixelTracksAlpaka#16 should be integrated in the branch of the PR first.

I would like to provide more updates, but I will need to have a full working recipe to make sure I won't break any of the new workflows 🤷🏻‍♂️

AdrianoDee · 2023-05-05T16:31:43Z

PixelTracksAlpaka#16 should be integrated in the branch of the PR first.

I would like to provide more updates, but I will need to have a full working recipe to make sure I won't break any of the new workflows 🤷🏻‍♂️

To be honest I had not noticed that (apparently I don't get notifications for PR being opened!). We are working on getting a working config, will come back as soon as possible.

smuzaffar · 2023-11-06T16:50:02Z

Milestone for this pull request has been moved to CMSSW_14_0_X.Please open a backport if it should also go in to CMSSW_13_3_X.

AdrianoDee · 2023-11-15T16:34:35Z

Closing since superseded by #43064

cmsbuild added this to the CMSSW_13_1_X milestone Apr 5, 2023

cmsbuild added pending-signatures tests-pending orp-pending new-package-pending code-checks-pending heterogeneous-pending labels Apr 5, 2023

cmsbuild added code-checks-approved and removed code-checks-pending labels Apr 5, 2023

cmsbuild added tests-started and removed tests-pending labels Apr 5, 2023

makortel reviewed Apr 5, 2023

View reviewed changes

cmsbuild added tests-approved and removed tests-started labels Apr 5, 2023

AdrianoDee mentioned this pull request Apr 6, 2023

Log Report for PR reviews PixelTracksAlpaka/cmssw#15

Closed

71 tasks

cmsbuild added tests-pending code-checks-pending and removed tests-approved new-package-pending code-checks-approved labels Apr 12, 2023

cmsbuild added tests-approved and removed tests-started labels Apr 16, 2023

smuzaffar modified the milestones: CMSSW_13_1_X, CMSSW_13_2_X May 4, 2023

cmsbuild mentioned this pull request May 5, 2023

Make every Alpaka EDProducer to store an unsigned short int for the backend #41564

Merged

smuzaffar modified the milestones: CMSSW_13_2_X, CMSSW_13_3_X Jul 18, 2023

cmsbuild mentioned this pull request Aug 13, 2023

Fix elements_with_stride_nd when the index is outside the extent #42558

Merged

cmsbuild mentioned this pull request Aug 29, 2023

[HGCAL trigger] Module splitting and updated seeding normalization #42246

Merged

This was referenced Oct 19, 2023

Ports prefixScan, OneToManyAssoc and HistoContainer from CUDAUtilities. #43064

Merged

Porting Pixel Tracks to Alpaka [Not to Merge] #41117

Closed

smuzaffar modified the milestones: CMSSW_13_3_X, CMSSW_14_0_X Nov 6, 2023

cmsbuild modified the milestones: CMSSW_14_0_X, CMSSW_13_3_X Nov 6, 2023

cmsbuild mentioned this pull request Nov 6, 2023

Introduce new utilities for writing Alpaka kernels #43205

Merged

cmsbuild modified the milestones: CMSSW_13_3_X, CMSSW_14_0_X Nov 6, 2023

cmsbuild mentioned this pull request Nov 14, 2023

ECAL unpacker and ECAL multifit algorithm migration to alpaka #43257

Merged

AdrianoDee closed this Nov 15, 2023

This was referenced Nov 15, 2023

Pixel Alpaka Migration: Configs and Fixes [VII] #43294

Merged

Pixel Alpaka Migration: Portable Product [II] #43295

Merged

Comments to #41285 PixelTracksAlpaka/cmssw#41

Open

AdrianoDee deleted the alpaka_port_13_1_utilities branch January 31, 2024 13:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pixel Alpaka Migration: `AlpakaInterface` Updates [II] #41284

Pixel Alpaka Migration: `AlpakaInterface` Updates [II] #41284

AdrianoDee commented Apr 5, 2023 •

edited

Loading

cmsbuild commented Apr 5, 2023

cmsbuild commented Apr 5, 2023

AdrianoDee commented Apr 5, 2023

makortel left a comment

makortel Apr 5, 2023

AdrianoDee Apr 12, 2023

makortel Apr 5, 2023

AdrianoDee Apr 12, 2023

cmsbuild commented Apr 5, 2023

cmsbuild commented Apr 12, 2023

slava77 commented Apr 28, 2023

slava77 commented May 4, 2023

fwyzard commented May 5, 2023

AdrianoDee commented May 5, 2023

smuzaffar commented Nov 6, 2023

AdrianoDee commented Nov 15, 2023

Pixel Alpaka Migration: AlpakaInterface Updates [II] #41284

Pixel Alpaka Migration: AlpakaInterface Updates [II] #41284

Conversation

AdrianoDee commented Apr 5, 2023 • edited Loading

PR description:

PR validation:

cmsbuild commented Apr 5, 2023

cmsbuild commented Apr 5, 2023

AdrianoDee commented Apr 5, 2023

makortel left a comment

Choose a reason for hiding this comment

makortel Apr 5, 2023

Choose a reason for hiding this comment

AdrianoDee Apr 12, 2023

Choose a reason for hiding this comment

makortel Apr 5, 2023

Choose a reason for hiding this comment

AdrianoDee Apr 12, 2023

Choose a reason for hiding this comment

cmsbuild commented Apr 5, 2023

Comparison Summary

cmsbuild commented Apr 12, 2023

slava77 commented Apr 28, 2023

slava77 commented May 4, 2023

fwyzard commented May 5, 2023

AdrianoDee commented May 5, 2023

smuzaffar commented Nov 6, 2023

AdrianoDee commented Nov 15, 2023

Pixel Alpaka Migration: `AlpakaInterface` Updates [II] #41284

Pixel Alpaka Migration: `AlpakaInterface` Updates [II] #41284

AdrianoDee commented Apr 5, 2023 •

edited

Loading