Fix implicit host-to-device copy for ES data products with labels in Alpaka ESProducers #41019

makortel · 2023-03-09T21:40:31Z

PR description:

It was noticed during the Patatrack hackathon that the implicit host-to-device copy of EventSetup data products did not work correctly in Alpaka ESProducers when the data product had a label set (either via explicit argument to setWhatProduced() or via the appendToDataLabel configuration parameter). The test code that I added in #40813 turned out to have a flaw, because of which the problem was not caught earlier.

Requiring the concrete ESProducer classes to explicitly call the base ESProducer class' constructor with the ParameterSet argument is a (minor) breaking change, but this was (by far) easiest way to fix the problem.

PR validation:

The unit tests succeeds on machines with and without a GPU.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

To be backported to 13_0_X (using the same commit).

…Alpaka ESProducers Taking the "appendToDataLabel" and module label from the module's ParameterSet was, by far, the easiest way to fix, supported by the hindsight that having all modules to pass the ParameterSet to the base class' constructor would have made some features much easier and cleaner to develop.

makortel · 2023-03-09T21:43:10Z

@fwyzard The branch of this PR is based on abba246 (i.e. #40849 / #40935), which, I think, was the latest commit/PR in our joint branch.

cmsbuild · 2023-03-09T21:48:26Z

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-41019/34541

This PR adds an extra 20KB to repository

cmsbuild · 2023-03-09T21:49:07Z

A new Pull Request was created by @makortel (Matti Kortelainen) for master.

It involves the following packages:

HeterogeneousCore/AlpakaCore (heterogeneous)
HeterogeneousCore/AlpakaTest (heterogeneous)

@cmsbuild, @makortel, @fwyzard can you please review it and eventually sign? Thanks.
@missirol, @rovere this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here

makortel · 2023-03-09T21:50:56Z

enable gpu

makortel · 2023-03-09T21:51:03Z

@cmsbuild, please test

cmsbuild · 2023-03-09T23:48:31Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d47084/31177/summary.html
COMMIT: aea87d6
CMSSW: CMSSW_13_1_X_2023-03-09-1100/el8_amd64_gcc11
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41019/31177/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially removed 10 lines from the logs
Reco comparison results: 19 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3530381
DQMHistoTests: Total failures: 122
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3530237
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 0 differences found in the comparisons
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19862
DQMHistoTests: Total failures: 10
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 19852
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

makortel · 2023-03-10T20:49:06Z

Hmh, many phase 2 workflows show differences in some JetMET histograms.

makortel · 2023-03-10T20:49:26Z

@cmsbuild, please test

Let's see if they persist

missirol · 2023-03-10T21:49:02Z

Hmh, many phase 2 workflows show differences in some JetMET histograms.

(Just for the record, I saw something similar yesterday, #41016 (comment).)

cmsbuild · 2023-03-10T23:10:44Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-d47084/31214/summary.html
COMMIT: aea87d6
CMSSW: CMSSW_13_1_X_2023-03-10-1100/el8_amd64_gcc11
Additional Tests: GPU
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/41019/31214/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

You potentially added 2 lines to the logs
Reco comparison results: 3 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3530381
DQMHistoTests: Total failures: 7
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3530352
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

GPU Comparison Summary

Summary:

No significant changes to the logs found
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 4
DQMHistoTests: Total histograms compared: 19862
DQMHistoTests: Total failures: 455
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 19407
DQMHistoTests: Total skipped: 0
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 3 files compared)
Checked 12 log files, 9 edm output root files, 4 DQM output files
TriggerResults: no differences found

makortel · 2023-03-13T09:15:41Z

The JetMET difference didn't repeat, but the latest incarnation of #39803 came visible.

makortel · 2023-03-13T09:15:50Z

@fwyzard Do you have any comments?

fwyzard · 2023-03-13T09:44:45Z

I trust you on this :-)

makortel · 2023-03-13T10:07:54Z

+heterogeneous

makortel · 2023-03-13T10:08:15Z

type bug-fix

cmsbuild · 2023-03-13T10:08:19Z

This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

perrotta · 2023-03-13T15:58:41Z

+1

cmsbuild added this to the CMSSW_13_1_X milestone Mar 9, 2023

cmsbuild added pending-signatures tests-pending orp-pending code-checks-pending heterogeneous-pending labels Mar 9, 2023

makortel mentioned this pull request Mar 9, 2023

Fix ES data product label usage in Alpaka ESProducer base class cms-sw/framework-team#508

Closed

2 tasks

cmsbuild added code-checks-approved and removed code-checks-pending labels Mar 9, 2023

cmsbuild added tests-started and removed tests-pending labels Mar 9, 2023

cmsbuild added tests-approved and removed tests-started labels Mar 9, 2023

cmsbuild added tests-started and removed tests-approved labels Mar 10, 2023

makortel mentioned this pull request Mar 10, 2023

Non-reproducibility in JetMET/{Jet,MET}Validation histograms in phase2 workflows #39754

Open

cmsbuild added tests-approved and removed tests-started labels Mar 10, 2023

cmsbuild added fully-signed bug-fix heterogeneous-approved and removed pending-signatures heterogeneous-pending labels Mar 13, 2023

makortel mentioned this pull request Mar 13, 2023

[13_0_X] Fix implicit host-to-device copy for ES data products with labels in Alpaka ESProducers #41038

Merged

cmsbuild added orp-approved and removed orp-pending labels Mar 13, 2023

cmsbuild merged commit fdaea76 into cms-sw:master Mar 13, 2023

makortel deleted the fixAlpakaESDataLabel_v2 branch March 17, 2023 07:27

makortel mentioned this pull request May 5, 2023

Make every Alpaka EDProducer to store an unsigned short int for the backend #41564

Merged

makortel mentioned this pull request Oct 17, 2023

Updates to AlpakaCore README and tests #43033

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix implicit host-to-device copy for ES data products with labels in Alpaka ESProducers #41019

Fix implicit host-to-device copy for ES data products with labels in Alpaka ESProducers #41019

makortel commented Mar 9, 2023

makortel commented Mar 9, 2023

cmsbuild commented Mar 9, 2023

cmsbuild commented Mar 9, 2023

makortel commented Mar 9, 2023

makortel commented Mar 9, 2023

cmsbuild commented Mar 9, 2023

makortel commented Mar 10, 2023

makortel commented Mar 10, 2023

missirol commented Mar 10, 2023

cmsbuild commented Mar 10, 2023

makortel commented Mar 13, 2023

makortel commented Mar 13, 2023

fwyzard commented Mar 13, 2023

makortel commented Mar 13, 2023

makortel commented Mar 13, 2023

cmsbuild commented Mar 13, 2023

perrotta commented Mar 13, 2023

Fix implicit host-to-device copy for ES data products with labels in Alpaka ESProducers #41019

Fix implicit host-to-device copy for ES data products with labels in Alpaka ESProducers #41019

Conversation

makortel commented Mar 9, 2023

PR description:

PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

makortel commented Mar 9, 2023

cmsbuild commented Mar 9, 2023

cmsbuild commented Mar 9, 2023

makortel commented Mar 9, 2023

makortel commented Mar 9, 2023

cmsbuild commented Mar 9, 2023

Comparison Summary

GPU Comparison Summary

makortel commented Mar 10, 2023

makortel commented Mar 10, 2023

missirol commented Mar 10, 2023

cmsbuild commented Mar 10, 2023

Comparison Summary

GPU Comparison Summary

makortel commented Mar 13, 2023

makortel commented Mar 13, 2023

fwyzard commented Mar 13, 2023

makortel commented Mar 13, 2023

makortel commented Mar 13, 2023

cmsbuild commented Mar 13, 2023

perrotta commented Mar 13, 2023