[TBB] Fix concurrent_[bounded_]queue correctness on weak memory models #8355

smuzaffar · 2023-03-03T15:40:59Z

Applied uxlfoundation/oneTBB#782 patch on top of v2021.8.0 .
This should fix the testFWCoreUtilities failure in ARM IBs

smuzaffar · 2023-03-03T15:41:26Z

please test

cmsbuild · 2023-03-03T15:41:27Z

A new Pull Request was created by @smuzaffar (Malik Shahzad Muzaffar) for branch IB/CMSSW_13_1_X/master.

@cmsbuild, @smuzaffar, @aandvalenzuela, @iarspider can you please review it and eventually sign? Thanks.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.
cms-bot commands are listed here

fwyzard · 2023-03-03T15:44:35Z

please test for el8_ppc64le_gcc11

smuzaffar · 2023-03-03T15:46:13Z

please test for el8_aarch64_gcc11

cmsbuild · 2023-03-03T16:00:16Z

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31047/summary.html
COMMIT: 58a8d94
CMSSW: CMSSW_13_1_X_2023-03-02-2300/el8_aarch64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8355/31047/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation error when building:

Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/oneapi/tbb/concurrent_queue.h.rej
patching file include/oneapi/tbb/concurrent_queue.h
patching file include/oneapi/tbb/detail/_concurrent_queue_base.h
Hunk #1 succeeded at 338 (offset 2 lines).
error: Bad exit status from /data/cmsbld/jenkins_a/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.FXHOBX (%prep)


RPM build errors:
line 37: It's not recommended to have unversioned Obsoletes: Obsoletes: external+tbb+v2021.8.0-f9a18214b66c48f0e1cdde8496021d24
Bad exit status from /data/cmsbld/jenkins_a/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.FXHOBX (%prep)

cmsbuild · 2023-03-03T16:02:26Z

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31046/summary.html
COMMIT: 58a8d94
CMSSW: CMSSW_13_1_X_2023-03-03-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8355/31046/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation error when building:

Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/oneapi/tbb/concurrent_queue.h.rej
patching file include/oneapi/tbb/concurrent_queue.h
patching file include/oneapi/tbb/detail/_concurrent_queue_base.h
Hunk #1 succeeded at 338 (offset 2 lines).
error: Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.YXCiAu (%prep)


RPM build errors:
line 37: It's not recommended to have unversioned Obsoletes: Obsoletes: external+tbb+v2021.8.0-cecd5e3ae11fdc708d76ad7509c71a73
Bad exit status from /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.YXCiAu (%prep)

cmsbuild · 2023-03-03T17:06:20Z

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31050/summary.html
COMMIT: 58a8d94
CMSSW: CMSSW_13_1_X_2023-03-02-2300/el8_ppc64le_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8355/31050/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation error when building:

Skipping patch.
1 out of 1 hunk ignored -- saving rejects to file include/oneapi/tbb/concurrent_queue.h.rej
patching file include/oneapi/tbb/concurrent_queue.h
patching file include/oneapi/tbb/detail/_concurrent_queue_base.h
Hunk #1 succeeded at 338 (offset 2 lines).
error: Bad exit status from /scratch/cmsbuild/jenkins_b/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.ytCZpg (%prep)


RPM build errors:
line 37: It's not recommended to have unversioned Obsoletes: Obsoletes: external+tbb+v2021.8.0-f9a18214b66c48f0e1cdde8496021d24
Bad exit status from /scratch/cmsbuild/jenkins_b/workspace/ib-run-pr-tests/testBuildDir/tmp/rpm-tmp.ytCZpg (%prep)

iarspider · 2023-03-03T20:54:59Z

please test for el8_ppc64le_gcc11

cmsbuild · 2023-03-03T20:55:03Z

Pull request #8355 was updated.

iarspider · 2023-03-03T20:55:06Z

please test for el8_aarch64_gcc11

iarspider · 2023-03-03T20:55:18Z

please test

tbb.spec

cmsbuild · 2023-03-03T21:07:17Z

-1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31056/summary.html
COMMIT: 3140f31
CMSSW: CMSSW_13_1_X_2023-03-02-2300/el8_aarch64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8355/31056/install.sh to create a dev area with all the needed externals and cmssw changes.

External Build

I found compilation warning when building: See details on the summary page.

Co-authored-by: Andrea Bocci <[email protected]>

cmsbuild · 2023-03-03T21:17:28Z

Pull request #8355 was updated.

iarspider · 2023-03-03T22:16:42Z

please test for el8_aarch64_gcc11

iarspider · 2023-03-03T22:16:50Z

please test for el8_ppc64le_gcc11

cmsbuild · 2023-03-04T01:54:35Z

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31060/summary.html
COMMIT: d5d58df
CMSSW: CMSSW_13_1_X_2023-03-03-1100/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8355/31060/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@cms-sw Update ROOT 6.26 in CMSSW master #8345

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31060/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31060/git-merge-result

Build

I found compilation error when building:

Traceback (most recent call last):
  File "/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_13_1_X_2023-03-03-1100/src/FWCore/Utilities/scripts/edmCheckClassVersion", line 181, in 
    raise RuntimeError("failed to load library '"+options.library+"'")
RuntimeError: failed to load library 'libCondCoreUtilities.so'
Suggestion: You can run 'scram build updateclassversion' to generate src/CondCore/Utilities/src/classes_def.xml.generated with updated ClassVersion
gmake: *** [tmp/el8_amd64_gcc11/edm_checks/libCondCoreUtilities.so] Error 1
@@@@ ----> OK  EDM Class Transients 
@@@@ ----> OK  EDM Class Transients 
@@@@ ----> OK  EDM Class Transients 
@@@@ ----> OK  EDM Class Transients 
@@@@ ----> OK  EDM Class Transients

cmsbuild · 2023-03-04T02:09:27Z

-1

Failed Tests: Build
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31061/summary.html
COMMIT: d5d58df
CMSSW: CMSSW_13_1_X_2023-03-02-2300/el8_ppc64le_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8355/31061/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31061/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31061/git-merge-result

Build

I found compilation error when building:

RuntimeError: failed to load library 'libCondCoreUtilities.so'
Disabling gnu++: clang has no __float128 support on this target!
@@@@ ----> OK  EDM Class Transients 
>> Checking EDM Class Version for src/DataFormats/FEDRawData/src/classes_def.xml in libDataFormatsFEDRawData.so
Suggestion: You can run 'scram build updateclassversion' to generate src/CondCore/Utilities/src/classes_def.xml.generated with updated ClassVersion
gmake: *** [tmp/el8_ppc64le_gcc11/edm_checks/libCondCoreUtilities.so] Error 1
>> Checking EDM Class Version for src/DataFormats/FTLDigi/src/classes_def.xml in libDataFormatsFTLDigi.so
@@@@ ----> OK  EDM Class Transients 
>> Checking EDM Class Version for src/DataFormats/FTLRecHit/src/classes_def.xml in libDataFormatsFTLRecHit.so
@@@@ ----> OK  EDM Class Transients 
@@@@ ----> OK  EDM Class Transients

iarspider · 2023-03-04T08:41:23Z

test parameters:

full_cmssw = true

iarspider · 2023-03-04T08:41:32Z

please test

iarspider · 2023-03-04T08:41:44Z

please test for el8_aarch64_gcc11

iarspider · 2023-03-04T08:41:57Z

please test for el8_ppc64le_gcc11

cmsbuild · 2023-03-04T12:07:30Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31064/summary.html
COMMIT: d5d58df
CMSSW: CMSSW_13_1_X_2023-03-03-2300/el8_aarch64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8355/31064/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@mmusich SiPixelPayloadInspectorHelper: fix warning: '<anonymous>' may be used uninitialized cmssw#40950
@guitargeek Clean unused dependencies from BuildFiles cmssw#40906

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31064/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31064/git-merge-result

cmsbuild · 2023-03-04T14:18:02Z

+1

Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31063/summary.html
COMMIT: d5d58df
CMSSW: CMSSW_13_1_X_2023-03-03-2300/el8_amd64_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8355/31063/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

@mmusich SiPixelPayloadInspectorHelper: fix warning: '<anonymous>' may be used uninitialized cmssw#40950
@guitargeek Clean unused dependencies from BuildFiles cmssw#40906

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31063/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31063/git-merge-result

Comparison Summary

Summary:

You potentially removed 178 lines from the logs
Reco comparison results: 8 differences found in the comparisons
DQMHistoTests: Total files compared: 49
DQMHistoTests: Total histograms compared: 3529727
DQMHistoTests: Total failures: 6
DQMHistoTests: Total nulls: 0
DQMHistoTests: Total successes: 3529699
DQMHistoTests: Total skipped: 22
DQMHistoTests: Total Missing objects: 0
DQMHistoSizes: Histogram memory added: 0.0 KiB( 48 files compared)
Checked 213 log files, 164 edm output root files, 49 DQM output files
TriggerResults: no differences found

cmsbuild · 2023-03-04T14:26:03Z

-1

Failed Tests: UnitTests
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31067/summary.html
COMMIT: d5d58df
CMSSW: CMSSW_13_1_X_2023-03-03-2300/el8_ppc64le_gcc11
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmsdist/8355/31067/install.sh to create a dev area with all the needed externals and cmssw changes.

The following merge commits were also included on top of IB + this PR after doing git cms-merge-topic:

You can see more details here:
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31067/git-recent-commits.json
https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-650752/31067/git-merge-result

Unit Tests

I found errors in the following unit tests:

---> test testONNXRuntime had ERRORS

fwyzard · 2023-03-04T17:54:41Z

Nice, this did fix testFWCoreUtilities on ARM and Power :-)

fwyzard · 2023-03-04T17:56:19Z

@smuzaffar could you squash the changes into a single commit ?

smuzaffar · 2023-03-04T18:18:19Z

+externals

cmsbuild · 2023-03-04T18:18:37Z

This pull request is fully signed and it will be integrated in one of the next IB/CMSSW_13_1_X/master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @perrotta, @dpiparo, @rappoccio (and backports should be raised in the release meeting by the corresponding L2)

[TBB] Fix concurrent_[bounded_]queue correctness on weak memory models

58a8d94

cmsbuild added tests-pending externals-pending pending-signatures orp-pending labels Mar 3, 2023

cmsbuild added tests-started and removed tests-pending labels Mar 3, 2023

cmsbuild added tests-pending and removed tests-started labels Mar 3, 2023

cmsbuild added tests-started and removed tests-pending labels Mar 3, 2023

cmsbuild added tests-rejected and removed tests-started labels Mar 3, 2023

Fix patch

3140f31

cmsbuild added tests-started and removed tests-rejected labels Mar 3, 2023

fwyzard reviewed Mar 3, 2023

View reviewed changes

tbb.spec Outdated Show resolved Hide resolved

Apply suggestions from code review

6afe37e

Co-authored-by: Andrea Bocci <[email protected]>

cmsbuild added tests-rejected and removed tests-started labels Mar 4, 2023

cmsbuild added tests-started and removed tests-rejected labels Mar 4, 2023

cmsbuild added tests-approved and removed tests-started labels Mar 4, 2023

fwyzard mentioned this pull request Mar 4, 2023

[TBB] Fix concurrent_[bounded_]queue correctness on weak memory models [13.0.x] #8358

Merged

cmsbuild added externals-approved fully-signed and removed externals-pending pending-signatures labels Mar 4, 2023

smuzaffar merged commit 89fe74f into IB/CMSSW_13_1_X/master Mar 4, 2023

iarspider deleted the smuzaffar-patch-5 branch March 5, 2023 09:15

makortel mentioned this pull request Mar 7, 2023

testFWCoreUtilities fails in ARM IBs cms-sw/cmssw#34647

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TBB] Fix concurrent_[bounded_]queue correctness on weak memory models #8355

[TBB] Fix concurrent_[bounded_]queue correctness on weak memory models #8355

smuzaffar commented Mar 3, 2023

smuzaffar commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

fwyzard commented Mar 3, 2023

smuzaffar commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

iarspider commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

iarspider commented Mar 3, 2023

iarspider commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

iarspider commented Mar 3, 2023

iarspider commented Mar 3, 2023

cmsbuild commented Mar 4, 2023

cmsbuild commented Mar 4, 2023

iarspider commented Mar 4, 2023

iarspider commented Mar 4, 2023

iarspider commented Mar 4, 2023

iarspider commented Mar 4, 2023

cmsbuild commented Mar 4, 2023

cmsbuild commented Mar 4, 2023

cmsbuild commented Mar 4, 2023

fwyzard commented Mar 4, 2023

fwyzard commented Mar 4, 2023

smuzaffar commented Mar 4, 2023

cmsbuild commented Mar 4, 2023

[TBB] Fix concurrent_[bounded_]queue correctness on weak memory models #8355

[TBB] Fix concurrent_[bounded_]queue correctness on weak memory models #8355

Conversation

smuzaffar commented Mar 3, 2023

smuzaffar commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

fwyzard commented Mar 3, 2023

smuzaffar commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

External Build

cmsbuild commented Mar 3, 2023

External Build

cmsbuild commented Mar 3, 2023

External Build

iarspider commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

iarspider commented Mar 3, 2023

iarspider commented Mar 3, 2023

cmsbuild commented Mar 3, 2023

External Build

cmsbuild commented Mar 3, 2023

iarspider commented Mar 3, 2023

iarspider commented Mar 3, 2023

cmsbuild commented Mar 4, 2023

Build

cmsbuild commented Mar 4, 2023

Build

iarspider commented Mar 4, 2023

iarspider commented Mar 4, 2023

iarspider commented Mar 4, 2023

iarspider commented Mar 4, 2023

cmsbuild commented Mar 4, 2023

cmsbuild commented Mar 4, 2023

Comparison Summary

cmsbuild commented Mar 4, 2023

Unit Tests

fwyzard commented Mar 4, 2023

fwyzard commented Mar 4, 2023

smuzaffar commented Mar 4, 2023

cmsbuild commented Mar 4, 2023