warn on tbb::global_control interferences #6814

xvallspl · 2020-11-16T09:56:21Z

warn when the number of threads set by the user is limited at runtime
by tbb::global_control

Addressing #6363

xvallspl · 2020-11-16T10:01:02Z

The current solution might be too noisy in some contexts, for instance RDataFrame, where "every MT event loop should be exactly two calls to Foreach but you might have a bunch of event loops per application" (@eguiraud 📝 ). I'll give it another look.

phsft-bot · 2021-01-07T10:05:45Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-fedora30/cxx14, ROOT-fedora31/noimt, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-01-07T10:31:11Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-fedora30/cxx14, ROOT-fedora31/noimt, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

xvallspl · 2021-01-07T10:32:22Z

Hah, I thought this one was merged already.

I need to add another test case in the threadExecutor test in roottest to account for this

phsft-bot · 2021-01-07T11:04:29Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Warnings:

[2021-01-07T10:42:04.581Z] /home/sftnight/build/workspace/root-pullrequests-build/root/core/imt/src/TThreadExecutor.cxx:145:15: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘std::size_t’ {aka ‘unsigned int’} [-Wformat=]
[2021-01-07T10:42:04.582Z] /home/sftnight/build/workspace/root-pullrequests-build/root/core/imt/src/TThreadExecutor.cxx:161:15: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘std::size_t’ {aka ‘unsigned int’} [-Wformat=]

phsft-bot · 2021-01-07T12:47:13Z

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:

phsft-bot · 2021-01-07T16:18:17Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-fedora30/cxx14, ROOT-fedora31/noimt, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-01-07T16:39:37Z

Build failed on ROOT-fedora30/cxx14.
Running on root-fedora30-1.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Failing tests:

projectroot.tree.ntuple.v7.test.gtest_tree_ntuple_v7_test_ntuple_extended

phsft-bot · 2021-01-07T16:50:31Z

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-1.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Failing tests:

And 440 more

phsft-bot · 2021-01-07T16:52:51Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Warnings:

[2021-01-07T16:31:50.667Z] /home/sftnight/build/workspace/root-pullrequests-build/root/core/imt/src/TThreadExecutor.cxx:160:15: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘std::size_t’ {aka ‘unsigned int’} [-Wformat=]
[2021-01-07T16:31:50.668Z] /home/sftnight/build/workspace/root-pullrequests-build/root/core/imt/src/TThreadExecutor.cxx:182:15: warning: format ‘%ld’ expects argument of type ‘long int’, but argument 3 has type ‘std::size_t’ {aka ‘unsigned int’} [-Wformat=]

phsft-bot · 2021-01-07T17:23:17Z

Build failed on mac11.0/cxx17.
Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:

projectroot.tree.ntuple.v7.test.gtest_tree_ntuple_v7_test_ntuple_extended

phsft-bot · 2021-01-08T08:20:08Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-fedora30/cxx14, ROOT-fedora31/noimt, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-01-08T08:55:02Z

Build failed on ROOT-debian10-i386/cxx14.
Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Warnings:

[2021-01-08T08:33:18.408Z] /home/sftnight/build/workspace/root-pullrequests-build/root/core/imt/src/TThreadExecutor.cxx:160:15: warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 3 has type ‘std::size_t’ {aka ‘unsigned int’} [-Wformat=]
[2021-01-08T08:33:18.408Z] /home/sftnight/build/workspace/root-pullrequests-build/root/core/imt/src/TThreadExecutor.cxx:182:15: warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 3 has type ‘std::size_t’ {aka ‘unsigned int’} [-Wformat=]

phsft-bot · 2021-01-08T09:00:19Z

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-1.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Failing tests:

And 440 more

phsft-bot · 2021-01-08T09:27:16Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-fedora30/cxx14, ROOT-fedora31/noimt, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-01-08T09:58:22Z

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-1.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Failing tests:

And 440 more

phsft-bot · 2021-01-12T07:19:15Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-fedora30/cxx14, ROOT-fedora31/noimt, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-01-12T08:03:07Z

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-1.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Failing tests:

And 440 more

phsft-bot · 2021-01-12T14:04:36Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-fedora30/cxx14, ROOT-fedora31/noimt, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-01-12T14:35:54Z

Build failed on ROOT-ubuntu16/nortcxxmod.
Running on sft-ubuntu-1604-3.cern.ch:/build/workspace/root-pullrequests-build
See console output.

Failing tests:

And 440 more

eguiraud · 2021-01-14T13:52:04Z

@phsft-bot build please

phsft-bot · 2021-01-14T13:52:14Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-fedora30/cxx14, ROOT-fedora31/noimt, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

eguiraud

Thanks Xavi! The comment about false positives is the only required change.

core/imt/test/testTBBGlobalControl.cxx

eguiraud · 2021-01-14T13:58:30Z

core/imt/src/TThreadExecutor.cxx

+              "tbb::global_control is limiting the number of parallel workers."
+              " Proceeding with %zu threads this time",
+              tbb::global_control::active_value(tbb::global_control::max_allowed_parallelism));
+   }


this logic repeated 3 times could be factored out in a little helper function

For these three times in such a short file, I feel creating a helper to print would be overkill, and creating a helper that includes the call to GetPoolSize() would complicate the code.

phsft-bot · 2021-01-14T14:58:18Z

Build failed on mac11.0/cxx17.
Running on macphsft23.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build
See console output.

Failing tests:

projectroot.roottest.root.multicore.roottest_root_multicore_tExecutor

warn when the number of threads set by the user is limited at runtime by tbb::global_control. Fix for github issue root-project#6363: root-project#6363

phsft-bot · 2021-01-14T15:25:24Z

Build failed on ROOT-performance-centos8-multicore/default.
Running on null:/data/sftnight/workspace/root-pullrequests-build
See console output.

Failing tests:

projectroot.runtutorials.tutorial_dataframe_df107_SingleTopAnalysis_py

phsft-bot · 2021-01-14T15:44:34Z

Starting build on ROOT-debian10-i386/cxx14, ROOT-performance-centos8-multicore/default, ROOT-fedora30/cxx14, ROOT-fedora31/noimt, ROOT-ubuntu16/nortcxxmod, mac1014/python3, mac11.0/cxx17, windows10/cxx14
How to customize builds

phsft-bot · 2021-01-14T19:33:38Z

Build failed on ROOT-performance-centos8-multicore/default.
Running on null:/data/sftnight/workspace/root-pullrequests-build
See console output.

Failing tests:

projectroot.runtutorials.tutorial_dataframe_df105_WBosonAnalysis_py

eguiraud

LGTM!

xvallspl requested a review from dpiparo as a code owner November 16, 2020 09:56

jblomer assigned xvallspl Nov 16, 2020

xvallspl force-pushed the globalControl branch from f113e4f to e25fc2b Compare January 7, 2021 10:05

xvallspl force-pushed the globalControl branch from e25fc2b to 78a2cba Compare January 7, 2021 10:31

xvallspl force-pushed the globalControl branch from 78a2cba to 7c21c51 Compare January 7, 2021 16:18

xvallspl force-pushed the globalControl branch from 77d8cc8 to 714f75b Compare January 8, 2021 09:27

xvallspl requested review from eguiraud and removed request for dpiparo January 11, 2021 10:52

xvallspl force-pushed the globalControl branch from 714f75b to ee2c8c2 Compare January 12, 2021 07:19

xvallspl force-pushed the globalControl branch from ee2c8c2 to f1cc84f Compare January 12, 2021 14:04

xvallspl requested a review from oshadura as a code owner January 12, 2021 14:04

eguiraud requested changes Jan 14, 2021

View reviewed changes

warn on tbb::global_control interferences

14aa8bc

warn when the number of threads set by the user is limited at runtime by tbb::global_control. Fix for github issue root-project#6363: root-project#6363

test the interactions between IMT and tbb::global_control

e7c1d1d

xvallspl force-pushed the globalControl branch from f1cc84f to e7c1d1d Compare January 14, 2021 15:44

eguiraud approved these changes Jan 15, 2021

View reviewed changes

xvallspl merged commit 064f7f7 into root-project:master Jan 15, 2021

xvallspl mentioned this pull request Jan 15, 2021

[IMT] ROOT::GetThreadPoolSize does not reflect tbb::global_control settings #6363

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

warn on tbb::global_control interferences #6814

warn on tbb::global_control interferences #6814

xvallspl commented Nov 16, 2020

xvallspl commented Nov 16, 2020

phsft-bot commented Jan 7, 2021

phsft-bot commented Jan 7, 2021

xvallspl commented Jan 7, 2021

phsft-bot commented Jan 7, 2021

phsft-bot commented Jan 7, 2021

phsft-bot commented Jan 7, 2021

phsft-bot commented Jan 7, 2021

phsft-bot commented Jan 7, 2021

phsft-bot commented Jan 7, 2021

phsft-bot commented Jan 7, 2021

phsft-bot commented Jan 8, 2021

phsft-bot commented Jan 8, 2021

phsft-bot commented Jan 8, 2021

phsft-bot commented Jan 8, 2021

phsft-bot commented Jan 8, 2021

phsft-bot commented Jan 12, 2021

phsft-bot commented Jan 12, 2021

phsft-bot commented Jan 12, 2021

phsft-bot commented Jan 12, 2021

eguiraud commented Jan 14, 2021

phsft-bot commented Jan 14, 2021

eguiraud left a comment

eguiraud Jan 14, 2021

xvallspl Jan 15, 2021

phsft-bot commented Jan 14, 2021

phsft-bot commented Jan 14, 2021

phsft-bot commented Jan 14, 2021

phsft-bot commented Jan 14, 2021

eguiraud left a comment

warn on tbb::global_control interferences #6814

warn on tbb::global_control interferences #6814

Conversation

xvallspl commented Nov 16, 2020

xvallspl commented Nov 16, 2020

phsft-bot commented Jan 7, 2021

phsft-bot commented Jan 7, 2021

xvallspl commented Jan 7, 2021

phsft-bot commented Jan 7, 2021

Warnings:

phsft-bot commented Jan 7, 2021

Failing tests:

phsft-bot commented Jan 7, 2021

phsft-bot commented Jan 7, 2021

Failing tests:

phsft-bot commented Jan 7, 2021

Failing tests:

phsft-bot commented Jan 7, 2021

Warnings:

phsft-bot commented Jan 7, 2021

Failing tests:

phsft-bot commented Jan 8, 2021

phsft-bot commented Jan 8, 2021

Warnings:

phsft-bot commented Jan 8, 2021

Failing tests:

phsft-bot commented Jan 8, 2021

phsft-bot commented Jan 8, 2021

Failing tests:

phsft-bot commented Jan 12, 2021

phsft-bot commented Jan 12, 2021

Failing tests:

phsft-bot commented Jan 12, 2021

phsft-bot commented Jan 12, 2021

Failing tests:

eguiraud commented Jan 14, 2021

phsft-bot commented Jan 14, 2021

eguiraud left a comment

Choose a reason for hiding this comment

eguiraud Jan 14, 2021

Choose a reason for hiding this comment

xvallspl Jan 15, 2021

Choose a reason for hiding this comment

phsft-bot commented Jan 14, 2021

Failing tests:

phsft-bot commented Jan 14, 2021

Failing tests:

phsft-bot commented Jan 14, 2021

phsft-bot commented Jan 14, 2021

Failing tests:

eguiraud left a comment

Choose a reason for hiding this comment