Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enables threadpoolctl to control number of numpy threads #1161

Merged
merged 3 commits into from
Jun 30, 2021

Conversation

franchuterivera
Copy link
Contributor

addresses #924

One can see with threadpool_info() that the number of threads is set to 1 (this at the end of abstract evaluator, for debug purposes):

[CRITICAL] [2021-06-18 09:10:32,054:Client-TrainEvaluator(5):breast_cancer] threadpool_info()=[{'filepath': '/home/chico/anaconda3/envs/py38/lib/python3.8/site-packages/scikit_learn.libs/libgomp-f7e03b3e.so.1.0.0', 'prefix': 'libgomp', 'user_api': 'openmp', 'internal_api': 'openmp', 'version': None, 'num_threads': 1}, {'filepath': '/home/chico/anaconda3/envs/py38/lib/python3.8/site-packages/numpy.libs/libopenblasp-r0-5bebc122.3.13.dev.so', 'prefix': 'libopenblas', 'user_api': 'blas', 'internal_api': 'openblas', 'version': '0.3.13.dev', 'num_threads': 1, 'threading_layer': 'pthreads'}, {'filepath': '/home/chico/anaconda3/envs/py38/lib/python3.8/site-packages/scipy.libs/libopenblasp-r0-085ca80a.3.9.so', 'prefix': 'libopenblas', 'user_api': 'blas', 'internal_api': 'openblas', 'version': '0.3.9', 'num_threads': 1, 'threading_layer': 'pthreads'}]

@codecov
Copy link

codecov bot commented Jun 18, 2021

Codecov Report

Merging #1161 (4010f80) into development (ba3f76f) will increase coverage by 0.23%.
The diff coverage is 100.00%.

Impacted file tree graph

@@               Coverage Diff               @@
##           development    #1161      +/-   ##
===============================================
+ Coverage        85.81%   86.05%   +0.23%     
===============================================
  Files              137      138       +1     
  Lines            10625    10663      +38     
===============================================
+ Hits              9118     9176      +58     
+ Misses            1507     1487      -20     
Impacted Files Coverage Δ
autosklearn/evaluation/abstract_evaluator.py 90.15% <100.00%> (+0.11%) ⬆️
...tosklearn/metalearning/metafeatures/metafeature.py 62.13% <0.00%> (-14.57%) ⬇️
...eline/components/feature_preprocessing/fast_ica.py 91.30% <0.00%> (-6.53%) ⬇️
...onents/data_preprocessing/rescaling/standardize.py 95.23% <0.00%> (-4.77%) ⬇️
...ents/data_preprocessing/rescaling/robust_scaler.py 96.87% <0.00%> (-3.13%) ⬇️
...data_preprocessing/rescaling/abstract_rescaling.py 91.30% <0.00%> (-1.01%) ⬇️
...omponents/data_preprocessing/data_preprocessing.py 90.09% <0.00%> (-0.23%) ⬇️
...osklearn/metalearning/metafeatures/metafeatures.py 94.59% <0.00%> (-0.12%) ⬇️
autosklearn/estimators.py 93.38% <0.00%> (-0.10%) ⬇️
autosklearn/util/logging_.py 62.68% <0.00%> (-0.02%) ⬇️
... and 34 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba3f76f...4010f80. Read the comment docs.

@franchuterivera franchuterivera marked this pull request as ready for review June 18, 2021 08:49
Copy link
Contributor

@mfeurer mfeurer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good, thanks a lot.

However, it would be great to document this, for example as also suggested in #1009

It would be especially helpful to remind the user that this does not apply at prediction time.

@franchuterivera
Copy link
Contributor Author

What do you think if we document it like this recent push suggest?

@mfeurer
Copy link
Contributor

mfeurer commented Jun 30, 2021

Thanks, the documentation looks good to me. But unfortunately, there's currently a documentation build error. What do you think about additionally linking https://scikit-learn.org/stable/computing/parallelism.html ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants