Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve logging and remove deadlock #1012

Merged
merged 4 commits into from
Nov 23, 2020
Merged

Improve logging and remove deadlock #1012

merged 4 commits into from
Nov 23, 2020

Conversation

mfeurer
Copy link
Contributor

@mfeurer mfeurer commented Nov 23, 2020

  • use forkserver context to start processes within the pynisher. This should remove a deadlock due to the use of logging, threading and multiprocessing. As a side-effect, we have to bump the minimal pynisher version to 0.6.2.
  • log messages from dask.distributed to a separate logfile
  • change a few log message severities or remove the log message alltogether
  • randomly try IPs for the logging server inside try/except. Previously, the ID would be determined before starting the server, and therefore the IP could be taken by a different process in the meantime.
  • Fix a bug in model deletion (if mem usage too high or max model on disk reached).
  • improve output of metadata generation

Failing tests are due to openml/OpenML#1078

* use forkserver context to start processes within the pynisher.
  This should remove a deadlock due to the use of logging, threading and
  multiprocessing. As a side-effect, we have to bump the minimal
  pynisher version to 0.6.2.
* log messages from dask.distributed to a separate logfile
* change a few log message severities or remove the log message
  alltogether
1. randomly try IPs for the logging server inside try/except.
   Previously, the ID would be determined before starting the server,
   and therefore the IP could be taken by a different process in the
   meantime.
2. Fix a bug in model deletion (if mem usage too high or max model on
   disk reached).
3. improve output of metadata generation
@mfeurer mfeurer changed the title Improve logging Improve logging and remove deadlock Nov 23, 2020
@codecov
Copy link

codecov bot commented Nov 23, 2020

Codecov Report

Merging #1012 (869feec) into development (168330e) will decrease coverage by 0.48%.
The diff coverage is 82.50%.

Impacted file tree graph

@@               Coverage Diff               @@
##           development    #1012      +/-   ##
===============================================
- Coverage        85.16%   84.67%   -0.49%     
===============================================
  Files              125      125              
  Lines             9820     9826       +6     
===============================================
- Hits              8363     8320      -43     
- Misses            1457     1506      +49     
Impacted Files Coverage Δ
autosklearn/estimators.py 92.12% <ø> (-0.79%) ⬇️
autosklearn/smbo.py 72.41% <ø> (-10.00%) ⬇️
autosklearn/util/logging_.py 72.63% <55.55%> (-1.06%) ⬇️
autosklearn/automl.py 84.49% <87.50%> (-0.39%) ⬇️
autosklearn/ensemble_builder.py 76.57% <87.50%> (+0.61%) ⬆️
autosklearn/evaluation/__init__.py 80.00% <100.00%> (-2.63%) ⬇️
autosklearn/metalearning/input/aslib_simple.py 95.45% <100.00%> (ø)
autosklearn/metalearning/mismbo.py 100.00% <100.00%> (ø)
autosklearn/util/pipeline.py 76.56% <0.00%> (-6.25%) ⬇️
... and 8 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 168330e...869feec. Read the comment docs.

@mfeurer mfeurer merged commit f730304 into development Nov 23, 2020
@mfeurer mfeurer deleted the improve_logging branch November 23, 2020 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant