Add faq #1109
Add faq #1109
Conversation
Codecov Report
@@ Coverage Diff @@
## development #1109 +/- ##
===============================================
- Coverage 85.82% 85.80% -0.02%
===============================================
Files 137 137
Lines 10625 10625
===============================================
- Hits 9119 9117 -2
- Misses 1506 1508 +2
Continue to review full report at Codecov.
|
doc/faq.rst
Outdated
| ``forkserver`` and ``spawn``. The default ``fork`` copies the whole process memory into the | ||
| subprocess. If the main process already uses 1.5GB of main memory and we apply a 3GB memory | ||
| limit to Auto-sklearn, it will only be able to use 1.5GB of that. We would have loved to use | ||
| ``forkserver`` or ``spawn`` instead, which both don't suffer from this issue (and have some |
There was a problem hiding this comment.
What is "this issue"? The mentioned link refers to deadlocks when using multi processing and how to solve them.
doc/faq.rst
Outdated
| memory limit and a time limit. To start such a process, Python gives three options: ``fork``, | ||
| ``forkserver`` and ``spawn``. The default ``fork`` copies the whole process memory into the | ||
| subprocess. If the main process already uses 1.5GB of main memory and we apply a 3GB memory | ||
| limit to Auto-sklearn, it will only be able to use 1.5GB of that. We would have loved to use |
There was a problem hiding this comment.
it -> executing a machine learning algorithm is limited to use at most 1.5GB.
doc/faq.rst
Outdated
|
|
||
| There are now two possible solutions: | ||
|
|
||
| 1. Use parallel Auto-sklearn: if you use Auto-sklean in parallel, it defaults to ``forkserver`` |
There was a problem hiding this comment.
Is there a link to an example or a hint how to do this? Is this just setting the n_jobs flag?
doc/faq.rst
Outdated
|
|
||
| We therefore suggest using one of the above settings by default. | ||
|
|
||
| Auto-sklearn is extremely memory hungry in a sequential setting |
There was a problem hiding this comment.
This is the same title as above
doc/faq.rst
Outdated
|
|
||
| When running Auto-sklearn in a parallel setting it starts new processes for evaluating machine | ||
| learning models using the ``forkserver`` mechanism. If not all code in the main script is guarded | ||
| by ``if __name__ == "__main__"`` it is executed for each subprocess. If now part of the code that |
There was a problem hiding this comment.
If now ... your RAM -> This sentence is hard to parse.
If the code loading your dataset is not guarded, it is executed for every evaluation of a machine learning algorithm and thus blocking your RAM.
doc/faq.rst
Outdated
| ----------------------------------- | ||
|
|
||
| In certain cases, for example for debugging, it can be helpful to limit the number of | ||
| models to try. We do not provide this as an argument in the API as we believe that it |
doc/manual.rst
Outdated
| @@ -77,6 +77,8 @@ For a full list please have a look at the source code (in `autosklearn/pipeline/ | |||
| * `Regressors <https://github.com/automl/auto-sklearn/tree/master/autosklearn/pipeline/components/regression>`_ | |||
| * `Preprocessors <https://github.com/automl/auto-sklearn/tree/master/autosklearn/pipeline/components/feature_preprocessing>`_ | |||
|
|
|||
| We do also provide an example `on how to restrict the classifiers to search over <examples/80_advanced/example_interpretable_models.html>`_. | |||
There was a problem hiding this comment.
I believe this is in "40_advanced"
No description provided.