Matthias Feurer: Fix calculate loss (#1123)

automl · Apr 12, 2021 · 25d78fc · 25d78fc
1 parent f08adf0
commit 25d78fc
Show file tree

Hide file tree

Showing 52 changed files with 499 additions and 345 deletions.
diff --git a/development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip b/development/_downloads/bc82bea3a5dd7bdba60b65220891d9e5/examples_python.zip
diff --git a/development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip b/development/_downloads/fb625db3c50d423b1b7881136ffdeec8/examples_jupyter.zip
diff --git a/development/_images/sphx_glr_example_pandas_train_test_001.png b/development/_images/sphx_glr_example_pandas_train_test_001.png
diff --git a/development/_images/sphx_glr_example_pandas_train_test_thumb.png b/development/_images/sphx_glr_example_pandas_train_test_thumb.png
diff --git a/development/_modules/autosklearn/metrics.html b/development/_modules/autosklearn/metrics.html
diff --git a/development/_sources/examples/20_basic/example_classification.rst.txt b/development/_sources/examples/20_basic/example_classification.rst.txt
@@ -355,7 +355,7 @@ Get the Score of the final ensemble
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 2 minutes  2.896 seconds)
+   **Total running time of the script:** ( 2 minutes  14.613 seconds)
 
 
 .. _sphx_glr_download_examples_20_basic_example_classification.py:

diff --git a/development/_sources/examples/20_basic/example_multilabel_classification.rst.txt b/development/_sources/examples/20_basic/example_multilabel_classification.rst.txt
@@ -235,7 +235,7 @@ Get the Score of the final ensemble
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 0 minutes  18.019 seconds)
+   **Total running time of the script:** ( 0 minutes  16.832 seconds)
 
 
 .. _sphx_glr_download_examples_20_basic_example_multilabel_classification.py:

diff --git a/development/_sources/examples/20_basic/example_regression.rst.txt b/development/_sources/examples/20_basic/example_regression.rst.txt
@@ -186,7 +186,7 @@ Get the Score of the final ensemble
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 1 minutes  58.682 seconds)
+   **Total running time of the script:** ( 2 minutes  8.146 seconds)
 
 
 .. _sphx_glr_download_examples_20_basic_example_regression.py:

diff --git a/development/_sources/examples/20_basic/sg_execution_times.rst.txt b/development/_sources/examples/20_basic/sg_execution_times.rst.txt
@@ -5,12 +5,12 @@
 
 Computation times
 =================
-**04:19.597** total execution time for **examples_20_basic** files:
+**04:39.591** total execution time for **examples_20_basic** files:
 
 +-------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_examples_20_basic_example_classification.py` (``example_classification.py``)                       | 02:02.896 | 0.0 MB |
+| :ref:`sphx_glr_examples_20_basic_example_classification.py` (``example_classification.py``)                       | 02:14.613 | 0.0 MB |
 +-------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_examples_20_basic_example_regression.py` (``example_regression.py``)                               | 01:58.682 | 0.0 MB |
+| :ref:`sphx_glr_examples_20_basic_example_regression.py` (``example_regression.py``)                               | 02:08.146 | 0.0 MB |
 +-------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_examples_20_basic_example_multilabel_classification.py` (``example_multilabel_classification.py``) | 00:18.019 | 0.0 MB |
+| :ref:`sphx_glr_examples_20_basic_example_multilabel_classification.py` (``example_multilabel_classification.py``) | 00:16.832 | 0.0 MB |
 +-------------------------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/development/_sources/examples/40_advanced/example_calc_multiple_metrics.rst.txt b/development/_sources/examples/40_advanced/example_calc_multiple_metrics.rst.txt
@@ -173,7 +173,7 @@ Get the Score of the final ensemble
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 2 minutes  3.318 seconds)
+   **Total running time of the script:** ( 1 minutes  57.389 seconds)
 
 
 .. _sphx_glr_download_examples_40_advanced_example_calc_multiple_metrics.py:

diff --git a/development/_sources/examples/40_advanced/example_feature_types.rst.txt b/development/_sources/examples/40_advanced/example_feature_types.rst.txt
@@ -158,7 +158,7 @@ Get the Score of the final ensemble
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 0 minutes  14.352 seconds)
+   **Total running time of the script:** ( 0 minutes  16.589 seconds)
 
 
 .. _sphx_glr_download_examples_40_advanced_example_feature_types.py:

diff --git a/development/_sources/examples/40_advanced/example_get_pipeline_components.rst.txt b/development/_sources/examples/40_advanced/example_get_pipeline_components.rst.txt
diff --git a/development/_sources/examples/40_advanced/example_metrics.rst.txt b/development/_sources/examples/40_advanced/example_metrics.rst.txt
@@ -363,17 +363,17 @@ Third example: Use own accuracy metric with additional argument
     Accuracy score 0.944056 using accu_add
     ################################################################################
     Use self defined error with additional argument
-    [WARNING] [2021-04-12 17:05:54,054:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
-    [WARNING] [2021-04-12 17:05:57,180:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
-    [WARNING] [2021-04-12 17:06:00,574:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
-    [WARNING] [2021-04-12 17:06:04,947:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
-    [WARNING] [2021-04-12 17:06:08,610:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
-    [WARNING] [2021-04-12 17:06:10,486:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
-    [WARNING] [2021-04-12 17:06:11,499:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
-    [WARNING] [2021-04-12 17:06:20,657:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
-    [WARNING] [2021-04-12 17:06:24,595:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
-    [WARNING] [2021-04-12 17:06:28,743:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
-    [WARNING] [2021-04-12 17:06:40,828:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
+    [WARNING] [2021-04-12 17:29:43,824:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
+    [WARNING] [2021-04-12 17:29:47,053:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
+    [WARNING] [2021-04-12 17:29:50,673:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
+    [WARNING] [2021-04-12 17:29:55,129:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
+    [WARNING] [2021-04-12 17:29:58,800:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
+    [WARNING] [2021-04-12 17:30:00,750:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
+    [WARNING] [2021-04-12 17:30:01,770:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
+    [WARNING] [2021-04-12 17:30:11,579:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
+    [WARNING] [2021-04-12 17:30:15,610:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
+    [WARNING] [2021-04-12 17:30:19,917:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
+    [WARNING] [2021-04-12 17:30:31,011:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
     Error rate 0.615385 using error_add
 
 
@@ -382,7 +382,7 @@ Third example: Use own accuracy metric with additional argument
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 5 minutes  22.864 seconds)
+   **Total running time of the script:** ( 5 minutes  6.577 seconds)
 
 
 .. _sphx_glr_download_examples_40_advanced_example_metrics.py:

diff --git a/development/_sources/examples/40_advanced/example_pandas_train_test.rst.txt b/development/_sources/examples/40_advanced/example_pandas_train_test.rst.txt
@@ -278,7 +278,7 @@ Plot the ensemble performance
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 1 minutes  59.472 seconds)
+   **Total running time of the script:** ( 1 minutes  59.784 seconds)
 
 
 .. _sphx_glr_download_examples_40_advanced_example_pandas_train_test.py:

diff --git a/development/_sources/examples/40_advanced/example_resampling.rst.txt b/development/_sources/examples/40_advanced/example_resampling.rst.txt
@@ -357,7 +357,7 @@ splitting it on the first feature.
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 6 minutes  6.164 seconds)
+   **Total running time of the script:** ( 6 minutes  12.380 seconds)
 
 
 .. _sphx_glr_download_examples_40_advanced_example_resampling.py:

diff --git a/development/_sources/examples/40_advanced/sg_execution_times.rst.txt b/development/_sources/examples/40_advanced/sg_execution_times.rst.txt
@@ -5,18 +5,18 @@
 
 Computation times
 =================
-**16:16.655** total execution time for **examples_40_advanced** files:
+**16:00.226** total execution time for **examples_40_advanced** files:
 
 +------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_examples_40_advanced_example_resampling.py` (``example_resampling.py``)                           | 06:06.164 | 0.0 MB |
+| :ref:`sphx_glr_examples_40_advanced_example_resampling.py` (``example_resampling.py``)                           | 06:12.380 | 0.0 MB |
 +------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_examples_40_advanced_example_metrics.py` (``example_metrics.py``)                                 | 05:22.864 | 0.0 MB |
+| :ref:`sphx_glr_examples_40_advanced_example_metrics.py` (``example_metrics.py``)                                 | 05:06.577 | 0.0 MB |
 +------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_examples_40_advanced_example_calc_multiple_metrics.py` (``example_calc_multiple_metrics.py``)     | 02:03.318 | 0.0 MB |
+| :ref:`sphx_glr_examples_40_advanced_example_pandas_train_test.py` (``example_pandas_train_test.py``)             | 01:59.784 | 0.0 MB |
 +------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_examples_40_advanced_example_pandas_train_test.py` (``example_pandas_train_test.py``)             | 01:59.472 | 0.0 MB |
+| :ref:`sphx_glr_examples_40_advanced_example_calc_multiple_metrics.py` (``example_calc_multiple_metrics.py``)     | 01:57.389 | 0.0 MB |
 +------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_examples_40_advanced_example_get_pipeline_components.py` (``example_get_pipeline_components.py``) | 00:30.486 | 0.0 MB |
+| :ref:`sphx_glr_examples_40_advanced_example_get_pipeline_components.py` (``example_get_pipeline_components.py``) | 00:27.507 | 0.0 MB |
 +------------------------------------------------------------------------------------------------------------------+-----------+--------+
-| :ref:`sphx_glr_examples_40_advanced_example_feature_types.py` (``example_feature_types.py``)                     | 00:14.352 | 0.0 MB |
+| :ref:`sphx_glr_examples_40_advanced_example_feature_types.py` (``example_feature_types.py``)                     | 00:16.589 | 0.0 MB |
 +------------------------------------------------------------------------------------------------------------------+-----------+--------+
diff --git a/...opment/_sources/examples/60_search/example_parallel_manual_spawning_cli.rst.txt b/...opment/_sources/examples/60_search/example_parallel_manual_spawning_cli.rst.txt
@@ -301,7 +301,7 @@ Start Auto-sklearn
  .. code-block:: none
 
     auto-sklearn results:
-      Dataset name: 561b9cd4-9bb2-11eb-8710-a7ef821b6498
+      Dataset name: aaa4768f-9bb5-11eb-87ae-911805395344
       Metric: accuracy
       Best validation score: 0.978723
       Number of target algorithm runs: 13
@@ -342,7 +342,7 @@ line.
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 0 minutes  39.212 seconds)
+   **Total running time of the script:** ( 0 minutes  40.362 seconds)
 
 
 .. _sphx_glr_download_examples_60_search_example_parallel_manual_spawning_cli.py:

diff --git a/...ent/_sources/examples/60_search/example_parallel_manual_spawning_python.rst.txt b/...ent/_sources/examples/60_search/example_parallel_manual_spawning_python.rst.txt
@@ -210,28 +210,28 @@ which means that it is automatically stopped once all computation is done.
 
  .. code-block:: none
 
-    [ERROR] [2021-04-12 17:13:47,884:asyncio] _GatheringFuture exception was never retrieved
+    [ERROR] [2021-04-12 17:37:39,346:asyncio] _GatheringFuture exception was never retrieved
     future: <_GatheringFuture finished exception=CancelledError()>
     asyncio.exceptions.CancelledError
     auto-sklearn results:
-      Dataset name: 6cf8b110-9bb2-11eb-8710-a7ef821b6498
+      Dataset name: c22f1231-9bb5-11eb-87ae-911805395344
       Metric: accuracy
       Best validation score: 0.978723
-      Number of target algorithm runs: 13
-      Number of successful target algorithm runs: 12
+      Number of target algorithm runs: 14
+      Number of successful target algorithm runs: 13
       Number of crashed target algorithm runs: 0
       Number of target algorithms that exceeded the time limit: 1
       Number of target algorithms that exceeded the memory limit: 0
 
-    Accuracy score 0.9440559440559441
+    Accuracy score 0.951048951048951
 
 
 
 
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 0 minutes  46.763 seconds)
+   **Total running time of the script:** ( 0 minutes  46.789 seconds)
 
 
 .. _sphx_glr_download_examples_60_search_example_parallel_manual_spawning_python.py:

diff --git a/development/_sources/examples/60_search/example_parallel_n_jobs.rst.txt b/development/_sources/examples/60_search/example_parallel_n_jobs.rst.txt
@@ -114,10 +114,10 @@ To use ``n_jobs_`` we must guard the code
       Dataset name: breast_cancer
       Metric: accuracy
       Best validation score: 0.992908
-      Number of target algorithm runs: 49
+      Number of target algorithm runs: 48
       Number of successful target algorithm runs: 44
       Number of crashed target algorithm runs: 1
-      Number of target algorithms that exceeded the time limit: 4
+      Number of target algorithms that exceeded the time limit: 3
       Number of target algorithms that exceeded the memory limit: 0
 
 
@@ -127,7 +127,7 @@ To use ``n_jobs_`` we must guard the code
 
 .. rst-class:: sphx-glr-timing
 
-   **Total running time of the script:** ( 2 minutes  5.060 seconds)
+   **Total running time of the script:** ( 2 minutes  11.232 seconds)
 
 
 .. _sphx_glr_download_examples_60_search_example_parallel_n_jobs.py: