Skip to content

Commit

Permalink
Eddie Bergman: Extends github action pytest to allow for tests to r…
Browse files Browse the repository at this point in the history
…un for 60m (up from 45m) (#1239)
  • Loading branch information
Github Actions committed Sep 10, 2021
1 parent 8d8bed7 commit af48d05
Show file tree
Hide file tree
Showing 67 changed files with 691 additions and 845 deletions.
Binary file not shown.
Binary file not shown.
Binary file modified development/_images/sphx_glr_example_inspect_predictions_001.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified development/_images/sphx_glr_example_inspect_predictions_thumb.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified development/_images/sphx_glr_example_pandas_train_test_001.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -157,7 +157,7 @@ View the models found by auto-sklearn
rank ensemble_weight type cost duration
model_id
2 1 1.0 random_forest 0.447294 4.004269
2 1 1.0 random_forest 0.447294 4.467062
Expand Down Expand Up @@ -265,7 +265,7 @@ Get the Score of the final ensemble
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 0 minutes 15.601 seconds)
**Total running time of the script:** ( 0 minutes 18.682 seconds)


.. _sphx_glr_download_examples_20_basic_example_multilabel_classification.py:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -122,9 +122,9 @@ View the models found by auto-sklearn

.. code-block:: none
rank ensemble_weight type cost duration
model_id
11 1 1.0 gaussian_process 6.213718e-10 6.625976
rank ensemble_weight type cost duration
model_id
11 1 1.0 gaussian_process 0.011395 5.272071
Expand All @@ -151,7 +151,7 @@ Print the final ensemble constructed by auto-sklearn

.. code-block:: none
[(1.000000, SimpleRegressionPipeline({'data_preprocessor:__choice__': 'feature_type', 'feature_preprocessor:__choice__': 'no_preprocessing', 'regressor:__choice__': 'gaussian_process', 'data_preprocessor:feature_type:categorical_transformer:categorical_encoding:__choice__': 'one_hot_encoding', 'data_preprocessor:feature_type:categorical_transformer:category_coalescence:__choice__': 'minority_coalescer', 'data_preprocessor:feature_type:numerical_transformer:imputation:strategy': 'most_frequent', 'data_preprocessor:feature_type:numerical_transformer:rescaling:__choice__': 'robust_scaler', 'regressor:gaussian_process:alpha': 1.6650724498551164e-11, 'regressor:gaussian_process:thetaL': 5.222627524598125e-10, 'regressor:gaussian_process:thetaU': 5801.524168449955, 'data_preprocessor:feature_type:categorical_transformer:category_coalescence:minority_coalescer:minimum_fraction': 0.0809214996879808, 'data_preprocessor:feature_type:numerical_transformer:rescaling:robust_scaler:q_max': 0.9020530113478731, 'data_preprocessor:feature_type:numerical_transformer:rescaling:robust_scaler:q_min': 0.10288728233229412},
[(1.000000, SimpleRegressionPipeline({'data_preprocessor:__choice__': 'feature_type', 'feature_preprocessor:__choice__': 'no_preprocessing', 'regressor:__choice__': 'gaussian_process', 'data_preprocessor:feature_type:categorical_transformer:categorical_encoding:__choice__': 'encoding', 'data_preprocessor:feature_type:categorical_transformer:category_coalescence:__choice__': 'minority_coalescer', 'data_preprocessor:feature_type:numerical_transformer:imputation:strategy': 'median', 'data_preprocessor:feature_type:numerical_transformer:rescaling:__choice__': 'quantile_transformer', 'regressor:gaussian_process:alpha': 0.00017904812123612402, 'regressor:gaussian_process:thetaL': 8.77129700430336e-07, 'regressor:gaussian_process:thetaU': 5761.3477044417605, 'data_preprocessor:feature_type:categorical_transformer:category_coalescence:minority_coalescer:minimum_fraction': 0.10003181055971523, 'data_preprocessor:feature_type:numerical_transformer:rescaling:quantile_transformer:n_quantiles': 1256, 'data_preprocessor:feature_type:numerical_transformer:rescaling:quantile_transformer:output_distribution': 'normal'},
dataset_properties={
'task': 5,
'sparse': False,
Expand Down Expand Up @@ -186,7 +186,7 @@ Get the Score of the final ensemble

.. code-block:: none
R2 score: 0.9999999974470909
R2 score: 0.994774606994838
Expand Down Expand Up @@ -409,7 +409,7 @@ Get the configuration space
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 1 minutes 57.654 seconds)
**Total running time of the script:** ( 1 minutes 56.272 seconds)


.. _sphx_glr_download_examples_20_basic_example_multioutput_regression.py:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -121,11 +121,11 @@ View the models found by auto-sklearn
rank ensemble_weight type cost duration
model_id
25 1 0.46 sgd 0.436679 0.742132
6 2 0.32 ard_regression 0.455042 0.777667
27 3 0.14 ard_regression 0.462249 0.743963
11 4 0.02 random_forest 0.507400 10.484822
7 5 0.06 gradient_boosting 0.518673 1.468356
25 1 0.46 sgd 0.436679 0.844572
6 2 0.32 ard_regression 0.455042 0.830707
27 3 0.14 ard_regression 0.462249 0.827883
11 4 0.02 random_forest 0.507400 10.990755
7 5 0.06 gradient_boosting 0.518673 1.583698
Expand Down Expand Up @@ -266,7 +266,7 @@ the true value).

.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 1 minutes 55.377 seconds)
**Total running time of the script:** ( 1 minutes 57.715 seconds)


.. _sphx_glr_download_examples_20_basic_example_regression.py:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@

Computation times
=================
**06:06.025** total execution time for **examples_20_basic** files:
**06:15.108** total execution time for **examples_20_basic** files:

+-------------------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_examples_20_basic_example_multioutput_regression.py` (``example_multioutput_regression.py``) | 01:57.654 | 0.0 MB |
| :ref:`sphx_glr_examples_20_basic_example_classification.py` (``example_classification.py``) | 02:02.438 | 0.0 MB |
+-------------------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_examples_20_basic_example_classification.py` (``example_classification.py``) | 01:57.393 | 0.0 MB |
| :ref:`sphx_glr_examples_20_basic_example_regression.py` (``example_regression.py``) | 01:57.715 | 0.0 MB |
+-------------------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_examples_20_basic_example_regression.py` (``example_regression.py``) | 01:55.377 | 0.0 MB |
| :ref:`sphx_glr_examples_20_basic_example_multioutput_regression.py` (``example_multioutput_regression.py``) | 01:56.272 | 0.0 MB |
+-------------------------------------------------------------------------------------------------------------------+-----------+--------+
| :ref:`sphx_glr_examples_20_basic_example_multilabel_classification.py` (``example_multilabel_classification.py``) | 00:15.601 | 0.0 MB |
| :ref:`sphx_glr_examples_20_basic_example_multilabel_classification.py` (``example_multilabel_classification.py``) | 00:18.682 | 0.0 MB |
+-------------------------------------------------------------------------------------------------------------------+-----------+--------+
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ Get the Score of the final ensemble
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 2 minutes 2.337 seconds)
**Total running time of the script:** ( 2 minutes 12.088 seconds)


.. _sphx_glr_download_examples_40_advanced_example_calc_multiple_metrics.py:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -174,12 +174,12 @@ Build and fit a classifier

.. code-block:: none
2021-09-10 13:47:41,171 - Client-EnsembleBuilder - INFO - DummyFuture: ([{'Timestamp': Timestamp('2021-09-10 13:47:41.154453'), 'ensemble_optimization_score': 0.9886363636363636, 'ensemble_test_score': 0.9899874843554443}], 50, None, None, None)/SingleThreadedClient() Started Ensemble builder job at 2021.09.10-13.47.41 for iteration 0.
2021-09-10 13:47:47,643 - Client-EnsembleBuilder - INFO - DummyFuture: ([], 50, None, None, None)/SingleThreadedClient() Started Ensemble builder job at 2021.09.10-13.47.47 for iteration 1.
2021-09-10 13:50:05,479 - Client-EnsembleBuilder - INFO - DummyFuture: ([{'Timestamp': Timestamp('2021-09-10 13:50:05.460828'), 'ensemble_optimization_score': 0.47853535353535354, 'ensemble_test_score': 0.47434292866082606}], 50, None, None, None)/SingleThreadedClient() Started Ensemble builder job at 2021.09.10-13.50.05 for iteration 0.
2021-09-10 13:50:12,227 - Client-EnsembleBuilder - INFO - DummyFuture: ([], 50, None, None, None)/SingleThreadedClient() Started Ensemble builder job at 2021.09.10-13.50.12 for iteration 1.
tmp_folder/.auto-sklearn
tmp_folder/space.json
tmp_folder/AutoML(1):fd616029-123d-11ec-8668-e1b8f8e61eaf.log
tmp_folder/distributed.log
tmp_folder/AutoML(1):a8633c52-123d-11ec-8642-6dc8225d4641.log
tmp_folder/smac3-output
Expand All @@ -188,7 +188,7 @@ Build and fit a classifier
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 0 minutes 18.529 seconds)
**Total running time of the script:** ( 0 minutes 20.726 seconds)


.. _sphx_glr_download_examples_40_advanced_example_debug_logging.py:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ Get the Score of the final ensemble
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 0 minutes 14.761 seconds)
**Total running time of the script:** ( 0 minutes 15.459 seconds)


.. _sphx_glr_download_examples_40_advanced_example_feature_types.py:
Expand Down

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ accelerometer and gyroscope data collected by a phone. For more information see

.. code-block:: none
Train score 0.9954285714285714
Train score 0.9951428571428571
Test score 0.982
Expand Down Expand Up @@ -143,12 +143,12 @@ in the `scikit-learn docs <https://scikit-learn.org/stable/modules/permutation_i

.. code-block:: none
gyro_y : -0.000 +/- 0.002
gyro_x : 0.019 +/- 0.001
gyro_z : 0.037 +/- 0.004
acceleration_x: 0.058 +/- 0.007
acceleration_z: 0.138 +/- 0.005
acceleration_y: 0.266 +/- 0.008
gyro_y : 0.000 +/- 0.001
gyro_x : 0.019 +/- 0.002
gyro_z : 0.036 +/- 0.004
acceleration_x: 0.066 +/- 0.008
acceleration_z: 0.130 +/- 0.005
acceleration_y: 0.267 +/- 0.008
Expand Down Expand Up @@ -230,7 +230,7 @@ these features. Again, we'll look at acceleration_y and acceleration_z.

.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 3 minutes 53.925 seconds)
**Total running time of the script:** ( 3 minutes 58.328 seconds)


.. _sphx_glr_download_examples_40_advanced_example_inspect_predictions.py:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -274,7 +274,7 @@ Get the Score of the final ensemble
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 1 minutes 55.647 seconds)
**Total running time of the script:** ( 1 minutes 56.130 seconds)


.. _sphx_glr_download_examples_40_advanced_example_interpretable_models.py:
Expand Down
52 changes: 25 additions & 27 deletions development/_sources/examples/40_advanced/example_metrics.rst.txt
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ First example: Use predefined accuracy metric
################################################################################
Use predefined accuracy metric
Accuracy score 0.944 using accuracy
Accuracy score 0.951 using accuracy
Expand Down Expand Up @@ -254,7 +254,7 @@ Second example: Use own accuracy metric
################################################################################
Use self defined accuracy metric
Accuracy score 0.951 using accu
Accuracy score 0.944 using accu
Expand Down Expand Up @@ -304,17 +304,16 @@ Third example: Use own error metric
################################################################################
Use self defined error metric
[WARNING] [2021-09-10 13:59:00,207:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 13:59:01,560:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 13:59:05,728:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 13:59:06,782:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 13:59:08,003:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 13:59:12,820:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 13:59:15,074:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 13:59:19,858:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 13:59:25,007:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 13:59:31,009:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
Error score -0.049 using error
[WARNING] [2021-09-10 14:01:51,419:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:01:52,678:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:01:56,975:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:01:58,095:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:01:59,395:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:02:04,642:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:02:06,858:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:02:12,100:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:02:17,954:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
Error score -0.056 using error
Expand Down Expand Up @@ -365,7 +364,7 @@ Fourth example: Use own accuracy metric with additional argument
################################################################################
Use self defined accuracy with additional argument
Accuracy score 0.951 using accu_add
Accuracy score 0.944 using accu_add
Expand Down Expand Up @@ -415,18 +414,17 @@ Fifth example: Use own accuracy metric with additional argument
################################################################################
Use self defined error with additional argument
[WARNING] [2021-09-10 14:00:45,951:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:00:50,227:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:00:57,619:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:00:58,740:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:01:02,678:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:01:03,653:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:01:04,799:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:01:11,194:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:01:16,254:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:01:21,841:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:01:24,377:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:01:30,530:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:03:33,988:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:03:38,525:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:03:46,598:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:03:47,780:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:03:52,006:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:03:53,045:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:03:54,249:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:04:01,176:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:04:06,618:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:04:12,524:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
[WARNING] [2021-09-10 14:04:15,272:smac.runhistory.runhistory2epm.RunHistory2EPM4LogCost] Got cost of smaller/equal to 0. Replace by 0.000010 since we use log cost.
Error score 0.615 using error_add
Expand All @@ -435,7 +433,7 @@ Fifth example: Use own accuracy metric with additional argument
.. rst-class:: sphx-glr-timing

**Total running time of the script:** ( 4 minutes 53.064 seconds)
**Total running time of the script:** ( 4 minutes 51.133 seconds)


.. _sphx_glr_download_examples_40_advanced_example_metrics.py:
Expand Down
Loading

0 comments on commit af48d05

Please sign in to comment.