GSoC '16 Testing Project, prelim task: skip slow tests (not to be merged) #508

kdexd · 2016-03-06T06:44:14Z

This PR is for discussions related to my preliminary task for Tardis Testing project (TEP001).
The task was to mark the full Tardis test as slow and skip it automatically during normal run of tests, for example at Travis CI, and include slow tests only while running on a dedicated server.
The full Tardis test is marked as slow and will only run when --run-slow parameter is provided while running the tests. In a nutshell:
- ./setup.py test will skip full Tardis test inherently, no special parameters to provide.
- ./setup.py test --run-slow will include Tardis slow test while running.
A brief description is provided as a note in the diffs below, please refer to it.

kdexd · 2016-03-06T06:46:37Z

tardis/conftest.py

@@ -2,6 +2,9 @@


 def pytest_addoption(parser):
+    parser.addoption("--run-slow", action="store_true",
+                     help="include running slow tests during run")
+


An option for detecting the command line flag --run-slow has to be added in conftest.py. The task of detecting it is silently handled by pytest behind the curtains. 😄

- If --run-slow flag is not provided with py.test, the full tardis test will automatically be skipped. It will be run along only with --run-slow flag.

- The script should be run with a trusted version of TARDIS and it will create two .npz files - each made from the ndarrays and astropy quantities of obtained radial1d model from a run with Stratified W7 setup. - The config file alongwith the abundancy and density profiles, and also the output .npz files are kept together under a directory 'tardis/tests/data/w7_13d'.

- setup method initializes the filepaths to the config file and atom data file, brings in the benchmark data and performs a run. - methods to perform asserions for j_estimators and j_blue_estimators have been added.

- Some changes are made in setup() method of the integration test - the description about it is now updated in the block comments.

- numpy.ndarray *= astropy.units.Unit() was changed to numpy.ndarray=numpy.ndarray*astropy.units.Unit().

kdexd · 2016-03-13T09:24:40Z

The slow integration test for Stratified W7 Setup now works fine!

I have made a lot of assertions - I currently have very little idea about the quantities involved in calculation so I might have missed/ included redundant assertions. Current set of assertions made as loosely adapted from test_tardis_full.py.
tardis_w7.yml, w7_densitites.dat, w7_abundancies.dat were collected from Add data directory for TEP 001 - example setups for slow tests tep#13
A believed-stable version of TARDIS was used to execute a single run with the Stratified W7 setup and benchmark data was collected for further assertions.
expected_ndarrays.npz is a collection of parameters in the radial1d_model, whose type is numpy.ndarray. The file size is 35.4 MB and it contains these parameters from the radial1d_model:
- last_interaction_type
- last_line_interaction_out_id
- last_line_interaction_in_id
- j_estimators
- j_blue_estimators
- last_line_interaction_shell_id
- nubar_estimators
- ws
expected_astropy_quantities.npz is a collection of parameters in the radial1d_model, whose type is 'astropy.units.quantity.Quantity, though only the value is stored and the unit of quantity is multiplied back before making assertions. The file size is 3.9 MB and it contains these parameters from theradial1d_model`:
- t_rads
- luminosity_inner
- montecarlo_luminosity
- montecarlo_virtual_luminosity
- time_of_simulation
- montecarlo_nu
- last_line_interaction_angstrom
- j_blues_norm_factor
I have tested my code - just one integration test and here is the screenshot after the final run:

- Individual modules were replaced and wrapped nicely inside a 'Collector' class. This class performs run of tardis against a setup and collects benchmark data for performing assertions in the future. - Updated documentation for better understanding and the code which was specific to w7 setup has been replaced to do the same job for any setup.

- base.py contains most of the code from previous version of test_w7.py and now test_w7.py has TestW& which simply is inheriting from class SlowTests of base.py and super calls are executed.

- The code is very much similar to integration test using W7 setup. A better way will be the use of parametrization, but doing it now will become a hindrance when some setup not much similar to these two is introduced. - So right now the integration test is made as the test of W7 setup - inheriting from base class and making super() calls.

unoebauer · 2016-05-10T14:50:41Z

@karandesai-96 - is this PR still needed? If not, we should close it.

kdexd · 2016-05-10T15:59:59Z

This PR had configuration files for AT setup as well, but I have the branch locally. Whatever building blocks were required, have been merged into master. Closing it finally.

kdexd changed the title ~~GSoC '16 Testing prelim task: skip slow tests (not to be merged)~~ GSoC '16 Testing Project, prelim task: skip slow tests (not to be merged) Mar 6, 2016

kdexd reviewed Mar 6, 2016
View reviewed changes

Marks tardis full test as slow and enables skipping it.

e5c4fbc

- If --run-slow flag is not provided with py.test, the full tardis test will automatically be skipped. It will be run along only with --run-slow flag.

kdexd force-pushed the skip-slow branch 2 times, most recently from 1fc82cd to e5c4fbc Compare March 12, 2016 04:45

kdexd force-pushed the skip-slow branch 2 times, most recently from 8b93732 to bc2a601 Compare March 12, 2016 18:21

Add integration test for Stratified W7 setup.

0f7367d

- setup method initializes the filepaths to the config file and atom data file, brings in the benchmark data and performs a run. - methods to perform asserions for j_estimators and j_blue_estimators have been added.

kdexd force-pushed the skip-slow branch from 300fc15 to 0f7367d Compare March 12, 2016 18:35

Add all the numpy ndarray assertions.

5bd18c9

kdexd force-pushed the skip-slow branch from 5affe08 to 5bd18c9 Compare March 12, 2016 18:52

Add tests for montecarlo properties, shell temperature

00784be

kdexd force-pushed the skip-slow branch 4 times, most recently from efa0b73 to e6cb426 Compare March 13, 2016 06:25

Relocate test_w7.py and add documentation.

23a84dd

- Some changes are made in setup() method of the integration test - the description about it is now updated in the block comments.

kdexd force-pushed the skip-slow branch from e6cb426 to 23a84dd Compare March 13, 2016 07:04

Fix TypeError by replacing syntax of multiplying.

9e73306

- numpy.ndarray *= astropy.units.Unit() was changed to numpy.ndarray=numpy.ndarray*astropy.units.Unit().

Reform directory structure to make generalization.

7dadfc5

kdexd force-pushed the skip-slow branch from 8ee7a4d to 7dadfc5 Compare March 14, 2016 19:08

karandesai-96 added 5 commits March 15, 2016 00:40

Add documentation and generalize code in w7_collect.py

f4ad8f1

Add generalized base class for building slow tests classes.

716d7b8

- base.py contains most of the code from previous version of test_w7.py and now test_w7.py has TestW& which simply is inheriting from class SlowTests of base.py and super calls are executed.

Add configuration and benchmark data for AT setup.

c3b6d44

kdexd force-pushed the skip-slow branch from 03f036f to b3064f3 Compare March 25, 2016 08:07

kdexd mentioned this pull request May 10, 2016

Integration Test class for Stratified W7 setup. #558

Merged

4 tasks

kdexd closed this May 10, 2016

kdexd deleted the skip-slow branch May 10, 2016 16:05

kdexd restored the skip-slow branch June 20, 2016 12:23

kdexd deleted the skip-slow branch June 28, 2016 11:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GSoC '16 Testing Project, prelim task: skip slow tests (not to be merged) #508

GSoC '16 Testing Project, prelim task: skip slow tests (not to be merged) #508

kdexd commented Mar 6, 2016

kdexd Mar 6, 2016

kdexd commented Mar 13, 2016

unoebauer commented May 10, 2016

kdexd commented May 10, 2016 •

edited

Loading

GSoC '16 Testing Project, prelim task: skip slow tests (not to be merged) #508

GSoC '16 Testing Project, prelim task: skip slow tests (not to be merged) #508

Conversation

kdexd commented Mar 6, 2016

kdexd Mar 6, 2016

Choose a reason for hiding this comment

kdexd commented Mar 13, 2016

The slow integration test for Stratified W7 Setup now works fine!

unoebauer commented May 10, 2016

kdexd commented May 10, 2016 • edited Loading

kdexd commented May 10, 2016 •

edited

Loading