Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up testing #930

Merged
merged 42 commits into from
Apr 4, 2024
Merged
Show file tree
Hide file tree
Changes from 39 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
bdb1fe1
Add missing test markers
f0uriest Mar 6, 2024
ee6b96b
Remove redundant tests
f0uriest Mar 6, 2024
fc48bd6
Refactor some tests to use parametrized testing
f0uriest Mar 6, 2024
929ae9f
Remove more redundant tests
f0uriest Mar 6, 2024
a9a3662
Move some slow tests from unit to regression
f0uriest Mar 6, 2024
f087284
Ensure consistent order for parameterized tests
f0uriest Mar 6, 2024
f468226
Add fixture to clear jax caches to reduce memory usage
f0uriest Mar 8, 2024
38cfca9
Reformat worklow ymls
f0uriest Mar 8, 2024
4368d53
Add pytest-xdist to further parallelize tests
f0uriest Mar 8, 2024
14dd706
Try to get xdist to play nice with cov
f0uriest Mar 8, 2024
55902cd
Revert "Add pytest-xdist to further parallelize tests"
f0uriest Mar 8, 2024
1260a96
Revert "Try to get xdist to play nice with cov"
f0uriest Mar 8, 2024
7cbb0bd
Remove eq solves from unit tests
f0uriest Mar 8, 2024
808f7d7
Remove unneeded tests and fixtures
f0uriest Mar 8, 2024
5c67906
Remove plot_field_lines
f0uriest Mar 8, 2024
7b2fafc
Don't jit objectives that will only be used once in tests
f0uriest Mar 9, 2024
f121cff
Refactor test for scalar resolution
f0uriest Mar 9, 2024
f9b4fc4
Merge branch 'master' into rc/tests
f0uriest Mar 9, 2024
6b0e4c7
Re-balance split test loading
f0uriest Mar 9, 2024
acba92f
Update baseline images, example DSHAPE is higher res than test one
f0uriest Mar 9, 2024
10caf60
Raise warnings to errors to stop earlier where possible
f0uriest Mar 11, 2024
8145bbd
Remove ESTELL regression test
f0uriest Mar 11, 2024
9bcfd2e
Reduce equilibrium resolution in unit tests
f0uriest Mar 11, 2024
c465d31
Remove plot_field_lines_sfl from docs
f0uriest Mar 11, 2024
09bb7c9
Cleanup
f0uriest Mar 12, 2024
4eca9a8
Use different splitting alg to keep related tests together
f0uriest Mar 12, 2024
09d4da0
Reorganize some stuff
f0uriest Mar 12, 2024
732fb9f
Merge branch 'master' into rc/tests
f0uriest Mar 14, 2024
171fd91
Fix typo
f0uriest Mar 16, 2024
97df4e7
Merge branch 'master' into rc/tests
YigitElma Mar 17, 2024
c571dcc
Merge branch 'master' into rc/tests
f0uriest Mar 27, 2024
d11e6b5
Merge branch 'master' into rc/tests
f0uriest Mar 28, 2024
6b85c03
Fix tests for free boundary and omnigenity
f0uriest Mar 28, 2024
6c8cb49
Update tests/test_plotting.py
f0uriest Mar 28, 2024
f601dbc
Merge branch 'master' into rc/tests
f0uriest Mar 28, 2024
7c23bf9
Merge branch 'master' into rc/tests
f0uriest Apr 1, 2024
272f184
Reorganize plotting tests, remove redundant ones
f0uriest Apr 2, 2024
60a2216
Remove more redundant tests
f0uriest Apr 2, 2024
89849fb
Switch splitting algorithm to avoid groups of 0 tests
f0uriest Apr 2, 2024
de19b7a
Merge branch 'master' into rc/tests
dpanici Apr 2, 2024
763898b
Merge branch 'master' into rc/tests
f0uriest Apr 3, 2024
56a7c4c
Fix int in resolution test
f0uriest Apr 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions .github/workflows/benchmark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ on:
inputs:
debug_enabled:
type: boolean
description: 'Run the build with tmate debugging enabled (https://github.com/marketplace/actions/debugging-with-tmate)'
description: '(https://github.com/marketplace/actions/debugging-with-tmate)'
required: false
default: false

Expand Down Expand Up @@ -44,7 +44,10 @@ jobs:
pwd
lscpu
cd tests/benchmarks
python -m pytest benchmark_cpu_small.py --benchmark-save='Latest_Commit' -vv --durations=0 --benchmark-save-data
python -m pytest benchmark_cpu_small.py -vv \
--benchmark-save='Latest_Commit' \
--durations=0 \
--benchmark-save-data
- name: Checkout current master
uses: actions/checkout@v4
with:
Expand All @@ -57,7 +60,10 @@ jobs:
pwd
lscpu
cd tests/benchmarks
python -m pytest benchmark_cpu_small.py --benchmark-save='master' -vv --durations=0 --benchmark-save-data
python -m pytest benchmark_cpu_small.py -vv \
--benchmark-save='master' \
--durations=0 \
--benchmark-save-data
- name: put benchmark results in same folder
run: |
pwd
Expand Down Expand Up @@ -88,6 +94,3 @@ jobs:
with:
name: benchmark_artifact
path: tests/benchmarks/.benchmarks
- name: Setup tmate session
if: ${{ failure() }}
uses: mxschmitt/action-tmate@v3
18 changes: 16 additions & 2 deletions .github/workflows/jax_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,15 @@ jobs:
strategy:
fail-fast: false
matrix:
jax-version: [0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5, 0.3.6, 0.3.7, 0.3.8, 0.3.9, 0.3.10, 0.3.11, 0.3.12, 0.3.13, 0.3.14, 0.3.15, 0.3.16, 0.3.17, 0.3.19, 0.3.20, 0.3.21, 0.3.22, 0.3.23, 0.3.24, 0.3.25, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.4.5, 0.4.6, 0.4.7, 0.4.8, 0.4.9, 0.4.10, 0.4.11, 0.4.12, 0.4.13, 0.4.14]
jax-version: [0.3.0, 0.3.1, 0.3.2, 0.3.3, 0.3.4, 0.3.5,
0.3.6, 0.3.7, 0.3.8, 0.3.9, 0.3.10, 0.3.11,
0.3.12, 0.3.13, 0.3.14, 0.3.15, 0.3.16, 0.3.17,
0.3.19, 0.3.20, 0.3.21, 0.3.22, 0.3.23, 0.3.24,
0.3.25, 0.4.1, 0.4.2, 0.4.3, 0.4.4, 0.4.5,
0.4.6, 0.4.7, 0.4.8, 0.4.9, 0.4.10, 0.4.11,
0.4.12, 0.4.13, 0.4.14, 0.4.16, 0.4.17, 0.4.18,
0.4.18, 0.4.19, 0.4.20, 0.4.21, 0.4.22, 0.4.23,
0.4.24, 0.4.25]
group: [1, 2]
steps:
- uses: actions/checkout@v4
Expand All @@ -36,4 +44,10 @@ jobs:
run: |
pwd
lscpu
python -m pytest -m unit --durations=0 --mpl --maxfail=1 --splits 3 --group ${{ matrix.group }} --splitting-algorithm least_duration
python -m pytest -m unit \
--durations=0 \
--mpl \
--maxfail=1 \
--splits 3 \
--group ${{ matrix.group }} \
--splitting-algorithm least_duration
7 changes: 6 additions & 1 deletion .github/workflows/nbtests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ jobs:
strategy:
matrix:
python-version: ['3.10']
group: [1, 2]

steps:
- uses: actions/checkout@v4
Expand All @@ -37,4 +38,8 @@ jobs:
pwd
lscpu
export PYTHONPATH=$(pwd)
pytest -v --nbmake "./docs/notebooks" --nbmake-timeout=1000 --ignore=./docs/notebooks/zernike_eval.ipynb --ignore=./docs/notebooks/DESC_Solve_from_pyQSC.ipynb
pytest -v --nbmake "./docs/notebooks" \
--nbmake-timeout=1000 \
--ignore=./docs/notebooks/zernike_eval.ipynb \
--splits 2 \
--group ${{ matrix.group }} \
13 changes: 12 additions & 1 deletion .github/workflows/regression_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,18 @@ jobs:
run: |
pwd
lscpu
python -m pytest -v -m regression --durations=0 --cov-report xml:cov.xml --cov-config=setup.cfg --cov=desc/ --mpl --mpl-results-path=mpl_results.html --mpl-generate-summary=html --splits 6 --group ${{ matrix.group }} --splitting-algorithm least_duration --db ./prof.db
python -m pytest -v -m regression \
--durations=0 \
--cov-report xml:cov.xml \
--cov-config=setup.cfg \
--cov=desc/ \
--mpl \
--mpl-results-path=mpl_results.html \
--mpl-generate-summary=html \
--splits 6 \
--group ${{ matrix.group }} \
--splitting-algorithm least_duration \
--db ./prof.db
- name: save coverage file and plot comparison results
if: always()
uses: actions/upload-artifact@v4
Expand Down
12 changes: 10 additions & 2 deletions .github/workflows/scheduled.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,11 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
combos: [{group: 1, python_version: '3.8'}, {group: 2, python_version: '3.9'}, {group: 3, python_version: '3.10'}, {group: 4, python_version: '3.11'}, {group: 5, python_version: '3.12'}]
combos: [{group: 1, python_version: '3.8'},
{group: 2, python_version: '3.9'},
{group: 3, python_version: '3.10'},
{group: 4, python_version: '3.11'},
{group: 5, python_version: '3.12'}]

steps:
- uses: actions/checkout@v4
Expand All @@ -31,4 +35,8 @@ jobs:
run: |
pwd
lscpu
python -m pytest -v -m unit --durations=0 --splits 5 --group ${{ matrix.combos.group }} --splitting-algorithm least_duration
python -m pytest -v -m unit \
--durations=0 \
--splits 5 \
--group ${{ matrix.combos.group }} \
--splitting-algorithm least_duration
18 changes: 16 additions & 2 deletions .github/workflows/unittest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,10 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
combos: [{group: 1, python_version: '3.9'}, {group: 2, python_version: '3.10'}, {group: 3, python_version: '3.11'}, {group: 4, python_version: '3.12'}]
combos: [{group: 1, python_version: '3.9'},
{group: 2, python_version: '3.10'},
{group: 3, python_version: '3.11'},
{group: 4, python_version: '3.12'}]

steps:
- uses: actions/checkout@v4
Expand All @@ -41,7 +44,18 @@ jobs:
run: |
pwd
lscpu
python -m pytest -v -m unit --durations=0 --cov-report xml:cov.xml --cov-config=setup.cfg --cov=desc/ --mpl --mpl-results-path=mpl_results.html --mpl-generate-summary=html --splits 4 --group ${{ matrix.combos.group }} --splitting-algorithm least_duration --db ./prof.db
python -m pytest -v -m unit \
--durations=0 \
--cov-report xml:cov.xml \
--cov-config=setup.cfg \
--cov=desc/ \
--mpl \
--mpl-results-path=mpl_results.html \
--mpl-generate-summary=html \
--splits 4 \
--group ${{ matrix.combos.group }} \
--splitting-algorithm least_duration \
--db ./prof.db
- name: save coverage file and plot comparison results
if: always()
uses: actions/upload-artifact@v4
Expand Down
Loading
Loading