Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: simplify autotest framework #1464

Merged
merged 28 commits into from
Dec 11, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
d2c4025
refactor: simplify autotest framework
wpbonelli Nov 22, 2023
507fb27
fix path printing in common_regression.py
wpbonelli Nov 23, 2023
58f593c
step closer to unified hooks, formatting
wpbonelli Nov 24, 2023
f4ea10b
fix failing tests, formatting
wpbonelli Nov 24, 2023
44152f2
remove separate libmf6 fixture (just use targets)
wpbonelli Nov 25, 2023
5032bd7
slight refactor to get_mf6_files()
wpbonelli Nov 25, 2023
8e45a3e
minor framework cleanup, docstrings in common_regression.py
wpbonelli Nov 26, 2023
d922a86
move external model tests to autotest/example
wpbonelli Nov 26, 2023
ad69f8d
simplify comparison fns
wpbonelli Nov 26, 2023
da31cec
misc refactoring, enforce toggles in top-level run(), lift prop acces…
wpbonelli Nov 27, 2023
8d161fc
action -> comparison, fold mf6_regression into comparison option
wpbonelli Nov 27, 2023
3e6c3bc
misc fixes
wpbonelli Nov 27, 2023
f6f3185
fix _compare_heads()
wpbonelli Nov 27, 2023
1b78b2b
cleanup in example/* tests
wpbonelli Nov 29, 2023
4419e60
fix testmodel/largetestmodel tests
wpbonelli Nov 30, 2023
c9ce519
remove pytest-cases usages and dependency
wpbonelli Nov 30, 2023
59e8f0e
rename comparison -> compare, remove comparison-related switches in f…
wpbonelli Dec 2, 2023
7f853eb
remove nested autotest/example dir, minor cleanup
wpbonelli Dec 2, 2023
aee7f5b
more cleanup, print statement formatting
wpbonelli Dec 2, 2023
fa3dd18
test fixes
wpbonelli Dec 2, 2023
8eea50e
switch to auto as default option, more cleanup
wpbonelli Dec 2, 2023
fab92a6
prep to support multiple comparison sims
wpbonelli Dec 3, 2023
afa70fe
fix DEVELOPER.md
wpbonelli Dec 3, 2023
2c22a38
require modflow-devtools>=1.1
wpbonelli Dec 4, 2023
743da91
fix GWT -> GWT6 model key
wpbonelli Dec 4, 2023
f69ab46
move comparison patterns to top-level var in common_regression.py, cl…
wpbonelli Dec 4, 2023
1117fd4
use f-strings consistently, remove unused functions
wpbonelli Dec 4, 2023
61b5027
update test_gwf_sfr_diversion.py
wpbonelli Dec 5, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/large.yml
Original file line number Diff line number Diff line change
Expand Up @@ -113,4 +113,4 @@ jobs:

- name: Run tests
working-directory: modflow6/autotest
run: pytest -v -n auto --durations 0 test_z03_${{ matrix.repo }}.py
run: pytest -v -n auto --durations 0 test_${{ matrix.repo }}.py
18 changes: 1 addition & 17 deletions DEVELOPER.md
Original file line number Diff line number Diff line change
Expand Up @@ -293,7 +293,7 @@ The binaries can then be found in the `bin` folder. `meson install` also trigger

MODFLOW 6 unit tests are written in Fortran with [`test-drive`](https://github.com/fortran-lang/test-drive).

MODFLOW 6 integration tests are written in Python with [`pytest`](https://docs.pytest.org/en/7.1.x/), with the help of plugins like `pytest-xdist` and `pytest-cases`. Integration testing dependencies are included in the Conda environment `environment.yml`.
MODFLOW 6 integration tests are written in Python with [`pytest`](https://docs.pytest.org/en/7.1.x/). Integration testing dependencies are included in the Conda environment `environment.yml`.

**Note:** the entire test suite should pass before a pull request is submitted. Tests run in GitHub Actions CI and a PR can only be merged with passing tests. See [`CONTRIBUTING.md`](CONTRIBUTING.md) for more information.

Expand Down Expand Up @@ -452,22 +452,6 @@ The `large` marker is a subset of the `repo` marker. To test models excluded fro
pytest -v -n auto -m "large"
```

Test scripts for external model repositories can also be run independently:

```shell
# MODFLOW 6 test models
pytest -v -n auto test_z01_testmodels_mf6.py

# MODFLOW 5 to 6 conversion test models
pytest -v -n auto test_z02_testmodels_mf5to6.py

# models from modflow6-examples repo
pytest -v -n auto test_z03_examples.py

# models from modflow6-largetestmodels repo
pytest -v -n auto test_z03_largetestmodels.py
```

Tests load external models from fixtures provided by `modflow-devtools`. External model tests can be selected by model or simulation name, or by packages used. See the [`modflow-devtools` documentation](https://modflow-devtools.readthedocs.io/en/latest/md/fixtures.html#filtering) for usage examples. Note that filtering options only apply to tests using external models, and will not filter tests defining models in code — for that, the `pytest` built-in `-k` option may be used.

### Writing tests
Expand Down
Loading