Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce testing time/memory #914

Open
f0uriest opened this issue Feb 29, 2024 · 6 comments
Open

Reduce testing time/memory #914

f0uriest opened this issue Feb 29, 2024 · 6 comments
Labels
P2 Medium Priority, not urgent but should be on the near-term agend

Comments

@f0uriest
Copy link
Member

f0uriest commented Feb 29, 2024

Running our basic unit tests now takes 1.5+ hrs for me locally, and fairly often it will get killed due to OOM issues if I'm trying to do anything else at the same time.

A few suggestions:

  • Reduce resolution as much as possible wherever possible
  • Drop resolution after doing desc.examples.get since in most cases we don't need all the detail
  • for VMEC IO tests, make new baseline cases with like, 10 surfaces, since all we're testing is that things get loaded and saved correctly. We already know DESC and VMEC both work.
  • See if we can eliminate some tests are are now redundant with others. There's ways to see which tests cover which lines, so we can cut tests that don't cover new ground: https://pytest-cov.readthedocs.io/en/latest/contexts.html
@unalmis
Copy link
Collaborator

unalmis commented Feb 29, 2024

one could combine all these tests into one function, computing everything at once to save time:

def test_mercier_vacuum():

def test_compute_d_mercier(DSHAPE_current, HELIOTRON_ex):

def test_compute_d_well(DSHAPE_current, HELIOTRON_ex):

etc.

@f0uriest
Copy link
Member Author

f0uriest commented Mar 6, 2024

For reference, here's a breakdown of all the unit tests and how long they take
https://gist.github.com/f0uriest/dc9f2b7633dae5b84bd0d5d69e74a2f5

total time is over 2hrs

@dpanici dpanici added the P2 Medium Priority, not urgent but should be on the near-term agend label Nov 11, 2024
@dpanici
Copy link
Collaborator

dpanici commented Nov 18, 2024

# TODO: move this coil set to conftest?

we could make this test use the conftest coils instead of making new ones, which is more expensive bc of intersection checks

@YigitElma
Copy link
Collaborator

YigitElma commented Dec 4, 2024

image
Here are some of the longest regression(bottom) and unit(top) tests. Taken from one of the CI runs on December 3rd.

@dpanici
Copy link
Collaborator

dpanici commented Dec 4, 2024

maybe we could reduce the number of examples we have? or move some to "not tested" part of examples so it still exists but people know not to expect it to work all the time?

Or edit input file to run faster (avoid continuation method, reduce res,, etc)

@f0uriest
Copy link
Member Author

f0uriest commented Dec 4, 2024

We don't do that many actual examples, I think its just DSHAPE, HELIOTRON and ATF. I don't think we can get rid of any of those without missing some important code paths

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 Medium Priority, not urgent but should be on the near-term agend
Projects
None yet
Development

No branches or pull requests

4 participants