Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Choose a consistent file structure for integration test data #116

Closed
dstansby opened this issue Oct 4, 2022 · 4 comments
Closed

Choose a consistent file structure for integration test data #116

dstansby opened this issue Oct 4, 2022 · 4 comments
Assignees
Labels
documentation Improvements or additions to documentation opinions wanted Collaborators and users are encouraged to contribute their opinions testing Adding or requesting more test coverage

Comments

@dstansby
Copy link
Member

dstansby commented Oct 4, 2022

In #101 I'm trying to simplify the integration tests so it's really easy to add new tests. I've noticed that there is not a consistent file structure in the test files that have been uploaded. For the low numbered tests (01, 02, 03) the structure is a single directory containing

  • a single input file named pstd_input_file.m
  • one or more input data files named pstd_{run_type}_input.mat.
    run_type can be any string.
  • corresponding reference output data files named pstd_{run_type}_reference_output.mat.
    For every input file there is a corresponding output file with the same run_type in the filename.

But in the high nubmered tests (09, 10) there are "in" and "out" directories. To make the test infastructure as simple as possible, it would be good to choose, document, and stick to a file structure for the integration test files.

@samcunliffe samcunliffe added documentation Improvements or additions to documentation testing Adding or requesting more test coverage opinions wanted Collaborators and users are encouraged to contribute their opinions labels Oct 4, 2022
@samcunliffe
Copy link
Member

samcunliffe commented Oct 4, 2022

Do you have a preference?

Peter uses directories in/ and out/ in his tarballs so

  • [Pro] it's in keeping with his workflow & scripts.
  • [Con] *_reference_output.mat is arguably more explicit.

@dstansby
Copy link
Member Author

dstansby commented Oct 5, 2022

I don't personally have a preference and think either is fine, so I'd say go with what Peter prefers since he'll be using the software after the ARC project finishes

@samcunliffe
Copy link
Member

Solved by @willGraham01 (quite a while ago now... I've lost the PR)

@willGraham01
Copy link
Collaborator

This feature comes in with #236, however reaches it's final (and present solution state) in #244.

The config_XX.yaml files explicitly provide the full path to necessary input/output/reference files for each test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation opinions wanted Collaborators and users are encouraged to contribute their opinions testing Adding or requesting more test coverage
Projects
None yet
Development

No branches or pull requests

3 participants