Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retire the tools/tests framework for CESM test system custom tests that do the same thing #2279

Closed
7 tasks done
ekluzek opened this issue Dec 4, 2023 · 3 comments · Fixed by #2539
Closed
7 tasks done
Assignees
Labels
code health improving internal code structure to make easier to maintain (sustainability)

Comments

@ekluzek
Copy link
Collaborator

ekluzek commented Dec 4, 2023

The test/tools framework should be retired for use of the CESM test system custom tests that tests tools.

Right now this framework only tests a few tools, with a few others that will be retired shortly. Testing for python scripts is now better handled with our python testing framework. The one thing the python testing framework can't do is to make sure the top level skeleton tools work in terms of paths. The tools/testing framework also made it easy to setup tools that weren't under the python framework, or older FORTRAN based tools, or tools based on perl, or NCL or another language.

Some reasons to shift:

  1. The test/tools/test_driver.sh frame is older
    It's from a few generations ago of a test framework for CAM
  2. CESM test_driver is supported by a larger user and development base
  3. Everyone on the CTSM software team is spunup on developing using CESM test_driver and NOT test_driver.sh
  4. test/tools is based on Bourne shell and not python which has a rich testing ecosystem

We did realize that having a test tool that tests the top level scripts is useful though, if the paths for the top level tool become screwed up. Since, that's almost the ONLY thing that can go wrong in the top level skeleton script any problems are going to be easy to solve. But, since the top level scripts are what users interact with in the end it's the thing we want to ensure is working for people.

This is something we agreed to at our last CTSM software meeting.

Definition of Done:

  • Remove test subdirectory and everything below it
  • Remove mention of it in bld/unit_testers/xFail
  • Remove .gitignore:test_driver_*.sh
  • Update test/tools/README @ekluzek this TODO and two of the next ones are obsolete as a result of "Remove /test and everything below it"
  • Remove test/tools/show_var_diffs.sh (and make sure it's not used anywhere)
  • Remove include in: doc/source/users_guide/testing/testing.rst
  • Remove tools/README.testing
@ekluzek ekluzek added the code health improving internal code structure to make easier to maintain (sustainability) label Dec 4, 2023
@ekluzek
Copy link
Collaborator Author

ekluzek commented Dec 4, 2023

Note, we've had various problems with calling python scripts from a sub shell rather than using the python directly, so we will need how to implement this for CESM create_tests tests that can do this well.

@ekluzek
Copy link
Collaborator Author

ekluzek commented Dec 7, 2023

This is the removal side of #2272

@ekluzek
Copy link
Collaborator Author

ekluzek commented May 20, 2024

In last Thursday's CTSM SE meeting we talked about this and decided we should just remove it since it's deprecated and NOT working on Derecho.

This means removing the entire test subdirectory and everything below it. As well as updating the README file at the top level to remove mention of it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
code health improving internal code structure to make easier to maintain (sustainability)
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants