Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Travis mods #6

Open
wants to merge 14 commits into
base: release-v4.1.1
Choose a base branch
from
Open

Travis mods #6

wants to merge 14 commits into from

Conversation

davegill
Copy link
Owner

@davegill davegill commented Apr 21, 2019

TYPE: enhancement

KEYWORDS: travis, docker

SOURCE: internal

DESCRIPTION OF CHANGES:
An initial effort to allow auto testing of the WRF source code.

Initial auto testing WRF with travis using docker containers

  1. Single build: MPI, em_real, Linux, -d, GNU
  2. Single run of a regression test case
  3. Uses mostly pre-built container, and adds WRF source code inside

LIST OF MODIFIED FILES:
A /testing/Dockerfile-template
A .travis.yml

TESTS CONDUCTED:

  1. It actually runs a single test case for em_real, using this fork, branch, version.

RELEASE NOTE: Nothing of note yet.

1. Single build
MPI, em_real, Linux, -d, GNU

2. Single run of a regression test case

3. Uses mostly pre-built container, and adds WRF source code inside

 Changes to be committed:
	new file:   .testing/Dockerfile
	new file:   .travis.yml
We end up editting this file to accommodate testing the actual
WRF source code for a particular PR.

 Changes to be committed:
	renamed:    Dockerfile -> Dockerfile-template
 Changes to be committed:
	modified:   .testing/Dockerfile-template
	modified:   .travis.yml
@davegill
Copy link
Owner Author

@weiwangncar @mgduda @dudhia @kkeene44 @smileMchen
Folks,
I have split the build of the docker container into two pieces.

  1. The first piece is always the same: I want Linux, I want a GNU 8, I build a few libs from source (openmpi, netcdf, hdf5), I yum install a few others for WPS, I download a SMALL subset of the regression data, and the regtest namelists. There are also some pieces of ssh that are related to having processors able to talk with each other for MPI jobs. I build this container (not in this job), and push it to docker hub (similar to git hub, only for docker containers).
  2. The second piece, which this job does, grabs that pushed docker container. It adds in the specific WRF repository that we are testing. It takes about 7 minutes to build the image, and another 15 seconds to start an instance of the container. Then that container builds a single job (Linux, em_real, MPI, GNU, -d, where the build takes the usual 5-ish minutes) and it runs a single test case (whatever case 01 is, takes a minute or so).

This travis test takes 13 minutes in total (travis is free). I can probably get that down a bit, but I have never been able to do less than 10 min, given that we put together a boat-load of stuff (libs, GNU, data, etc).

I have set up the testing via these docker containers since they are portable. In a separate repository (github.com/davegill/wrf-coop, the master branch), which is definitely not ready for prime-time yet, I am putting together the pieces of tests that users can conduct for themselves, on their own machines, without travis (but requiring docker). This is a way for us to require users to conduct specific tests.

Of course, I am interested in getting this into the WRF repo. I proposed this to the v4.1.1 branch for no particular reason. We can add these files in with no consequences, until we turn on the testing on the travis-ci page for the wrf-model/WRF repository (where we link travis and github).

@davegill
Copy link
Owner Author

@weiwangncar @mgduda @dudhia @kkeene44 @smileMchen
Folks,
I would like for this travis proposal to be considered for either v4.1.1 or v4.1.2. It only has two files (both new). They do nothing until we hook up travis and github. This will give us an idea of what automatic testing does. Only a single test case is run, so a total of 13-14 minutes after a PR is made. A test is also conducted for each new push to the branch that the user started for the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant