Update test features: add create_fails script, QC process test, and wait on baseline results #644
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| dt = 3456.0 | ||
| npt_unit = 'y' | ||
| npt = 5 | ||
| year_init = 2005 | ||
| month_init = 1 | ||
| day_init = 1 | ||
| sec_init = 0 | ||
| use_leap_years = .false. | ||
| fyear_init = 2005 | ||
| ycycle = 1 | ||
| dumpfreq = 'm' | ||
| dumpfreq_n = 12 | ||
| diagfreq = 24 | ||
| histfreq = 'd','x','x','x','x' | ||
| f_hi = 'd' | ||
| hist_avg = .false. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| #!/bin/csh | ||
|
|
||
| echo " " | ||
| set tmpfile = create_fails.tmp | ||
| set outfile = fails.ts | ||
|
|
||
| ./results.csh >& /dev/null | ||
| cat results.log | grep ' run\| test' | grep -v PASS | cut -f 2 -d " " | sort -u >! $tmpfile | ||
|
|
||
| echo "# Test Grid PEs Sets" >! fails.ts | ||
| foreach line ( "`cat $tmpfile`" ) | ||
| #echo $line | ||
| set test = `echo $line | cut -d "_" -f 3` | ||
| set grid = `echo $line | cut -d "_" -f 4` | ||
| set pes = `echo $line | cut -d "_" -f 5` | ||
| set opts = `echo $line | cut -d "_" -f 6- | sed 's/_/,/g'` | ||
| echo "$test $grid $pes $opts" >> fails.ts | ||
| end | ||
|
|
||
| rm $tmpfile | ||
| echo "$0 done" | ||
| echo "Not passed tests written to file...... $outfile" | ||
| echo "To setup a new test suite, try something like" | ||
| echo " ./cice.setup --suite fails.ts ..." | ||
|
There was a problem hiding this comment. does this work ? I guess you first have to move There was a problem hiding this comment. You're right that It wouldn't work exactly as described. You can move fails.ts to the cice.setup directory too. It works from there. But you're right, maybe I'll clarify that a bit. I don't want to get too deep in the weeds though. |
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,5 +1,6 @@ | ||
| # Test Grid PEs Sets BFB-compare | ||
| smoke gx3 8x2 diag1,run5day | ||
| restart gx3 4x2x25x29x4 dslenderX2 | ||
| smoke gx3 4x2x25x29x4 debug,run2day,dslenderX2 | ||
| logbfb gx3 4x2x25x29x4 dslenderX2,diag1,reprosum | ||
| smoke gx3 1x2 run2day |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,6 @@ | ||
| # Test Grid PEs Sets BFB-compare | ||
| qcchk gx3 72x1 qc,medium qcchk_gx3_72x1_medium_qc | ||
| qcchk gx1 144x1 qc,medium | ||
| smoke gx1 144x2 gx1prod,long,run10year | ||
| qcchkf gx3 72x1 qc,medium,alt02 qcchk_gx3_72x1_medium_qc | ||
| qcchk gx3 72x1 qcnonbfb,medium qcchk_gx3_72x1_medium_qc |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
|
|
||
| cp ${ICE_SANDBOX}/configuration/scripts/tests/QC/CICE_t_critical_p0.8.nc . | ||
| cp ${ICE_SANDBOX}/configuration/scripts/tests/QC/CICE_Lookup_Table_p0.8_n1825.nc . | ||
|
|
||
| #---------------------------------------------------- | ||
| # Run the CICE model | ||
| # cice.run returns -1 if run did not complete successfully | ||
|
|
||
| ./cice.run | ||
| set res="$status" | ||
|
|
||
| set log_file = `ls -t1 ${ICE_RUNDIR}/cice.runlog* | head -1` | ||
| set ttimeloop = `grep TimeLoop ${log_file} | grep Timer | cut -c 22-32` | ||
| set tdynamics = `grep Dynamics ${log_file} | grep Timer | cut -c 22-32` | ||
| set tcolumn = `grep Column ${log_file} | grep Timer | cut -c 22-32` | ||
| if (${ttimeloop} == "") set ttimeloop = -1 | ||
| if (${tdynamics} == "") set tdynamics = -1 | ||
| if (${tcolumn} == "") set tcolumn = -1 | ||
|
|
||
| mv -f ${ICE_CASEDIR}/test_output ${ICE_CASEDIR}/test_output.prev | ||
| cat ${ICE_CASEDIR}/test_output.prev | grep -iv "${ICE_TESTNAME} run" >! ${ICE_CASEDIR}/test_output | ||
| mv -f ${ICE_CASEDIR}/test_output ${ICE_CASEDIR}/test_output.prev | ||
| cat ${ICE_CASEDIR}/test_output.prev | grep -iv "${ICE_TESTNAME} test" >! ${ICE_CASEDIR}/test_output | ||
| rm -f ${ICE_CASEDIR}/test_output.prev | ||
|
|
||
| set grade = PASS | ||
| if ( $res != 0 ) then | ||
| set grade = FAIL | ||
| echo "$grade ${ICE_TESTNAME} run ${ttimeloop} ${tdynamics} ${tcolumn}" >> ${ICE_CASEDIR}/test_output | ||
| echo "$grade ${ICE_TESTNAME} test " >> ${ICE_CASEDIR}/test_output | ||
| exit 99 | ||
| endif | ||
|
|
||
| echo "$grade ${ICE_TESTNAME} run ${ttimeloop} ${tdynamics} ${tcolumn}" >> ${ICE_CASEDIR}/test_output | ||
| echo "$grade ${ICE_TESTNAME} test " >> ${ICE_CASEDIR}/test_output | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
|
|
||
| cp ${ICE_SANDBOX}/configuration/scripts/tests/QC/CICE_t_critical_p0.8.nc . | ||
| cp ${ICE_SANDBOX}/configuration/scripts/tests/QC/CICE_Lookup_Table_p0.8_n1825.nc . | ||
|
|
||
| #---------------------------------------------------- | ||
| # Run the CICE model | ||
| # cice.run returns -1 if run did not complete successfully | ||
|
|
||
| ./cice.run | ||
| set res="$status" | ||
|
|
||
| set log_file = `ls -t1 ${ICE_RUNDIR}/cice.runlog* | head -1` | ||
| set ttimeloop = `grep TimeLoop ${log_file} | grep Timer | cut -c 22-32` | ||
| set tdynamics = `grep Dynamics ${log_file} | grep Timer | cut -c 22-32` | ||
| set tcolumn = `grep Column ${log_file} | grep Timer | cut -c 22-32` | ||
| if (${ttimeloop} == "") set ttimeloop = -1 | ||
| if (${tdynamics} == "") set tdynamics = -1 | ||
| if (${tcolumn} == "") set tcolumn = -1 | ||
|
|
||
| mv -f ${ICE_CASEDIR}/test_output ${ICE_CASEDIR}/test_output.prev | ||
| cat ${ICE_CASEDIR}/test_output.prev | grep -iv "${ICE_TESTNAME} run" >! ${ICE_CASEDIR}/test_output | ||
| mv -f ${ICE_CASEDIR}/test_output ${ICE_CASEDIR}/test_output.prev | ||
| cat ${ICE_CASEDIR}/test_output.prev | grep -iv "${ICE_TESTNAME} test" >! ${ICE_CASEDIR}/test_output | ||
| rm -f ${ICE_CASEDIR}/test_output.prev | ||
|
|
||
| set grade = PASS | ||
| if ( $res != 0 ) then | ||
| set grade = FAIL | ||
| echo "$grade ${ICE_TESTNAME} run ${ttimeloop} ${tdynamics} ${tcolumn}" >> ${ICE_CASEDIR}/test_output | ||
| echo "$grade ${ICE_TESTNAME} test " >> ${ICE_CASEDIR}/test_output | ||
| exit 99 | ||
| endif | ||
|
|
||
| echo "$grade ${ICE_TESTNAME} run ${ttimeloop} ${tdynamics} ${tcolumn}" >> ${ICE_CASEDIR}/test_output | ||
| echo "$grade ${ICE_TESTNAME} test " >> ${ICE_CASEDIR}/test_output | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe "Failed tests" instead ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Failed tests isn't quite accurate. Failed != Not passed. But I think you're right on reflection that failed is probably a better word here. I'll change it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated.