-
Notifications
You must be signed in to change notification settings - Fork 132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Testing suite #9
Changes from 26 commits
095f3bf
34cd5f0
cfabe15
0c87661
576c0cb
b2aafa2
5668668
17d722d
f52288d
d2dbf28
0db5c73
d224516
2ddbdd9
4b7b976
a3c6c85
473d158
91e432f
16b7c4b
ff71bdb
5729c95
6482090
f5e4543
374955d
356f958
7716032
39b3e56
204fd6c
cb845ef
5445671
3984d99
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
CICE version 6 testing | ||
June 2017 | ||
|
||
This README file documents how to use the testing suite developed for the | ||
CICE Consortium sea ice model. The tests currently supported are: | ||
1. "Smoke" test - Ensures that the model compiles successfully and runs | ||
to completion for a 1 day simulation. Validation is | ||
determined by performing a binary comparison on restart | ||
files to a baseline dataset. | ||
2. 10-day test - Ensures that the model compiles successfully and runs | ||
to completion for a 10 day simulation. Validation is | ||
determined by performing a binary comparison on restart | ||
files to a baseline dataset. | ||
3. Exact restart - Ensures that the output from a cold-start simulation | ||
and a restart simulation provide bit-for-bit identical | ||
results. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. State explicitly that a baseline does not need to be created for restart tests |
||
4. Annual test - Ensures that the model compiles successfully and runs | ||
to completion for an annual simulation. Validation is | ||
determined by performing a binary comparison on restart | ||
files to a baseline dataset. | ||
|
||
--- To generate a baseline dataset for a test case --- | ||
|
||
Quickstart (example): | ||
|
||
./create.case -t smoke -m conrad -b | ||
cd smoke_baseline_gx3_conrad_4x1 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. remove _baseline from first directory and add .t00 to second one |
||
./cice.build | ||
./cice.submit | ||
# After job finishes, check output | ||
cat test_output | ||
|
||
|
||
--- To run a test case and compare to a baseline dataset --- | ||
|
||
Quickstart (example): | ||
|
||
./create.case -t smoke -m conrad | ||
cd smoke_gx3_conrad_4x1 | ||
./cice.build | ||
./cice.submit | ||
# After job finishes, check output | ||
cat test_output | ||
|
||
|
||
--- To run a regression test (compare cicev6.0.1 output to cicev6.0.0 output) --- | ||
|
||
Quickstart (example): | ||
|
||
./create.case -t smoke -m conrad -b -bg cicev6.0.1 -bc cicev6.0.0 | ||
cd smoke_gx3_conrad_4x1 | ||
./cice.build | ||
./cice.submit | ||
# After job finishes, check output | ||
cat test_output | ||
|
||
|
||
Additional Details: | ||
- The '-b' flag can be combined with '-bd' to specify the location where you | ||
want the baseline dataset to be written. Without specifying '-bd', the | ||
baseline dataset will be written to the default baseline directory found | ||
in the env.<machine> file (CICE_MACHINE_BASELINE). | ||
- If '-b' is not passed, the '-bd' option will specify the location of the | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. should this be -bc instead of -bd? |
||
baseline dataset that you want to compare to. Without specifying '-bd', | ||
the scripts will look for baseline datasets in the default baseline | ||
directory found in the env.<machine> file (CICE_MACHINE_BASELINE). | ||
- To generate a baseline dataset for a specific version (for regression testing), | ||
use '-b' in conjunction to '-bg <version_name>'. The scripts will then place | ||
the baseline dataset in $CICE_MACHINE_BASELINE/<version_name>/ |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,12 +1,15 @@ | ||
#! /bin/csh -f | ||
|
||
echo ${0} | ||
if ( $1 != "" ) then | ||
echo ${0} ${1} | ||
else | ||
echo ${0} | ||
endif | ||
|
||
source ./cice.settings | ||
source ${CICE_CASEDIR}/env.${CICE_MACHINE} || exit 2 | ||
|
||
set jobfile = cice.10day | ||
set subfile = cice.submit | ||
set jobfile = $1 | ||
|
||
set ntasks = ${CICE_NTASKS} | ||
set nthrds = ${CICE_NTHRDS} | ||
|
@@ -24,9 +27,6 @@ if ($ptile > ${maxtpn} / 2) @ ptile = ${maxtpn} / 2 | |
|
||
#========================================== | ||
|
||
# Create test script that runs cice.build, cice.run, and validates | ||
#========================================== | ||
|
||
cat >! ${jobfile} << EOF0 | ||
#!/bin/csh -f | ||
EOF0 | ||
|
@@ -48,6 +48,20 @@ cat >> ${jobfile} << EOFB | |
#BSUB -P ${acct} | ||
EOFB | ||
|
||
#========================================== | ||
|
||
else if (${CICE_MACHINE} =~ cheyenne*) then | ||
cat >> ${jobfile} << EOFB | ||
#PBS -j oe | ||
#PBS -m ae | ||
#PBS -V | ||
#PBS -q regular | ||
#PBS -N ${CICE_CASENAME} | ||
#PBS -A ${CICE_ACCT} | ||
#PBS -l select=${nnodes}:ncpus=${ntasks}:mpiprocs=${ntasks} | ||
#PBS -l walltime=02:00:00 | ||
EOFB | ||
|
||
else if (${CICE_MACHINE} =~ thunder* || ${CICE_MACHINE} =~ gordon* || ${CICE_MACHINE} =~ conrad*) then | ||
cat >> ${jobfile} << EOFB | ||
#PBS -N ${CICE_CASENAME} | ||
|
@@ -78,88 +92,30 @@ EOFB | |
else if (${CICE_MACHINE} =~ wolf*) then | ||
cat >> ${jobfile} << EOFB | ||
#SBATCH -J ${CICE_CASENAME} | ||
#SBATCH -t 0:10:00 | ||
#SBATCH -t 0:45:00 | ||
#SBATCH -A ${acct} | ||
#SBATCH -N ${nnodes} | ||
#SBATCH -e slurm%j.err | ||
#SBATCH -o slurm%j.out | ||
#SBATCH --mail-type FAIL | ||
#SBATCH [email protected] | ||
###SBATCH --mail-type END,FAIL | ||
###SBATCH [email protected] | ||
#SBATCH --qos=low | ||
EOFB | ||
|
||
endif | ||
|
||
cat >> ${jobfile} << EOF2 | ||
cd ${CICE_CASEDIR} | ||
source ./cice.settings || exit 2 | ||
source ./env.\${CICE_MACHINE} || exit 2 | ||
|
||
# Compile the CICE code | ||
./cice.build | ||
set rc_build = \$? | ||
|
||
# Run the CICE model | ||
./cice.run | ||
set rc_run = \$? | ||
|
||
EOF2 | ||
|
||
if ($1 != "") then | ||
cat >> ${jobfile} << EOF3 | ||
# Get the final output filename | ||
foreach file (\${CICE_RUNDIR}/restart/*) | ||
set test_data = \$file | ||
end | ||
|
||
set baseline_data = $1/\$test_data:t | ||
|
||
if ( { cmp -s \$test_data \$baseline_data } ) then | ||
set rc_valid = 0 | ||
else | ||
set rc_valid = 1 | ||
endif | ||
EOF3 | ||
endif | ||
|
||
cat >> ${jobfile} << EOF4 | ||
|
||
if (\$rc_build == 0) then | ||
echo "Build: [0;92mPASS[0;0m" | ||
else | ||
echo "Build: [0;41mFAIL[0;0m" | ||
endif | ||
|
||
if (\$rc_run == 0) then | ||
echo "Run: [0;92mPASS[0;0m" | ||
else | ||
echo "Run: [0;41mFAIL[0;0m" | ||
endif | ||
EOF4 | ||
|
||
if ($1 != "") then | ||
cat >> ${jobfile} << EOF5 | ||
else if (${CICE_MACHINE} =~ pinto*) then | ||
cat >> ${jobfile} << EOFB | ||
#SBATCH -J ${CICE_CASENAME} | ||
#SBATCH -t 0:45:00 | ||
#SBATCH -A ${acct} | ||
#SBATCH -N ${nnodes} | ||
#SBATCH -e slurm%j.err | ||
#SBATCH -o slurm%j.out | ||
###SBATCH --mail-type END,FAIL | ||
###SBATCH [email protected] | ||
#SBATCH --qos=standby | ||
EOFB | ||
|
||
if (\$rc_valid == 0) then | ||
echo "Validation: [0;92mPASS[0;0m" | ||
else | ||
echo "Validation: [0;41mFAIL[0;0m" | ||
endif | ||
EOF5 | ||
echo "${0} ERROR ${CICE_MACHINE} unknown" | ||
exit -1 | ||
endif | ||
|
||
#========================================== | ||
|
||
chmod +x ${jobfile} | ||
|
||
#========================================== | ||
|
||
cat >! ${subfile} << EOFS | ||
#!/bin/csh -f | ||
|
||
${CICE_MACHINE_SUBMIT} ${jobfile} | ||
echo "\`date\` \${0}: ${CICE_CASENAME} job submitted" >> ${CICE_CASEDIR}/README.test | ||
|
||
EOFS | ||
|
||
chmod +x ${subfile} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
#! /bin/csh -f | ||
|
||
echo ${0} | ||
|
||
source ./cice.settings | ||
|
||
set jobfile = $1 | ||
|
||
set ntasks = ${CICE_NTASKS} | ||
set nthrds = ${CICE_NTHRDS} | ||
|
||
#========================================== | ||
|
||
if (${CICE_MACHINE} =~ yellowstone*) then | ||
cat >> ${jobfile} << EOFR | ||
setenv MP_TASK_AFFINITY core:\${OMP_NUM_THREADS} | ||
mpirun.lsf ./cice >&! \$CICE_RUNLOG_FILE | ||
EOFR | ||
|
||
else if (${CICE_MACHINE} =~ cheyenne*) then | ||
cat >> ${jobfile} << EOFR | ||
mpiexec_mpt -n ${ntasks} ./cice >&! \$CICE_RUNLOG_FILE | ||
EOFR | ||
|
||
else if (${CICE_MACHINE} =~ thunder*) then | ||
cat >> ${jobfile} << EOFR | ||
mpiexec_mpt -np ${ntasks} omplace ./cice >&! \$CICE_RUNLOG_FILE | ||
EOFR | ||
|
||
else if (${CICE_MACHINE} =~ gordon* || ${CICE_MACHINE} =~ conrad*) then | ||
cat >> ${jobfile} << EOFR | ||
aprun -n ${ntasks} -N ${ntasks} -d ${nthrds} ./cice >&! \$CICE_RUNLOG_FILE | ||
EOFR | ||
|
||
else if (${CICE_MACHINE} =~ cori*) then | ||
cat >> ${jobfile} << EOFR | ||
srun -n ${ntasks} -c ${nthrds} ./cice >&! \$CICE_RUNLOG_FILE | ||
EOFR | ||
|
||
else if (${CICE_MACHINE} =~ wolf*) then | ||
cat >> ${jobfile} << EOFR | ||
mpirun -np ${ntasks} ./cice >&! \$CICE_RUNLOG_FILE | ||
EOFR | ||
|
||
else if (${CICE_MACHINE} =~ pinto*) then | ||
cat >> ${jobfile} << EOFR | ||
mpirun -np ${ntasks} ./cice >&! \$CICE_RUNLOG_FILE | ||
EOFR | ||
#cat >> ${jobfile} << EOFR | ||
#srun -n ${ntasks} -c ${nthrds} ./cice >&! \$CICE_RUNLOG_FILE | ||
#EOFR | ||
|
||
else | ||
echo "${0} ERROR ${CICE_MACHINE} unknown" | ||
exit -1 | ||
endif |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add the actual arguments needed for -t (smoke, 10day, restart, annual)