-
Notifications
You must be signed in to change notification settings - Fork 28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Release/rrfsv1.0 #497
Merged
ShelleyMelchior-NOAA
merged 52 commits into
NOAA-EMC:feature/rrfs_refs_v1
from
MarcelCaron-NOAA:release/rrfsv1.0
Aug 7, 2024
Merged
Release/rrfsv1.0 #497
Changes from all commits
Commits
Show all changes
52 commits
Select commit
Hold shift + click to select a range
ed0f1a2
Add default COMINrrfs
ae9bc18
Add rrfs prep drivers
f67ffc8
Add rrfs stats drivers
37657a5
Add configs for rrfs jobs
e26ac32
Add METplus configs for rrfs severe
b808171
Add EVS scripts for rrfs prep jobs
5057f5f
Add EVS scripts for rrfs stats jobs
90bdfec
Add rrfs_firewxnest stats job
f164ed7
Add flexibility with using zero accum for snowfall (needed for rrfs)
d98fb22
finish adding rrfs_firewxnest stats jobs
23348f6
Fix typo
a182e2d
Continue adding rrfs stats jobs and modifications
cf7638d
add explicit NEST_INPUT_TEMPLATE (needed for rrfs firewx domain verif)
7897a0b
Firewx Nest plots job modifications due to adding RRFS
8fc29af
Radar and Severe plots modification due to adding RRFS
dad4d4d
Grid2obs, Headline, Precip, and Snowfall plots modification due to ad…
5d65a64
Change default COMINrrfs to anticipated operational compath
920d9b6
Fix typos
00f2674
Fix log warning and mail message for missing obsproc data
7c06b9d
Update refs mem filenames
88a89f3
Add ADPUPA verification gather step
ddab1cc
Fix typo
a121bf3
Fix typo
5614f36
Increase mem allocation
45e3b40
Merge branch 'feature/rrfs_refs_v1' into release/rrfsv1.0
28f70e6
Add 2d evs ver variable
a633e6b
Adjust maillist
2c0f9dc
Add rrfs jobs to ecf directories and workflow scripts
32ec555
Reduce walltime
fac8b20
increase mem allocation
845b4dd
Increase mem allocation
0e5a944
Increase mem allocation (ecf script)
677305e
Add GRIB2_pdt option for HGT/L0 (ceiling) verification
b3a67e8
Update deprecated METplus variables
4fbb2be
Add GRIB2_pdt options for TCDC verification
8ce1049
Add GRIB2_pdt option for HGT/L0 (ceiling) verification (2)
f71efd6
Update input prep file name
a2a27d7
Fix bash syntax error
5a9af25
Adjust GRIB2_pdt options for TCDC verification
8d194d3
Adjust GRIB2_pdt options for TCDC verification (2)
b50089a
merge feature/rrfs_refs_v1
ab47117
Increase ncpu allocation
d267f2b
Increase ncpu allocation (ecf)
eed69f7
Separate plots jobs by EVAL_PERIOD and LINE_TYPE
2a507b1
Add ecf for radar and severe plots jobs
19d771f
Fix typo
adc582c
Fix typo (radar)
d7dbdb7
Increase cpu allocation
9b3d906
Fix typo + increase cpu allocation
ae980ee
Combine duplicate "KEEPDATA"
603cd51
Add job summary
fb5d55b
Edit mail list
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,10 +1,10 @@ | ||
#PBS -N jevs_cam_radar_plots_00 | ||
#PBS -N jevs_cam_radar_nbrcnt_plots_last31days_00 | ||
#PBS -j oe | ||
#PBS -S /bin/bash | ||
#PBS -q dev | ||
#PBS -A VERF-DEV | ||
#PBS -l walltime=1:50:00 | ||
#PBS -l place=vscatter:exclhost,select=1:ncpus=64 | ||
#PBS -l place=vscatter:exclhost,select=1:ncpus=128 | ||
#PBS -l debug=true | ||
#PBS -V | ||
|
||
|
@@ -31,34 +31,34 @@ module reset | |
module load prod_envir/${prod_envir_ver} | ||
|
||
source $HOMEevs/dev/modulefiles/$COMPONENT/${COMPONENT}_${STEP}.sh | ||
evs_ver_2d=$(echo $evs_ver | cut -d'.' -f1-2) | ||
|
||
|
||
############################################################ | ||
# For dev testing | ||
############################################################ | ||
export envir=prod | ||
export DATAROOT=/lfs/h2/emc/stmp/${USER}/evs_test/$envir/tmp | ||
export KEEPDATA=YES | ||
export VERIF_CASE=radar | ||
export MODELNAME=${COMPONENT} | ||
export job=${PBS_JOBNAME:-jevs_${MODELNAME}_${VERIF_CASE}_${LINE_TYPE}_${STEP}} | ||
export jobid=$job.${PBS_JOBID:-$$} | ||
export COMIN=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver | ||
export COMOUT=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver/$STEP/$COMPONENT | ||
export COMIN=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d | ||
export COMOUT=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d/$STEP/$COMPONENT | ||
export nproc=64 | ||
############################################################ | ||
|
||
export vhr=${vhr:-${vhr}} | ||
export EVAL_PERIOD=${EVAL_PERIOD:-${EVAL_PERIOD}} | ||
export LINE_TYPE=${LINE_TYPE:-${LINE_TYPE}} | ||
export EVAL_PERIOD=${EVAL_PERIOD:-LAST31DAYS} | ||
export LINE_TYPE=${LINE_TYPE:-nbrcnt} | ||
|
||
export SENDMAIL=${SENDMAIL:-YES} | ||
export SENDCOM=${SENDCOM:-YES} | ||
export SENDECF=${SENDECF:-YES} | ||
export SENDDBN=${SENDDBN:-NO} | ||
export KEEPDATA=${KEEPDATA:-NO} | ||
export KEEPDATA=${KEEPDATA:-YES} | ||
|
||
export MAILTO=${MAILTO:-'[email protected],alicia.bentley@noaa.gov'} | ||
export MAILTO=${MAILTO:-'[email protected],andrew.benjamin@noaa.gov'} | ||
|
||
if [ -z "$MAILTO" ]; then | ||
|
||
|
79 changes: 79 additions & 0 deletions
79
dev/drivers/scripts/plots/cam/jevs_cam_radar_nbrcnt_plots_last90days.sh
malloryprow marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
#PBS -N jevs_cam_radar_nbrcnt_plots_last90days_00 | ||
#PBS -j oe | ||
#PBS -S /bin/bash | ||
#PBS -q dev | ||
#PBS -A VERF-DEV | ||
#PBS -l walltime=1:50:00 | ||
#PBS -l place=vscatter:exclhost,select=1:ncpus=128 | ||
#PBS -l debug=true | ||
#PBS -V | ||
|
||
|
||
set -x | ||
|
||
cd $PBS_O_WORKDIR | ||
|
||
|
||
############################################################ | ||
# Load modules | ||
############################################################ | ||
|
||
|
||
export model=evs | ||
export NET=evs | ||
export COMPONENT=cam | ||
export STEP=plots | ||
export RUN=atmos | ||
|
||
export HOMEevs=/lfs/h2/emc/vpppg/noscrub/$USER/EVS | ||
source $HOMEevs/versions/run.ver | ||
module reset | ||
module load prod_envir/${prod_envir_ver} | ||
|
||
source $HOMEevs/dev/modulefiles/$COMPONENT/${COMPONENT}_${STEP}.sh | ||
evs_ver_2d=$(echo $evs_ver | cut -d'.' -f1-2) | ||
|
||
|
||
############################################################ | ||
# For dev testing | ||
############################################################ | ||
export envir=prod | ||
export DATAROOT=/lfs/h2/emc/stmp/${USER}/evs_test/$envir/tmp | ||
export VERIF_CASE=radar | ||
export MODELNAME=${COMPONENT} | ||
export job=${PBS_JOBNAME:-jevs_${MODELNAME}_${VERIF_CASE}_${LINE_TYPE}_${STEP}} | ||
export jobid=$job.${PBS_JOBID:-$$} | ||
export COMIN=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d | ||
export COMOUT=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d/$STEP/$COMPONENT | ||
export nproc=64 | ||
############################################################ | ||
|
||
export vhr=${vhr:-${vhr}} | ||
export EVAL_PERIOD=${EVAL_PERIOD:-LAST90DAYS} | ||
export LINE_TYPE=${LINE_TYPE:-nbrcnt} | ||
|
||
export SENDMAIL=${SENDMAIL:-YES} | ||
export SENDCOM=${SENDCOM:-YES} | ||
export SENDECF=${SENDECF:-YES} | ||
export SENDDBN=${SENDDBN:-NO} | ||
export KEEPDATA=${KEEPDATA:-YES} | ||
|
||
export MAILTO=${MAILTO:-'[email protected],[email protected]'} | ||
|
||
if [ -z "$MAILTO" ]; then | ||
|
||
echo "MAILTO variable is not defined. Exiting without continuing." | ||
|
||
else | ||
|
||
# CALL executable job script here | ||
$HOMEevs/jobs/JEVS_CAM_PLOTS | ||
|
||
fi | ||
|
||
|
||
###################################################################### | ||
# Purpose: This job generates radar verification graphics | ||
# for the CAM component (deterministic and ensemble CAMs) | ||
###################################################################### | ||
|
79 changes: 79 additions & 0 deletions
79
dev/drivers/scripts/plots/cam/jevs_cam_radar_nbrctc_plots_last31days.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
#PBS -N jevs_cam_radar_nbrctc_plots_last31days_00 | ||
#PBS -j oe | ||
#PBS -S /bin/bash | ||
#PBS -q dev | ||
#PBS -A VERF-DEV | ||
#PBS -l walltime=1:50:00 | ||
#PBS -l place=vscatter:exclhost,select=1:ncpus=128 | ||
#PBS -l debug=true | ||
#PBS -V | ||
|
||
|
||
set -x | ||
|
||
cd $PBS_O_WORKDIR | ||
|
||
|
||
############################################################ | ||
# Load modules | ||
############################################################ | ||
|
||
|
||
export model=evs | ||
export NET=evs | ||
export COMPONENT=cam | ||
export STEP=plots | ||
export RUN=atmos | ||
|
||
export HOMEevs=/lfs/h2/emc/vpppg/noscrub/$USER/EVS | ||
source $HOMEevs/versions/run.ver | ||
module reset | ||
module load prod_envir/${prod_envir_ver} | ||
|
||
source $HOMEevs/dev/modulefiles/$COMPONENT/${COMPONENT}_${STEP}.sh | ||
evs_ver_2d=$(echo $evs_ver | cut -d'.' -f1-2) | ||
|
||
|
||
############################################################ | ||
# For dev testing | ||
############################################################ | ||
export envir=prod | ||
export DATAROOT=/lfs/h2/emc/stmp/${USER}/evs_test/$envir/tmp | ||
export VERIF_CASE=radar | ||
export MODELNAME=${COMPONENT} | ||
export job=${PBS_JOBNAME:-jevs_${MODELNAME}_${VERIF_CASE}_${LINE_TYPE}_${STEP}} | ||
export jobid=$job.${PBS_JOBID:-$$} | ||
export COMIN=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d | ||
export COMOUT=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d/$STEP/$COMPONENT | ||
export nproc=64 | ||
############################################################ | ||
|
||
export vhr=${vhr:-${vhr}} | ||
export EVAL_PERIOD=${EVAL_PERIOD:-LAST31DAYS} | ||
export LINE_TYPE=${LINE_TYPE:-nbrctc} | ||
|
||
export SENDMAIL=${SENDMAIL:-YES} | ||
export SENDCOM=${SENDCOM:-YES} | ||
export SENDECF=${SENDECF:-YES} | ||
export SENDDBN=${SENDDBN:-NO} | ||
export KEEPDATA=${KEEPDATA:-YES} | ||
|
||
export MAILTO=${MAILTO:-'[email protected],[email protected]'} | ||
|
||
if [ -z "$MAILTO" ]; then | ||
|
||
echo "MAILTO variable is not defined. Exiting without continuing." | ||
|
||
else | ||
|
||
# CALL executable job script here | ||
$HOMEevs/jobs/JEVS_CAM_PLOTS | ||
|
||
fi | ||
|
||
|
||
###################################################################### | ||
# Purpose: This job generates radar verification graphics | ||
# for the CAM component (deterministic and ensemble CAMs) | ||
###################################################################### | ||
|
79 changes: 79 additions & 0 deletions
79
dev/drivers/scripts/plots/cam/jevs_cam_radar_nbrctc_plots_last90days.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,79 @@ | ||
#PBS -N jevs_cam_radar_nbrctc_plots_last90days_00 | ||
#PBS -j oe | ||
#PBS -S /bin/bash | ||
#PBS -q dev | ||
#PBS -A VERF-DEV | ||
#PBS -l walltime=1:50:00 | ||
#PBS -l place=vscatter:exclhost,select=1:ncpus=128 | ||
#PBS -l debug=true | ||
#PBS -V | ||
|
||
|
||
set -x | ||
|
||
cd $PBS_O_WORKDIR | ||
|
||
|
||
############################################################ | ||
# Load modules | ||
############################################################ | ||
|
||
|
||
export model=evs | ||
export NET=evs | ||
export COMPONENT=cam | ||
export STEP=plots | ||
export RUN=atmos | ||
|
||
export HOMEevs=/lfs/h2/emc/vpppg/noscrub/$USER/EVS | ||
source $HOMEevs/versions/run.ver | ||
module reset | ||
module load prod_envir/${prod_envir_ver} | ||
|
||
source $HOMEevs/dev/modulefiles/$COMPONENT/${COMPONENT}_${STEP}.sh | ||
evs_ver_2d=$(echo $evs_ver | cut -d'.' -f1-2) | ||
|
||
|
||
############################################################ | ||
# For dev testing | ||
############################################################ | ||
export envir=prod | ||
export DATAROOT=/lfs/h2/emc/stmp/${USER}/evs_test/$envir/tmp | ||
export VERIF_CASE=radar | ||
export MODELNAME=${COMPONENT} | ||
export job=${PBS_JOBNAME:-jevs_${MODELNAME}_${VERIF_CASE}_${LINE_TYPE}_${STEP}} | ||
export jobid=$job.${PBS_JOBID:-$$} | ||
export COMIN=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d | ||
export COMOUT=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d/$STEP/$COMPONENT | ||
export nproc=64 | ||
############################################################ | ||
|
||
export vhr=${vhr:-${vhr}} | ||
export EVAL_PERIOD=${EVAL_PERIOD:-LAST90DAYS} | ||
export LINE_TYPE=${LINE_TYPE:-nbrctc} | ||
|
||
export SENDMAIL=${SENDMAIL:-YES} | ||
export SENDCOM=${SENDCOM:-YES} | ||
export SENDECF=${SENDECF:-YES} | ||
export SENDDBN=${SENDDBN:-NO} | ||
export KEEPDATA=${KEEPDATA:-YES} | ||
|
||
export MAILTO=${MAILTO:-'[email protected],[email protected]'} | ||
|
||
if [ -z "$MAILTO" ]; then | ||
|
||
echo "MAILTO variable is not defined. Exiting without continuing." | ||
|
||
else | ||
|
||
# CALL executable job script here | ||
$HOMEevs/jobs/JEVS_CAM_PLOTS | ||
|
||
fi | ||
|
||
|
||
###################################################################### | ||
# Purpose: This job generates radar verification graphics | ||
# for the CAM component (deterministic and ensemble CAMs) | ||
###################################################################### | ||
|
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nproc was increase to 128 in the job specifications, but not further down on the driver script. In the log file for this job is the mpi/cfp command using only 64? If so we shouldn't request resources we aren't using. This seems to be common across the drivers so please check the others that don't match too.