Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release/rrfsv1.0 #497

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
ed0f1a2
Add default COMINrrfs
Jun 12, 2024
ae9bc18
Add rrfs prep drivers
Jun 12, 2024
f67ffc8
Add rrfs stats drivers
Jun 12, 2024
37657a5
Add configs for rrfs jobs
Jun 12, 2024
e26ac32
Add METplus configs for rrfs severe
Jun 12, 2024
b808171
Add EVS scripts for rrfs prep jobs
Jun 12, 2024
5057f5f
Add EVS scripts for rrfs stats jobs
Jun 12, 2024
90bdfec
Add rrfs_firewxnest stats job
Jun 26, 2024
f164ed7
Add flexibility with using zero accum for snowfall (needed for rrfs)
Jun 27, 2024
d98fb22
finish adding rrfs_firewxnest stats jobs
Jun 27, 2024
23348f6
Fix typo
Jun 27, 2024
a182e2d
Continue adding rrfs stats jobs and modifications
Jun 27, 2024
cf7638d
add explicit NEST_INPUT_TEMPLATE (needed for rrfs firewx domain verif)
Jun 27, 2024
7897a0b
Firewx Nest plots job modifications due to adding RRFS
Jun 27, 2024
8fc29af
Radar and Severe plots modification due to adding RRFS
Jun 27, 2024
dad4d4d
Grid2obs, Headline, Precip, and Snowfall plots modification due to ad…
Jun 27, 2024
5d65a64
Change default COMINrrfs to anticipated operational compath
Jun 28, 2024
920d9b6
Fix typos
Jul 3, 2024
00f2674
Fix log warning and mail message for missing obsproc data
Jul 5, 2024
7c06b9d
Update refs mem filenames
Jul 5, 2024
88a89f3
Add ADPUPA verification gather step
Jul 8, 2024
ddab1cc
Fix typo
Jul 11, 2024
a121bf3
Fix typo
Jul 11, 2024
5614f36
Increase mem allocation
Jul 11, 2024
45e3b40
Merge branch 'feature/rrfs_refs_v1' into release/rrfsv1.0
Jul 11, 2024
28f70e6
Add 2d evs ver variable
Jul 11, 2024
a633e6b
Adjust maillist
Jul 11, 2024
2c0f9dc
Add rrfs jobs to ecf directories and workflow scripts
Jul 12, 2024
32ec555
Reduce walltime
Jul 12, 2024
fac8b20
increase mem allocation
Jul 15, 2024
845b4dd
Increase mem allocation
Jul 16, 2024
0e5a944
Increase mem allocation (ecf script)
Jul 16, 2024
677305e
Add GRIB2_pdt option for HGT/L0 (ceiling) verification
Jul 19, 2024
b3a67e8
Update deprecated METplus variables
Jul 22, 2024
4fbb2be
Add GRIB2_pdt options for TCDC verification
Jul 22, 2024
8ce1049
Add GRIB2_pdt option for HGT/L0 (ceiling) verification (2)
Jul 22, 2024
f71efd6
Update input prep file name
Jul 22, 2024
a2a27d7
Fix bash syntax error
Jul 22, 2024
5a9af25
Adjust GRIB2_pdt options for TCDC verification
Jul 24, 2024
8d194d3
Adjust GRIB2_pdt options for TCDC verification (2)
Jul 24, 2024
b50089a
merge feature/rrfs_refs_v1
Jul 26, 2024
ab47117
Increase ncpu allocation
Jul 26, 2024
d267f2b
Increase ncpu allocation (ecf)
Jul 26, 2024
eed69f7
Separate plots jobs by EVAL_PERIOD and LINE_TYPE
Aug 1, 2024
2a507b1
Add ecf for radar and severe plots jobs
Aug 1, 2024
19d771f
Fix typo
Aug 6, 2024
adc582c
Fix typo (radar)
Aug 6, 2024
d7dbdb7
Increase cpu allocation
Aug 6, 2024
9b3d906
Fix typo + increase cpu allocation
Aug 6, 2024
ae980ee
Combine duplicate "KEEPDATA"
Aug 6, 2024
603cd51
Add job summary
Aug 7, 2024
fb5d55b
Edit mail list
Aug 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
#PBS -N jevs_cam_nam_firewxnest_grid2obs_plots
#PBS -N jevs_cam_firewxnest_grid2obs_plots
#PBS -j oe
#PBS -S /bin/bash
#PBS -q "dev"
#PBS -A VERF-DEV
#PBS -l walltime=00:50:00
#PBS -l place=shared,select=1:ncpus=1:mem=100GB
#PBS -l place=shared,select=1:ncpus=2:mem=100GB
#PBS -l debug=true

export model=evs
Expand Down Expand Up @@ -37,9 +37,7 @@ export STEP=plots
export COMPONENT=cam
export RUN=atmos
export VERIF_CASE=grid2obs
export MODELNAME=nam_firewxnest
export modsys=nam
export mod_ver=${nam_ver}
export MODELNAME=firewxnest

source $HOMEevs/dev/modulefiles/$COMPONENT/${COMPONENT}_${STEP}.sh
evs_ver_2d=$(echo $evs_ver | cut -d'.' -f1-2)
Expand All @@ -54,9 +52,9 @@ export COMOUT=/lfs/h2/emc/ptmp/$USER/$NET/$evs_ver_2d/$STEP/$COMPONENT
$HOMEevs/jobs/JEVS_CAM_PLOTS

######################################################################
## Purpose: This job will generate the grid2obs statistics for the NAM_FIREWXNEST
## model and generate stat files.
## Purpose: This job will generate the grid2obs plots for the NAM_FIREWXNEST
## and RRFS_FIREWXNEST models and generate plots tar files.
#######################################################################
#

exit

5 changes: 5 additions & 0 deletions dev/drivers/scripts/plots/cam/jevs_cam_grid2obs_plots.sh
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,8 @@ export COMOUT=/lfs/h2/emc/ptmp/$USER/$NET/$evs_ver_2d/$STEP/$COMPONENT

# Job Settings and Run
. ${HOMEevs}/jobs/JEVS_CAM_PLOTS

######################################################################
# Purpose: This job generates grid2obs verification graphics
# for the CAM component (deterministic CAMs)
######################################################################
5 changes: 5 additions & 0 deletions dev/drivers/scripts/plots/cam/jevs_cam_headline_plots.sh
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,8 @@ export vhr=${vhr:-${vhr}}

# Job Settings and Run
. ${HOMEevs}/jobs/JEVS_CAM_PLOTS

######################################################################
# Purpose: This job generates headline verification graphics
# for the CAM component (deterministic CAMs)
######################################################################
5 changes: 5 additions & 0 deletions dev/drivers/scripts/plots/cam/jevs_cam_precip_plots.sh
Original file line number Diff line number Diff line change
Expand Up @@ -55,3 +55,8 @@ export COMOUT=/lfs/h2/emc/ptmp/$USER/$NET/$evs_ver_2d/$STEP/$COMPONENT

# Job Settings and Run
. ${HOMEevs}/jobs/JEVS_CAM_PLOTS

######################################################################
# Purpose: This job generates precip verification graphics
# for the CAM component (deterministic CAMs)
######################################################################
Copy link
Contributor

@malloryprow malloryprow Aug 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nproc was increase to 128 in the job specifications, but not further down on the driver script. In the log file for this job is the mpi/cfp command using only 64? If so we shouldn't request resources we aren't using. This seems to be common across the drivers so please check the others that don't match too.

Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
#PBS -N jevs_cam_radar_plots_00
#PBS -N jevs_cam_radar_nbrcnt_plots_last31days_00
#PBS -j oe
#PBS -S /bin/bash
#PBS -q dev
#PBS -A VERF-DEV
#PBS -l walltime=1:50:00
#PBS -l place=vscatter:exclhost,select=1:ncpus=64
#PBS -l place=vscatter:exclhost,select=1:ncpus=128
#PBS -l debug=true
#PBS -V

Expand All @@ -31,34 +31,34 @@ module reset
module load prod_envir/${prod_envir_ver}

source $HOMEevs/dev/modulefiles/$COMPONENT/${COMPONENT}_${STEP}.sh
evs_ver_2d=$(echo $evs_ver | cut -d'.' -f1-2)


############################################################
# For dev testing
############################################################
export envir=prod
export DATAROOT=/lfs/h2/emc/stmp/${USER}/evs_test/$envir/tmp
export KEEPDATA=YES
export VERIF_CASE=radar
export MODELNAME=${COMPONENT}
export job=${PBS_JOBNAME:-jevs_${MODELNAME}_${VERIF_CASE}_${LINE_TYPE}_${STEP}}
export jobid=$job.${PBS_JOBID:-$$}
export COMIN=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver
export COMOUT=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver/$STEP/$COMPONENT
export COMIN=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d
export COMOUT=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d/$STEP/$COMPONENT
export nproc=64
############################################################

export vhr=${vhr:-${vhr}}
export EVAL_PERIOD=${EVAL_PERIOD:-${EVAL_PERIOD}}
export LINE_TYPE=${LINE_TYPE:-${LINE_TYPE}}
export EVAL_PERIOD=${EVAL_PERIOD:-LAST31DAYS}
export LINE_TYPE=${LINE_TYPE:-nbrcnt}

export SENDMAIL=${SENDMAIL:-YES}
export SENDCOM=${SENDCOM:-YES}
export SENDECF=${SENDECF:-YES}
export SENDDBN=${SENDDBN:-NO}
export KEEPDATA=${KEEPDATA:-NO}
export KEEPDATA=${KEEPDATA:-YES}

export MAILTO=${MAILTO:-'[email protected],alicia.bentley@noaa.gov'}
export MAILTO=${MAILTO:-'[email protected],andrew.benjamin@noaa.gov'}

if [ -z "$MAILTO" ]; then

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
#PBS -N jevs_cam_radar_nbrcnt_plots_last90days_00
#PBS -j oe
#PBS -S /bin/bash
#PBS -q dev
#PBS -A VERF-DEV
#PBS -l walltime=1:50:00
#PBS -l place=vscatter:exclhost,select=1:ncpus=128
#PBS -l debug=true
#PBS -V


set -x

cd $PBS_O_WORKDIR


############################################################
# Load modules
############################################################


export model=evs
export NET=evs
export COMPONENT=cam
export STEP=plots
export RUN=atmos

export HOMEevs=/lfs/h2/emc/vpppg/noscrub/$USER/EVS
source $HOMEevs/versions/run.ver
module reset
module load prod_envir/${prod_envir_ver}

source $HOMEevs/dev/modulefiles/$COMPONENT/${COMPONENT}_${STEP}.sh
evs_ver_2d=$(echo $evs_ver | cut -d'.' -f1-2)


############################################################
# For dev testing
############################################################
export envir=prod
export DATAROOT=/lfs/h2/emc/stmp/${USER}/evs_test/$envir/tmp
export VERIF_CASE=radar
export MODELNAME=${COMPONENT}
export job=${PBS_JOBNAME:-jevs_${MODELNAME}_${VERIF_CASE}_${LINE_TYPE}_${STEP}}
export jobid=$job.${PBS_JOBID:-$$}
export COMIN=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d
export COMOUT=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d/$STEP/$COMPONENT
export nproc=64
############################################################

export vhr=${vhr:-${vhr}}
export EVAL_PERIOD=${EVAL_PERIOD:-LAST90DAYS}
export LINE_TYPE=${LINE_TYPE:-nbrcnt}

export SENDMAIL=${SENDMAIL:-YES}
export SENDCOM=${SENDCOM:-YES}
export SENDECF=${SENDECF:-YES}
export SENDDBN=${SENDDBN:-NO}
export KEEPDATA=${KEEPDATA:-YES}

export MAILTO=${MAILTO:-'[email protected],[email protected]'}

if [ -z "$MAILTO" ]; then

echo "MAILTO variable is not defined. Exiting without continuing."

else

# CALL executable job script here
$HOMEevs/jobs/JEVS_CAM_PLOTS

fi


######################################################################
# Purpose: This job generates radar verification graphics
# for the CAM component (deterministic and ensemble CAMs)
######################################################################

Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
#PBS -N jevs_cam_radar_nbrctc_plots_last31days_00
#PBS -j oe
#PBS -S /bin/bash
#PBS -q dev
#PBS -A VERF-DEV
#PBS -l walltime=1:50:00
#PBS -l place=vscatter:exclhost,select=1:ncpus=128
#PBS -l debug=true
#PBS -V


set -x

cd $PBS_O_WORKDIR


############################################################
# Load modules
############################################################


export model=evs
export NET=evs
export COMPONENT=cam
export STEP=plots
export RUN=atmos

export HOMEevs=/lfs/h2/emc/vpppg/noscrub/$USER/EVS
source $HOMEevs/versions/run.ver
module reset
module load prod_envir/${prod_envir_ver}

source $HOMEevs/dev/modulefiles/$COMPONENT/${COMPONENT}_${STEP}.sh
evs_ver_2d=$(echo $evs_ver | cut -d'.' -f1-2)


############################################################
# For dev testing
############################################################
export envir=prod
export DATAROOT=/lfs/h2/emc/stmp/${USER}/evs_test/$envir/tmp
export VERIF_CASE=radar
export MODELNAME=${COMPONENT}
export job=${PBS_JOBNAME:-jevs_${MODELNAME}_${VERIF_CASE}_${LINE_TYPE}_${STEP}}
export jobid=$job.${PBS_JOBID:-$$}
export COMIN=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d
export COMOUT=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d/$STEP/$COMPONENT
export nproc=64
############################################################

export vhr=${vhr:-${vhr}}
export EVAL_PERIOD=${EVAL_PERIOD:-LAST31DAYS}
export LINE_TYPE=${LINE_TYPE:-nbrctc}

export SENDMAIL=${SENDMAIL:-YES}
export SENDCOM=${SENDCOM:-YES}
export SENDECF=${SENDECF:-YES}
export SENDDBN=${SENDDBN:-NO}
export KEEPDATA=${KEEPDATA:-YES}

export MAILTO=${MAILTO:-'[email protected],[email protected]'}

if [ -z "$MAILTO" ]; then

echo "MAILTO variable is not defined. Exiting without continuing."

else

# CALL executable job script here
$HOMEevs/jobs/JEVS_CAM_PLOTS

fi


######################################################################
# Purpose: This job generates radar verification graphics
# for the CAM component (deterministic and ensemble CAMs)
######################################################################

Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
#PBS -N jevs_cam_radar_nbrctc_plots_last90days_00
#PBS -j oe
#PBS -S /bin/bash
#PBS -q dev
#PBS -A VERF-DEV
#PBS -l walltime=1:50:00
#PBS -l place=vscatter:exclhost,select=1:ncpus=128
#PBS -l debug=true
#PBS -V


set -x

cd $PBS_O_WORKDIR


############################################################
# Load modules
############################################################


export model=evs
export NET=evs
export COMPONENT=cam
export STEP=plots
export RUN=atmos

export HOMEevs=/lfs/h2/emc/vpppg/noscrub/$USER/EVS
source $HOMEevs/versions/run.ver
module reset
module load prod_envir/${prod_envir_ver}

source $HOMEevs/dev/modulefiles/$COMPONENT/${COMPONENT}_${STEP}.sh
evs_ver_2d=$(echo $evs_ver | cut -d'.' -f1-2)


############################################################
# For dev testing
############################################################
export envir=prod
export DATAROOT=/lfs/h2/emc/stmp/${USER}/evs_test/$envir/tmp
export VERIF_CASE=radar
export MODELNAME=${COMPONENT}
export job=${PBS_JOBNAME:-jevs_${MODELNAME}_${VERIF_CASE}_${LINE_TYPE}_${STEP}}
export jobid=$job.${PBS_JOBID:-$$}
export COMIN=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d
export COMOUT=/lfs/h2/emc/vpppg/noscrub/${USER}/$NET/$evs_ver_2d/$STEP/$COMPONENT
export nproc=64
############################################################

export vhr=${vhr:-${vhr}}
export EVAL_PERIOD=${EVAL_PERIOD:-LAST90DAYS}
export LINE_TYPE=${LINE_TYPE:-nbrctc}

export SENDMAIL=${SENDMAIL:-YES}
export SENDCOM=${SENDCOM:-YES}
export SENDECF=${SENDECF:-YES}
export SENDDBN=${SENDDBN:-NO}
export KEEPDATA=${KEEPDATA:-YES}

export MAILTO=${MAILTO:-'[email protected],[email protected]'}

if [ -z "$MAILTO" ]; then

echo "MAILTO variable is not defined. Exiting without continuing."

else

# CALL executable job script here
$HOMEevs/jobs/JEVS_CAM_PLOTS

fi


######################################################################
# Purpose: This job generates radar verification graphics
# for the CAM component (deterministic and ensemble CAMs)
######################################################################

Loading