Improvements to WE2E testing system (NOAA-EMC#296)

This PR modifies the script that runs the WE2E tests (run_experiments.sh) as well as the individual WE2E configuration files to allow tests to run on hera and cheyenne without the need to manually change settings (e.g. directories) in the individual test configuration files.  This capability can be easily extended to other platforms by adding appropriate stanzas in run_experiments.sh.

## DESCRIPTION OF CHANGES:
* Set the following workflow parameters in the run_experiments.sh script and write them to the workflow configuration file instead of having them defined in each WE2E configuration file (i.e. remove them from each WE2E configuration file):
 MACHINE, ACCOUNT, EXPT_SUBDIR, USE_CRON_TO_RELAUNCH, CRON_RELAUNCH_INTVL_MNTS, VERBOSE
Note that all these parameters except EXPT_SUBDIR can now be set on the command line when calling run_experiments.sh.  If they are not set on the command line, they get set to default values.  Also, EXPT_SUBDIR always gets set to the name of the WE2E test.
* Add new arguments stmp, ptmp, and verbose to run_experiments.sh so that users can specify them on the command line if they don't like the defaults.
* In run_experiments.sh, source the default workflow configuration file (config_defaults.sh) to have all user-specifiable workflow variables defined in some way (even if some of those are set to nonsensical default values).
* Put in a check to make sure the CCPP physics suite definition file exists in the ufs-weather-model repo.
* Bug fix:  Change default blending halo (HALO_BLEND) to 0 (no blending) to avoid bug in halo-blending PR.  Bug is that the make_lbcs task does not create a blending zone (i.e. it assumes halo_blend is zero).

## TESTS CONDUCTED:
Ran all WE2E tests except user_donwload_extrn_files on hera and cheyenne.  All passed except regional_010, which has a preexisting bug related to FV3 namelist settings.  Did not run user_donwload_extrn_files because it interrupts the progression of the test script (it needs improvements; but this PR does not affect downloading of external model files from NOMADS).

Note that on cheyenne, the make_ics, make_lbcs, and run_post tasks often have to be run multiple times before they succeed (especially the latter two).

tests/baseline_configs/config.FV3_RRFS_v1beta.sh

@@ -1,23 +1,3 @@
-#
-# The values of the variables MACHINE, ACCOUNT, and EXPT_SUBDIR are required
-# inputs to the script that launces the WE2E test experiments.  That script 
-# will use those inputs to fill in the values of these variables below.
-#
-MACHINE=""
-ACCOUNT=""
-EXPT_SUBDIR=""
-#
-# The values of the variables USE_CRON_TO_RELAUNCH and CRON_RELAUNCH_INTVL_MNTS
-# are optional inputs to the script that launces the WE2E test experiments.  
-# If one or both of these values are specified, then that script will 
-# replace the default values of these variables below with those values.
-# Otherwise, it will keep the default values.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="community"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.GSD_RAP13km.sh

@@ -1,33 +1,3 @@
-#
-# MACHINE will be set by the workflow launch script (launch_FV3LAM_-
-# wflow.sh) to value passed in as an argument to that script.
-#
-MACHINE=""
-#
-# ACCOUNT will be set by the workflow launch script (launch_FV3LAM_-
-# wflow.sh) to value passed in as an argument to that script.
-#
-ACCOUNT=""
-#
-# EXPT_SUBDIR will be set by the workflow launch script (launch_FV3LAM_-
-# wflow.sh) to a value obtained from the name of this file.
-#
-EXPT_SUBDIR=""
-#
-# USE_CRON_TO_RELAUNCH may be reset by the workflow launch script
-# (launch_FV3LAM_wflow.sh) to value passed in as an argument to that
-# script, but in case it is not, we give it a default value here.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-#
-# CRON_RELAUNCH_INTVL_MNTS may be reset by the workflow launch script
-# (launch_FV3LAM_wflow.sh) to value passed in as an argument to that
-# script, but in case it is not, we give it a default value here.
-#
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="community"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.community_ensemble_008mems.sh

@@ -1,23 +1,3 @@
-#
-# The values of the variables MACHINE, ACCOUNT, and EXPT_SUBDIR are required
-# inputs to the script that launces the WE2E test experiments.  That script 
-# will use those inputs to fill in the values of these variables below.
-#
-MACHINE=""
-ACCOUNT=""
-EXPT_SUBDIR=""
-#
-# The values of the variables USE_CRON_TO_RELAUNCH and CRON_RELAUNCH_INTVL_MNTS
-# are optional inputs to the script that launces the WE2E test experiments.  
-# If one or both of these values are specified, then that script will 
-# replace the default values of these variables below with those values.
-# Otherwise, it will keep the default values.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="community"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.community_ensemble_2mems.sh

@@ -1,23 +1,3 @@
-#
-# The values of the variables MACHINE, ACCOUNT, and EXPT_SUBDIR are required
-# inputs to the script that launces the WE2E test experiments.  That script 
-# will use those inputs to fill in the values of these variables below.
-#
-MACHINE=""
-ACCOUNT=""
-EXPT_SUBDIR=""
-#
-# The values of the variables USE_CRON_TO_RELAUNCH and CRON_RELAUNCH_INTVL_MNTS
-# are optional inputs to the script that launces the WE2E test experiments.  
-# If one or both of these values are specified, then that script will 
-# replace the default values of these variables below with those values.
-# Otherwise, it will keep the default values.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="community"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.nco_GSD_HRRR25km_HRRRX_RAPX.sh

@@ -1,23 +1,3 @@
-#
-# The values of the variables MACHINE, ACCOUNT, and EXPT_SUBDIR are required
-# inputs to the script that launces the WE2E test experiments.  That script 
-# will use those inputs to fill in the values of these variables below.
-#
-MACHINE=""
-ACCOUNT=""
-EXPT_SUBDIR=""
-#
-# The values of the variables USE_CRON_TO_RELAUNCH and CRON_RELAUNCH_INTVL_MNTS
-# are optional inputs to the script that launces the WE2E test experiments.  
-# If one or both of these values are specified, then that script will 
-# replace the default values of these variables below with those values.
-# Otherwise, it will keep the default values.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="nco"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.nco_GSD_HRRR3km_FV3GFS_FV3GFS.sh

@@ -1,23 +1,3 @@
-#
-# The values of the variables MACHINE, ACCOUNT, and EXPT_SUBDIR are required
-# inputs to the script that launces the WE2E test experiments.  That script 
-# will use those inputs to fill in the values of these variables below.
-#
-MACHINE=""
-ACCOUNT=""
-EXPT_SUBDIR=""
-#
-# The values of the variables USE_CRON_TO_RELAUNCH and CRON_RELAUNCH_INTVL_MNTS
-# are optional inputs to the script that launces the WE2E test experiments.  
-# If one or both of these values are specified, then that script will 
-# replace the default values of these variables below with those values.
-# Otherwise, it will keep the default values.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="nco"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.nco_GSD_HRRR3km_HRRRX_RAPX.sh

@@ -1,23 +1,3 @@
-#
-# The values of the variables MACHINE, ACCOUNT, and EXPT_SUBDIR are required
-# inputs to the script that launces the WE2E test experiments.  That script 
-# will use those inputs to fill in the values of these variables below.
-#
-MACHINE=""
-ACCOUNT=""
-EXPT_SUBDIR=""
-#
-# The values of the variables USE_CRON_TO_RELAUNCH and CRON_RELAUNCH_INTVL_MNTS
-# are optional inputs to the script that launces the WE2E test experiments.  
-# If one or both of these values are specified, then that script will 
-# replace the default values of these variables below with those values.
-# Otherwise, it will keep the default values.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="nco"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.nco_GSD_SUBCONUS3km_HRRRX_RAPX.sh

@@ -1,23 +1,3 @@
-#
-# The values of the variables MACHINE, ACCOUNT, and EXPT_SUBDIR are required
-# inputs to the script that launces the WE2E test experiments.  That script 
-# will use those inputs to fill in the values of these variables below.
-#
-MACHINE=""
-ACCOUNT=""
-EXPT_SUBDIR=""
-#
-# The values of the variables USE_CRON_TO_RELAUNCH and CRON_RELAUNCH_INTVL_MNTS
-# are optional inputs to the script that launces the WE2E test experiments.  
-# If one or both of these values are specified, then that script will 
-# replace the default values of these variables below with those values.
-# Otherwise, it will keep the default values.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="nco"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.nco_conus.sh

@@ -1,23 +1,3 @@
-#
-# The values of the variables MACHINE, ACCOUNT, and EXPT_SUBDIR are required
-# inputs to the script that launces the WE2E test experiments.  That script 
-# will use those inputs to fill in the values of these variables below.
-#
-MACHINE=""
-ACCOUNT=""
-EXPT_SUBDIR=""
-#
-# The values of the variables USE_CRON_TO_RELAUNCH and CRON_RELAUNCH_INTVL_MNTS
-# are optional inputs to the script that launces the WE2E test experiments.  
-# If one or both of these values are specified, then that script will 
-# replace the default values of these variables below with those values.
-# Otherwise, it will keep the default values.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="nco"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.nco_conus_c96.sh

@@ -1,23 +1,3 @@
-#
-# The values of the variables MACHINE, ACCOUNT, and EXPT_SUBDIR are required
-# inputs to the script that launces the WE2E test experiments.  That script 
-# will use those inputs to fill in the values of these variables below.
-#
-MACHINE=""
-ACCOUNT=""
-EXPT_SUBDIR=""
-#
-# The values of the variables USE_CRON_TO_RELAUNCH and CRON_RELAUNCH_INTVL_MNTS
-# are optional inputs to the script that launces the WE2E test experiments.  
-# If one or both of these values are specified, then that script will 
-# replace the default values of these variables below with those values.
-# Otherwise, it will keep the default values.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="nco"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.nco_ensemble.sh

@@ -1,23 +1,3 @@
-#
-# The values of the variables MACHINE, ACCOUNT, and EXPT_SUBDIR are required
-# inputs to the script that launces the WE2E test experiments.  That script 
-# will use those inputs to fill in the values of these variables below.
-#
-MACHINE=""
-ACCOUNT=""
-EXPT_SUBDIR=""
-#
-# The values of the variables USE_CRON_TO_RELAUNCH and CRON_RELAUNCH_INTVL_MNTS
-# are optional inputs to the script that launces the WE2E test experiments.  
-# If one or both of these values are specified, then that script will 
-# replace the default values of these variables below with those values.
-# Otherwise, it will keep the default values.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="nco"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.new_ESGgrid.sh

@@ -1,33 +1,3 @@
-#
-# MACHINE will be set by the workflow launch script (launch_FV3LAM_-
-# wflow.sh) to value passed in as an argument to that script.
-#
-MACHINE=""
-#
-# ACCOUNT will be set by the workflow launch script (launch_FV3LAM_-
-# wflow.sh) to value passed in as an argument to that script.
-#
-ACCOUNT=""
-#
-# EXPT_SUBDIR will be set by the workflow launch script (launch_FV3LAM_-
-# wflow.sh) to a value obtained from the name of this file.
-#
-EXPT_SUBDIR=""
-#
-# USE_CRON_TO_RELAUNCH may be reset by the workflow launch script
-# (launch_FV3LAM_wflow.sh) to value passed in as an argument to that
-# script, but in case it is not, we give it a default value here.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-#
-# CRON_RELAUNCH_INTVL_MNTS may be reset by the workflow launch script
-# (launch_FV3LAM_wflow.sh) to value passed in as an argument to that
-# script, but in case it is not, we give it a default value here.
-#
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="community"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.regional_001.sh

@@ -1,23 +1,3 @@
-#
-# The values of the variables MACHINE, ACCOUNT, and EXPT_SUBDIR are required
-# inputs to the script that launces the WE2E test experiments.  That script 
-# will use those inputs to fill in the values of these variables below.
-#
-MACHINE=""
-ACCOUNT=""
-EXPT_SUBDIR=""
-#
-# The values of the variables USE_CRON_TO_RELAUNCH and CRON_RELAUNCH_INTVL_MNTS
-# are optional inputs to the script that launces the WE2E test experiments.  
-# If one or both of these values are specified, then that script will 
-# replace the default values of these variables below with those values.
-# Otherwise, it will keep the default values.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="community"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.regional_002.sh

@@ -1,23 +1,3 @@
-#
-# The values of the variables MACHINE, ACCOUNT, and EXPT_SUBDIR are required
-# inputs to the script that launces the WE2E test experiments.  That script 
-# will use those inputs to fill in the values of these variables below.
-#
-MACHINE=""
-ACCOUNT=""
-EXPT_SUBDIR=""
-#
-# The values of the variables USE_CRON_TO_RELAUNCH and CRON_RELAUNCH_INTVL_MNTS
-# are optional inputs to the script that launces the WE2E test experiments.  
-# If one or both of these values are specified, then that script will 
-# replace the default values of these variables below with those values.
-# Otherwise, it will keep the default values.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="community"
 PREEXISTING_DIR_METHOD="rename"
 

tests/baseline_configs/config.regional_003.sh

@@ -1,23 +1,3 @@
-#
-# The values of the variables MACHINE, ACCOUNT, and EXPT_SUBDIR are required
-# inputs to the script that launces the WE2E test experiments.  That script 
-# will use those inputs to fill in the values of these variables below.
-#
-MACHINE=""
-ACCOUNT=""
-EXPT_SUBDIR=""
-#
-# The values of the variables USE_CRON_TO_RELAUNCH and CRON_RELAUNCH_INTVL_MNTS
-# are optional inputs to the script that launces the WE2E test experiments.  
-# If one or both of these values are specified, then that script will 
-# replace the default values of these variables below with those values.
-# Otherwise, it will keep the default values.
-#
-USE_CRON_TO_RELAUNCH="TRUE"
-CRON_RELAUNCH_INTVL_MNTS="02"
-
-VERBOSE="TRUE"
-
 RUN_ENVIR="community"
 PREEXISTING_DIR_METHOD="rename"