Port workflow to Orion (ufs-community#309)

## DESCRIPTION OF CHANGES: * Add stanzas for Orion where necessary. * Add new module files for Orion. * On Orion, both the slurm partition and the slurm QOS need to be specified in the rocoto XML in order to be able to have wall times longer than 30 mins (the partition needs to be specified because it is by default "debug", which has a limit of 30 mins). Thus, introduce modifications to more easily specify slurm partitions: * Remove the workflow variables QUEUE_DEFAULT_TAG, QUEUE_HPSS_TAG, and QUEUE_FCST_TAG that are currently used to determine whether QUEUE_DEFAULT, QUEUE_HPSS, and QUEUE_FCST specify the names of queue/QOS's or slurm partitions. * Add the workflow variables PARTITION_DEFAULT_TAG, PARTITION_HPSS_TAG, and PARTITION_FCST_TAG. These will be used to specify slurm partitions only, and the variables QUEUE_DEFAULT, QUEUE_HPSS, and QUEUE_FCST will be used to specify queues/QOS's only. IMPORTANT NOTE: On Orion, in order to load the regional_workflow environment needed for generating an experiment, the user must first issue the following commands: ``` module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles module load miniconda3 conda activate regional_workflow ``` ## TESTS CONDUCTED: Ran 11 WE2E tests on Orion, Hera, and Cheyenne. Results on Orion: ``` community_ensemble_2mems SUCCESS DOT_OR_USCORE SUCCESS grid_GSD_HRRR_AK_50km FAILURE - In the run_fcst task. * Error message: !!! (1) Error in subr radiation_aerosols: unrealistic surface pressure = 1 NaN new_ESGgrid SUCCESS new_GFDLgrid SUCCESS regional_001 SUCCESS regional_002 SUCCESS suite_FV3_GFS_v15p2 SUCCESS suite_FV3_GFS_v16beta SUCCESS suite_FV3_GSD_SAR SUCCESS suite_FV3_GSD_v0 SUCCESS ``` Results on Hera: ``` community_ensemble_2mems SUCCESS DOT_OR_USCORE SUCCESS grid_GSD_HRRR_AK_50km SUCCESS new_ESGgrid SUCCESS new_GFDLgrid SUCCESS regional_001 SUCCESS regional_002 SUCCESS suite_FV3_GFS_v15p2 SUCCESS suite_FV3_GFS_v16beta SUCCESS suite_FV3_GSD_SAR SUCCESS suite_FV3_GSD_v0 SUCCESS ``` Results on Cheyenne: ``` community_ensemble_2mems SUCCESS DOT_OR_USCORE SUCCESS grid_GSD_HRRR_AK_50km FAILURE - In run_fcst task. * Error message: !!! (1) Error in subr radiation_aerosols: unrealistic surface pressure = 1 NaN new_ESGgrid SUCCESS new_GFDLgrid SUCCESS regional_001 SUCCESS regional_002 SUCCESS suite_FV3_GFS_v15p2 SUCCESS suite_FV3_GFS_v16beta SUCCESS suite_FV3_GSD_SAR SUCCESS suite_FV3_GSD_v0 SUCCESS ``` All succeed except GSD_HRRR_AK_50km on Orion and Cheyenne. It is not clear why grid_GSD_HRRR_AK_50km fails on Orion and Cheyenne but not Hera. Seems to point to a bug in the forecast model. These two failures are not so important since this grid will soon be deprecated. Also tested successfully on Jet by @JeffBeck-NOAA and on Odin and Stampede by @ywangwof. ## ISSUE: This resolves Issue ufs-community#152. ## CONTRIBUTORS: @JeffBeck-NOAA @ywangwof @christinaholtNOAA
natalie-perlin · Oct 26, 2020 · fb8f539 · fb8f539
1 parent 06ac876
commit fb8f539
Show file tree

Hide file tree

Showing 24 changed files with 602 additions and 485 deletions.
diff --git a/modulefiles/tasks/orion/get_extrn_ics b/modulefiles/tasks/orion/get_extrn_ics
@@ -0,0 +1,6 @@
+#%Module#####################################################
+## Module file for get_extrn_ics task.
+#############################################################
+
+module purge
+
diff --git a/modulefiles/tasks/orion/get_extrn_lbcs b/modulefiles/tasks/orion/get_extrn_lbcs
@@ -0,0 +1,6 @@
+#%Module#####################################################
+## Module file for get_extrn_lbcs task.
+#############################################################
+
+module purge
+
diff --git a/modulefiles/tasks/orion/make_grid.local b/modulefiles/tasks/orion/make_grid.local
@@ -0,0 +1,5 @@
+module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles
+module load miniconda3
+if [module-info mode load] {
+  system "conda activate regional_workflow"
+}
diff --git a/modulefiles/tasks/orion/make_ics.local b/modulefiles/tasks/orion/make_ics.local
@@ -0,0 +1,5 @@
+module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles
+module load miniconda3
+if [module-info mode load] {
+  system "conda activate regional_workflow"
+}
diff --git a/modulefiles/tasks/orion/make_lbcs.local b/modulefiles/tasks/orion/make_lbcs.local
@@ -0,0 +1,5 @@
+module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles
+module load miniconda3
+if [module-info mode load] {
+  system "conda activate regional_workflow"
+}
diff --git a/modulefiles/tasks/orion/run_fcst.local b/modulefiles/tasks/orion/run_fcst.local
@@ -0,0 +1,5 @@
+module use -a /apps/contrib/miniconda3-noaa-gsl/modulefiles
+module load miniconda3
+if [module-info mode load] {
+  system "conda activate regional_workflow"
+}