Feature 1230 stratosphere metrics (#1354)

* Initial checkin for Meridial Mean use case

* Changed the name and directories

* Added some documentation

* Fixed config variables

* Issue 1230 Stratospheric metrics (zonal/meridional use case) put INPUT_BASE in the user_env_vars so the user doesn't need to set the INPUT_BASE environment in the current working shell

* Issue #1230 remove import of metplotpy-this isn't called

* Issue #1230_stratosphere_metric Add use case to the list of use cases to be run for testing

* Issue 1230 Remove the INPUT_BASE from the user_env_vars section

* Issue #1230 redundant files

* Issue #1230 redundant files

* Issue #1230 redundant files

* Issue #120 replace INPUT_BASE with INPUT_FILE_NAME

* Issue #1230 remove entry for INPUT_FILE_NAME, this goes in the system.conf

* Update all_use_cases.txt

Copy and pasted #11 from s2s use case for the Stratosphere use case but forgot to update the index to 12.

* Update use_case_groups.json

added use case #12 from s2s to test stratosphere use case

* Issue #1230 put INPUT_FILE_NAME back under the user_env_vars

* Issue #1230 forgot to include the filename

* Issue #1230 clean up config file, remove uneccessary comments, group related entries in the user_env_vars

* Issue #1230 remove extraneous and incorrect path to the input_filename setting

* Removed pingouin dependency

* Issue #1230 another cut and paste error fixed for the Stratosphere use case

* Issue #1230 type in name, obs_Only should be obsOnly

* Issue #1230 added the metdatadb to the env, code imports metdatadb and may require some of these dependencies

* issue #1230 removed pingouin dependency from comment to reduce any confusion

* issue #1230 turn off the test for the Stratosphere metrics use case

* Added use case image

Co-authored-by: Hank Fisher <[email protected]>
Co-authored-by: Minna Win <[email protected]>
Co-authored-by: bikegeek <[email protected]>
Co-authored-by: Christina Kalb <[email protected]>

.github/parm/use_case_groups.json

@@ -154,6 +154,11 @@
     "index_list": "11",
     "run": false
   },
+  {
+    "category": "s2s",
+    "index_list": "12",
+    "run": false 
+  },
   {
     "category": "space_weather",
     "index_list": "0-1",

ci/docker/docker_env/scripts/metplotpy_env.sh

@@ -10,7 +10,6 @@
 #   matplotlib==3.3.0
 #   scipy==1.5.1
 #   plotly==4.9.0
-#   pingouin==0.3.8
 #   cartopy==0.18.0
 #   eofs==1.3.0
 #   cmocean==2.0
@@ -34,7 +33,6 @@ conda create -y --clone ${BASE_ENV} --name ${ENV_NAME}
 conda install -y --name ${ENV_NAME} -c conda-forge matplotlib==3.3.0
 conda install -y --name ${ENV_NAME} -c conda-forge scipy==1.5.1
 conda install -y --name ${ENV_NAME} -c conda-forge plotly==4.9.0
-conda install -y --name ${ENV_NAME} -c conda-forge pingouin==0.3.8
 conda install -y --name ${ENV_NAME} -c conda-forge cartopy==0.18.0
 conda install -y --name ${ENV_NAME} -c conda-forge eofs==1.3.0
 conda install -y --name ${ENV_NAME} -c conda-forge cmocean==2.0

docs/use_cases/model_applications/s2s/UserScript_obsERA_obsOnly_Stratosphere.py

@@ -0,0 +1,137 @@
+"""
+UserScript: Make zonal and meridonial means 
+========================================================================
+
+model_applications/
+s2s/
+UserScript_obsERA_obsOnly_Stratosphere.py
+
+"""
+
+##############################################################################
+# Scientific Objective
+# --------------------
+#
+# This use case calls functions in METcalcpy to create zonal and meridonial 
+# means 
+#
+
+##############################################################################
+# Datasets
+# --------
+#
+# SSWC_v1.0_varFull_ERAi_d20130106_s20121107_e20130307_c20160701.nc
+
+##############################################################################
+# METplus Components
+# ------------------
+#
+# This use case runs the UserScript wrapper tool to run a user provided script,
+# in this case, meridonial.py.
+#
+
+##############################################################################
+# METplus Workflow
+# ----------------
+#
+# This use case does not loop but plots the entire time period of data
+# 
+# UserScript
+# This uses data from 20130106,20121107,20130307,20160701 
+#
+#
+
+##############################################################################
+# METplus Configuration
+# ---------------------
+#
+# METplus first loads all of the configuration files found in parm/metplus_config,
+# then it loads any configuration files passed to METplus via the command line
+# with the -c option, i.e. -c parm/use_cases/model_applications/s2s/UserScript_obsERA_obsOnly_Stratosphere.conf
+#
+# .. highlight:: bash
+# .. literalinclude:: ../../../../parm/use_cases/model_applications/s2s/UserScript_obsERA_obsOnly_Stratosphere.conf
+#
+
+#############################################################################
+# MET Configuration
+# ---------------------
+#
+# There are no MET tools used in this use case.
+#
+
+##############################################################################
+# Python Embedding
+# ----------------
+#
+# There is no python embedding in this use case
+#
+
+##############################################################################
+# Running METplus
+# ---------------
+#
+# This use case can be run two ways:
+#
+# 1) Passing in meridonial_means.conf, 
+# then a user-specific system configuration file::
+#
+#        run_metplus.py \
+#        -c /path/to/METplus/parm/use_cases/model_applications/s2s/UserScript_obsERA_obsOnly_Stratosphere.conf \
+#        -c /path/to/user_system.conf
+#
+# 2) Modifying the configurations in parm/metplus_config, then passing in meridonial.conf::
+#
+#        run_metplus.py \
+#        -c /path/to/METplus/parm/use_cases/model_applications/s2s/UserScript_obsERA_obsOnly_Stratosphere.conf
+#
+# The former method is recommended. Whether you add them to a user-specific configuration file or modify the metplus_config files, the following variables must be set correctly:
+#
+# * **INPUT_BASE** - Path to directory where sample data tarballs are unpacked (See Datasets section to obtain tarballs). This is not required to run METplus, but it is required to run the examples in parm/use_cases
+# * **OUTPUT_BASE** - Path where METplus output will be written. This must be in a location where you have write permissions
+# * **MET_INSTALL_DIR** - Path to location where MET is installed locally
+#
+#  and for the [exe] section, you will need to define the location of NON-MET executables.
+#  If the executable is in the user's path, METplus will find it from the name. 
+#  If the executable is not in the path, specify the full path to the executable here (i.e. RM = /bin/rm)  
+#  The following executables are required for performing series analysis use cases:
+#
+# Example User Configuration File::
+#
+#   [dir]
+#   INPUT_BASE = /path/to/sample/input/data
+#   OUTPUT_BASE = /path/to/output/dir
+#   MET_INSTALL_DIR = /path/to/met-X.Y
+#
+#   [exe]
+#   RM = /path/to/rm
+#   CUT = /path/to/cut
+#   TR = /path/to/tr
+#   NCAP2 = /path/to/ncap2
+#   CONVERT = /path/to/convert
+#   NCDUMP = /path/to/ncdump
+#
+
+##############################################################################
+# Expected Output
+# ---------------
+#
+# A successful run will output the following both to the screen and to the logfile::
+#
+#   INFO: METplus has successfully finished running.
+#
+
+##############################################################################
+# Keywords
+# --------
+#
+# .. note::
+#
+#   * UserScriptUseCase
+#   * S2SAppUseCase
+#
+#   Navigate to the :ref:`quick-search` page to discover other similar use cases.
+#
+#
+#
+# sphinx_gallery_thumbnail_path = '_static/s2s-zonal_means.png'

internal_tests/use_cases/all_use_cases.txt

@@ -130,6 +130,7 @@ Category: s2s
 9:: UserScript_obsERA_obsOnly_OMI:: model_applications/s2s/UserScript_obsERA_obsOnly_OMI.conf:: spacetime_env, metdatadb
 10:: UserScript_obsERA_obsOnly_RMM:: model_applications/s2s/UserScript_obsERA_obsOnly_RMM.conf:: spacetime_env, metdatadb
 11:: UserScript_fcstGFS_obsERA_WeatherRegime:: model_applications/s2s/UserScript_fcstGFS_obsERA_WeatherRegime.conf:: weatherregime_env,cartopy,metplus
+12:: UserScript_obsERA_obsOnly_Stratosphere:: model_applications/s2s/UserScript_obsERA_obsOnly_Stratosphere.conf:: metplotpy_env,metdatadb,metplus
 
 Category: space_weather
 0::GridStat_fcstGloTEC_obsGloTEC_vx7:: model_applications/space_weather/GridStat_fcstGloTEC_obsGloTEC_vx7.conf

parm/use_cases/model_applications/s2s/UserScript_obsERA_obsOnly_Stratosphere.conf

@@ -0,0 +1,56 @@
+
+[config]
+
+# time looping - options are INIT, VALID, RETRO, and REALTIME
+# If set to INIT or RETRO:
+#   INIT_TIME_FMT, INIT_BEG, INIT_END, and INIT_INCREMENT must also be set
+# If set to VALID or REALTIME:
+#   VALID_TIME_FMT, VALID_BEG, VALID_END, and VALID_INCREMENT must also be set
+LOOP_BY = REALTIME 
+
+# %Y = 4 digit year, %m = 2 digit month, %d = 2 digit day, etc.
+# see www.strftime.org for more information
+# %Y%m%d%H expands to YYYYMMDDHH
+VALID_TIME_FMT = %Y%m%d%H
+
+# BLank for this usecase but the parameter still needs to be there 
+VALID_BEG = 
+
+# BLank for this usecase but the parameter still needs to be there 
+VALID_END =
+
+# BLank for this usecase but the parameter still needs to be there 
+VALID_INCREMENT = 
+
+# List of forecast leads to process for each run time (init or valid)
+# In hours if units are not specified
+# If unset, defaults to 0 (don't loop through forecast leads)
+LEAD_SEQ = 
+
+# Order of loops to process data - Options are times, processes
+# Not relevant if only one item is in the PROCESS_LIST
+# times = run all wrappers in the PROCESS_LIST for a single run time, then
+#   increment the run time and run all wrappers again until all times have
+#   been evaluated.
+# processes = run the first wrapper in the PROCESS_LIST for all times
+#   specified, then repeat for the next item in the PROCESS_LIST until all
+#   wrappers have been run
+LOOP_ORDER = processes
+
+PROCESS_LIST = UserScript
+
+USER_SCRIPT_RUNTIME_FREQ = RUN_ONCE 
+
+USER_SCRIPT_COMMAND = {PARM_BASE}/use_cases/model_applications/s2s/UserScript_obsERA_obsOnly_Stratosphere/meridonial_mean.py
+
+
+[user_env_vars]
+INPUT_FILE_NAME = {INPUT_BASE}/model_applications/s2s/UserScript_obsERA_obsOnly_Stratosphere/SSWC_v1.0_varFull_ERAi_d20130106_s20121107_e20130307_c20160701.nc
+YAML_CONFIG_NAME = {METPLUS_BASE}/parm/use_cases/model_applications/s2s/UserScript_obsERA_obsOnly_Stratosphere/meridonial_mean.yaml
+
+
+LOG_FILE = "Meridonial_means.log"
+
+LOG_LEVEL = "INFO"
+
+OUTPUT_DIR = {OUTPUT_BASE}

parm/use_cases/model_applications/s2s/UserScript_obsERA_obsOnly_Stratosphere/README

@@ -0,0 +1,9 @@
+These files are a use case to show how to compute meridial and zonal means
+
+You need METcalcpy and METdatadb in your python path or your conda environment
+i.e.
+export PYTHONPATH=<path>/METdatadb:<path>/METcalcpy
+
+The file SSWC_v1.0_varFull_ERAi_d20130106_s20121107_e20130307_c20160701.nc needs to be
+on disk somewhere on your computer and referenced correctly in the file
+meridial_mean.yaml

parm/use_cases/model_applications/s2s/UserScript_obsERA_obsOnly_Stratosphere/meridonial_mean.py

@@ -0,0 +1,82 @@
+#!/usr/bin/env python3
+
+"""
+Create meridonial mean statistics
+
+"""
+import os
+import sys
+import logging
+import yaml
+import xarray as xr  # http://xarray.pydata.org/
+import metcalcpy.util.read_env_vars_in_config as readconfig
+import metcalcpy.pre_processing.directional_means as directional_means
+import METreadnc.util.read_netcdf as read_netcdf
+
+
+def main():
+    """
+    Use existing default meridonial mean config file found in METcalcpy to 
+    grab the test file    
+    """
+
+
+    """
+    Read Meridial Mean YAML configuration file
+    user can use their own, if none specified at the command line,
+    use the "default" example YAML config file, spectra_plot_coh2.py
+    Using a custom YAML reader so we can use environment variables
+    """
+
+    try:
+        input_config_file = os.getenv("YAML_CONFIG_NAME","meridonial_mean.yaml")
+        config = readconfig.parse_config(input_config_file)
+        logging.info(config)
+    except yaml.YAMLError as exc:
+        logging.error(exc)
+
+    """
+    Read METplus config file paramaters
+    """
+    #input_file_name = os.environ.get("INPUT_FILE_NAME","SSWC_v1.0_varFull_ERAi_d20130106_s20121107_e20130307_c20160701.nc")
+    input_file = config["input_filename"]
+
+    """
+    Setup logging
+    """
+    logfile = "meridonial_mean.log"
+    logging_level = os.environ.get("LOG_LEVEL","logging.INFO")
+    logging.basicConfig(stream=logfile, level=logging_level)
+
+    """
+    Read dataset
+    """
+    try:
+        logging.info('Opening ' + input_file[0])
+        file_reader = read_netcdf.ReadNetCDF()
+
+        #file_reader returns a list of xarrays even if there is only one file requested to be read
+        #so we change it from a list to a single 
+        ds = file_reader.read_into_xarray(input_file)[0]
+    except IOError as exc:
+        logging.error('Unable to open ' + input_file)
+        logging.error(exc)
+        sys.exit(1)
+    logging.debug(ds)
+    ds = ds[['uwndFull_TS','vwndFull_TS','tempFull_TS','geopFull_TS']]
+    ds = ds.rename({'timeEv60':'time',
+                    'lat':'latitude', # pyzome currently expects dimensions named latitude and longitude
+                    'lon':'longitude',
+                    'uwndFull_TS':'u',
+                    'vwndFull_TS':'v',
+                    'tempFull_TS':'T',
+                    'geopFull_TS':'Z'})
+
+    uzm = directional_means.zonal_mean(ds.u)
+    Tzm = directional_means.zonal_mean(ds.T)
+    T_6090 = directional_means.meridional_mean(Tzm, 60, 90)
+
+    print(T_6090)
+
+if __name__ == '__main__':
+    main()

parm/use_cases/model_applications/s2s/UserScript_obsERA_obsOnly_Stratosphere/meridonial_mean.yaml

@@ -0,0 +1,2 @@
+input_filename:
+- !ENV '${INPUT_FILE_NAME}'