Feature 1266 gen ens prod missing ensembles #1275

georgemccabe · 2021-11-16T18:37:04Z

This PR involves:

Adding a function to handle finding ensemble members that is used for both EnsembleStat and GenEnsProd
- Includes logic to check <APP_NAME>_N_MEMBERS against number of ensembles found
- Includes logic to remove control member from ensemble list for ease of configuration of the tool (users do not need to remove the control member from the ensemble list, it will be done automatically)
Enhanced the logic for finding input files to fill in missing files with a "MISSING" identifier if requested
Fixed bug in logic for setting ens.ens_thresh and ens.vld_thresh.
Enhanced basic GenEnsProd use case to demonstrate logic for handling control members, missing ensemble members, and thresholds for ensembles found.
Unrelated fix: Updated logic to find field info indices ( in VAR) to only consider BOTH_ variables when finding indices if looking for FCST or OBS

Pull Request Testing

Describe testing already performed for these changes:

Tested various ens.ens_thresh values to ensure correct behavior occurs. Updated use case to demonstrate expected behavior.

Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

Review code changes, ensure all use case tests pass (except noted below)

Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]

Updated documentation for use case to describe threshold and control member logic

Do these changes include sufficient testing updates? [Yes]
Will this PR result in changes to the test suite? [Yes]

Multiple use cases that run GenEnsProd and EnsembleStat will contain differences.

GenEnsProd.conf basic use case will likely have different output since configurations have changed
The only difference for EnsembleStat use cases should be the change in name of the file-list files to include "ensemble_stat" instead of "ensemble" (the logic to create the filename is used by both GenEnsProd and EnsembleStat and uses the name of the app instead of a hard-coded name) -- It would be good to check that the renamed files contain the same content, although some of the cases have already been checked and the final output remains unchanged.

Please complete this pull request review by 11/16/2021.

Pull Request Checklist

See the METplus Workflow for details.

Complete the PR definition above.
Ensure the PR title matches the feature or bugfix branch name.
Define the PR metadata, as permissions allow.
Select: Reviewer(s)
Select: Organization level software support Project or Repository level development cycle Project
Select: Milestone as the version that will include these changes
After submitting the PR, select Linked issues with the original issue number.
After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
Close the linked issue and delete your feature or bugfix branch from GitHub.

…d in EnsembleStat and GenEnsProd wrappers

…, not all are mandatory, and fill in missing FCST files with missing identifier in file list

…ng through list

…where the list isn't printed earlier

…les. List out 7 ensemble paths, 1 will be used as control and 1 does not exist in sample input data. Set n members to 6 and set ens.ens_thresh value to 0.8 so that the use case will still run with the missing ensemble. Updated documentation for use case to describe this logic.

…e BOTH keyword should only be used to find field info (VAR<n>) indices if searching for FCST or OBS. I made a quick fix to prevent BOTH from being considered if looking for another type

…r regrid_data_plane or gen_vx_mask grid info) to skip check if file exists in this case

JohnHalleyGotway

I reviewed the code/doc modifications and they all look good to me.

Please review the Use Case Tests for S2S. The output logs are huge and it looks like the differences flagged for them are NOT expected.

I reviewed the differences flagged by the GHA, and see 10 of the Use Case Tests were flagged:

Use Case Tests (met_tool_wrapper:0-29) renames file lists from "ensemble" to "ensemble_stat".
Use Case Tests (met_tool_wrapper:30-58) modifies existing TC-Gen output files due to Enhance TC-Gen to verify genesis probabilities from ATCF e-deck files. MET#1809.
Use Case Tests (air_quality_and_comp:0) renames file lists from "ensemble" to "ensemble_stat".
Use Case Tests (convection_allowing_models:0) renames file lists from "ensemble" to "ensemble_stat".
Use Case Tests (convection_allowing_models:2-6) renames file lists from "ensemble" to "ensemble_stat".
Use Case Tests (precipitation:0) renames file lists from "ensemble" to "ensemble_stat".
Use Case Tests (s2s:6-7) renames a png file:

ERROR: Some differences were found
file not found
A:/data/truth/UserScript_fcstGFS_obsERA_PhaseDiagram/s2s/UserScript_fcstGFS_obsERA_PhaseDiagram/plots/RMM_phase_diagram.png
B:

file not found (new output)
A:
B:/data/output/UserScript_obsERA_obsOnly_PhaseDiagram/metplus_final.conf

file not found (new output)
A:
B:/data/output/UserScript_obsERA_obsOnly_PhaseDiagram/s2s/UserScript_obsERA_obsOnly_PhaseDiagram/plots/RMM_phase_diagram.png

Use Case Tests (s2s:7-9) per Slack discussion, these are expected and fixed by Feature 1019 harmonic preprocessing #1272.
Use Case Tests (s2s:10) per Slack discussion, these are expected and fixed by Feature 1019 harmonic preprocessing #1272.
Use Case Tests (s2s:11) per Slack discussion, these are expected and fixed by Feature 1019 harmonic preprocessing #1272.

JohnHalleyGotway · 2021-11-16T21:19:01Z

After discussion via Slack, I approve of these changes.

The seemingly problematic S2S use cases are fixed by #1272.

Co-authored-by: George McCabe <[email protected]> Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: johnhg <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: lisagoodrich <[email protected]>

* Feature 1091 extent cycloneplotter (#1218) * Github #1091 add support to define plot's extent * Github #1091 Add support for defining plot's extent * Github #1091 add support for defining plot extent * GitHub #1091 Provide support to define plotting a bounding box of interest and replace deprecated cartopy attributes. * Github Issue #1091 Add configuration support to define the bounding box defining the area of interest to plot. * Github Issue #1091 provide support to define bounding box defining the area of interest to plot. * Github Issue #1091 Removed unused shapely imports * GItHub Issue #1091 Removed duplicate CYCLONE_PLOTTER_GLOBAL_PLOT * Github #1091 Include config settings needed to indicate the region of interest to be plotted. * Github Issue #1091 Keep only the North Hemisphere lon and lats in the config file to serve as example for plotting a specific area of the map. * convert file back to unix format via dos2unix and added change that was lost in merge * return None from function instead of exiting so that METplus clean up functionality will still run * fixed indentation * fixed indentation - for loop should not be nested inside other for loop * fixed check for failure in retrieve_data function * feature 1223 error if file not found (#1238) * feature 1252 allow dictionary value for time_summary.width (#1253) * feature 1213 obs_quality_inc/exc (#1260) * Feature 1203 ioda2nc (#1262) * Add default title for the new use case issue template. * Feature 1019 harmonic preprocessing (#1272) Co-authored-by: George McCabe <[email protected]> * Feature 1266 gen ens prod missing ensembles (#1275) * Feature 1049 statistics list (#1271) * first attempt at a table #1049 * 2nd attempt, smaller, different formatting table #1049 * trying to get rid of git error message by adding this to the index #1049 * alignment #1049 * trying to text wrap in a table #1049 * attempting to create text wrapping in tables #1049 * removing role, trying to wrap text #1049 * removing html break, trying to wrap text #1049 * adding code to wrap text #1049 * removing code to wrap text #1049 * fixing naming typo #1049 * hitting returns in table #1049 * removing typo file, removing line breaks #1049 * attempting a simple table for text wrapping #1049 * attempting to bold header row in simple table #1049 * simple table line break attempt #1049 * simple table line break attempt #2 #1049 * simple table line break attempt #3 #1049 * trying another | #1049 * 2 trying another | #1049 * 3 trying another | #1049 * trying a blank line #1049 * trying reformating grid #1049 * 2 trying reformating grid #1049 * 3 trying reformating grid #1049 * 4 trying reformating grid #1049 * adding in one more table row #1049 * adding more text #1049 * trying glossary format #1049 * 2 trying glossary format #1049 * 3 trying glossary format #1049 * 4 trying glossary format #1049 * adding glossary format 2D objects #1049 * fixing glossary format 2D objects #1049 * adding glossary items #1049 * fixing glossary format #1049 * hopefully all examples are working #1049 * changing reference to tool * removing 2D object examples * removing the unwrapped table * adding ACC into the example * fixing formatting #1049 * fixing formatting take 2 #1049 * fixing formatting take 3 #1049 * bolding and language change #1049 * fixing spacing #1049 * fixing spacing again #1049 * changing title #1049 * trying superscript #1049 * superscript glossary #1049 * removing quotes #1049 * fixing spacing #1049 * Tara has decided to go with the glossary format. Removing table example. #1049 * Adding 2D objects #1049 * 2D objects fix #1049 * 2D objects fix attempt 2 #1049 * 2D objects fix attempt 3 #1049 * 2D objects fix attempt 4 #1049 * 2D objects fix attempt 5 #1049 * 2D objects fix attempt 7 #1049 * 2D objects fix attempt 8 #1049 * ABR added #1049 * adding remaining AB items to list #1049 * adding remaining ACC_ items to list #1049 * ADLAND & AFSS entries #1049 * A entries #1049 * fixing formatting #1049 * fixing formatting #2 #1049 * removing new entries to make it run #1049 * adding a couple back in #1049 * adding a couple more back in #1049 * tool key at the bottom #1049 * spacing changes #1049 * spacing changes another try #1049 * fixing spacing #1049 * fixing spacing attempt 2 #1049 * fixing spacing attempt 3 #1049 * through AREA items #1049 * AMODEL listed twice but it's not #1049 * AMODEL listed twice but it's not #2 #1049 * AMODEL added question marks so it will run #1049 * final A entries #1049 * naming the glossary statistics in hopes of not conflicting with original glossary #1049 * removing glossary statistics space #1049 * Testing multiple glossary names * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, removing code with attempts at mutiple glossaries * Per #1067, adding back three question marks due to duplicate term * paring down list, adding a table for review * problems with table * still trying to get it to publish * still trying to get a second line in the table * still trying to get a second line in the table 2 * testing different formatting * testing different formatting * trying to comment out the glossary inner workings * formatting again * line breaks in table #1049 * adding in some more for an example #1049 * adding new entries in through AREA #1049 * fixing line breaks #1049 * fixing line breaks #1049 #2 * fixing line breaks and warning messages #1049 * fixing line breaks #1049 * adding AREA_RATIO through ASPECT_DIFF * fixing typo #1049 * fixing typo #1049 * fixing typo #1049 take 2 * adding AXIS_ANG to BCMSE #1049 * adding spacing #1049 * adding spacing removing commas #1049 * removing comma #1049 * BOUNDARY_DIST thru BSS_SMPL #1049 * BOUNDARY_DIST splitting across 2 lines #1049 * cleaning up typos #1049 * calibration thru centriod_dist #1049 * centriod_lat thru centroid_y #1049 * fixing spacing #1049 * removing test glossary #1049 * climo_mean thru crtk_err #1049 * fixing crtk_err spacing #1049 * fixing spacing #1049 * CSI to CURVATURE_Y #1049 * CURVATURE_X & Y spacing #1049 * DEV_CAT to DURATION_DIFF #1049 * EC_VALUE to F #1049 * F_RATE TO FBS #1049 * Fixing spacing #1049 * fcst_clus thru fcst_conv_radius #1049 * removing CTOP_PRS #1049 * fixing the order of tools for FBAR and FBIAS #1049 * fixing spacing #1049 * adding grid-stat to all point-stat entries #1049 * adding fixing spacing #1049 * adding fixing spacing take 2 #1049 * adding fixing spacing take 3 #1049 * adding fixing spacing take 4 #1049 * adding fixing spacing take 5 #1049 * adding fixing spacing take 6 #1049 * adding fixing spacing take 7 #1049 * fixing spacing with a period take 7 #1049 * first attempt fcst_ #1049 * fixing typos #1049 * thru FOBAR #1049 * thru end of F #1049 * g thru h #1049 * i thru intensity #s #1049 * fixing typos #1049 * capturing example for Julie #1049 * thru k #1049 * fixing typos #1049 * thru L #1049 * thru MG #1049 * thru N_ENS #1049 * thru all N #1049 * fixing FBIAS alignment #1049 * fixing ME and MSE alignment #1049 * fixing ME alignment take 2 #1049 * thru OBS_E #1049 * fixing alignment #1049 * fixing alignment n_thresh #1049 * thru OBS_thresh #1049 * thru O #1049 * fixing OOBAR formating #1049 * thru PR_CORR #1049 * commented lines out with line total info #1049 * commented lines out with line total info take 2 #1049 * thru R #1049 * thru SPEED #1049 * thru S #1049 * thru T #1049 * thru U #1049 * thru V #1049 * thru V #1049 * thru Z #1049 * Update statistics_list.rst Tara is testing editing in UI * Update statistics_list.rst Updates through the A's * Update statistics_list.rst Cleaning up the A's * Update statistics_list.rst Standardizing MODE and MTD entries * Update statistics_list.rst Updating B's and C's * Update statistics_list.rst Testing adding TC-Stat and TCST to an entry * Update statistics_list.rst Clean up of a few A-Cs and then update of Ds * Update statistics_list.rst A few clean-ups and Es * Update statistics_list.rst Halfway through Fs... * Update statistics_list.rst A little clean up and the rest of Fs * Update statistics_list.rst G, H, I, Ks * Update statistics_list.rst L, M, Ns * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst Rs * Update statistics_list.rst S and Ts * Update statistics_list.rst The rest of the list * Update statistics_list.rst Removed Attr from Stat Type thru E * Update statistics_list.rst Remove Attr from Statistics Type through Gs * Update statistics_list.rst Remove Attr from Statistic Type through Rs * Update statistics_list.rst Remove Attr from Stat Type to the end * Update statistics_list.rst Cleaned up some Line Type typos * Update statistics_list.rst Still more Attr cleanup Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> * Feature 1263 v4.1.0 beta4 (#1277) * update version to note development towards beta5 * Feature 934 release stage doc (#1235) * Per #934 add stages of the METplus release cycle. * Per #934, adding link to descriptions of the release cycle in the User's Guide. * Per #934, made corrections * Per #934, changed Beta and Release Candidate (rc) from bold to subsubsections. * Update index.rst Co-authored-by: Julie Prestopnik <[email protected]> * Feature 344 met util refactor (#1292) * removed deprecated sections from config examples * minor change to METplus release guide to add a link to the PDF of the User's Guide instead of downloading it and attaching it to the release * Feature 1285 extract tiles mtd times (#1315) * Feature 896 more met config (#1322) * removed incorrect search keyword * added workflow_dispatch event so workflow can be triggered by an external repository such as MET to test to ensure that changes from that repo will break anything in METplus * added another input argument for workflow_dispatch event * added job with name that shows the event name or the repository name if triggered by an external repository such as MET * GHA: add username that triggered external event to event info job name * added required input argument for external trigger that contains the commit hash of the push event that triggered in the other repo * change event info to show commit hash instead of username that merged the PR * changed input names to match names of event in repository that triggered workflow * feature 1320 OMP_NUM_THREADS (#1338) * Feature 1183 memory documentation (#1340) Co-authored-by: George McCabe <[email protected]> * add email address of user who triggered push event to job name * Feature 1166 series analysis field info (#1353) * Feature 1116 usecase smos (#1348) Co-authored-by: Mrinal Biswas <[email protected]> * turn off new use case from every push * feature 1236 Control Members in EnsembleStat and GenEnsProd (#1357) * added optional argument to change the directory to untar new input data into so the same Dockerfile can be used to add data for other METplus components such as MET * feature 1358 v4.1.0-beta5 release (#1359) * update version for next development cycle * Feature 1216 usecase smap (#1361) * Adding a conf file for SMAP * Adding a directory to host the read file * Removing temp file * Updated the valid dates to match Todd's code * Adding documentation for SMAP case * Updates the valis dates to match Todd's code * Removing a tmp file * Typo in file name * Updating the input RTOFS to have the init time instead of the valid time ii the file name * updated file paths, tesing * updated use case descriptions, rearranged use case group testing * put new use case into its own group so that the diff logic can evaluate marine_and_cryosphere:3. The truth data for 3-4 does not exist yet so the diff fails. Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * updated marine_and_cryo grouping Co-authored-by: bikegeek <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: George McCabe <[email protected]> Co-authored-by: johnhg <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: lisagoodrich <[email protected]> Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]>

* Feature 1091 extent cycloneplotter (#1218) * Github #1091 add support to define plot's extent * Github #1091 Add support for defining plot's extent * Github #1091 add support for defining plot extent * GitHub #1091 Provide support to define plotting a bounding box of interest and replace deprecated cartopy attributes. * Github Issue #1091 Add configuration support to define the bounding box defining the area of interest to plot. * Github Issue #1091 provide support to define bounding box defining the area of interest to plot. * Github Issue #1091 Removed unused shapely imports * GItHub Issue #1091 Removed duplicate CYCLONE_PLOTTER_GLOBAL_PLOT * Github #1091 Include config settings needed to indicate the region of interest to be plotted. * Github Issue #1091 Keep only the North Hemisphere lon and lats in the config file to serve as example for plotting a specific area of the map. * convert file back to unix format via dos2unix and added change that was lost in merge * return None from function instead of exiting so that METplus clean up functionality will still run * fixed indentation * fixed indentation - for loop should not be nested inside other for loop * fixed check for failure in retrieve_data function * feature 1223 error if file not found (#1238) * feature 1252 allow dictionary value for time_summary.width (#1253) * feature 1213 obs_quality_inc/exc (#1260) * Feature 1203 ioda2nc (#1262) * Add default title for the new use case issue template. * Feature 1019 harmonic preprocessing (#1272) Co-authored-by: George McCabe <[email protected]> * Feature 1266 gen ens prod missing ensembles (#1275) * Feature 1049 statistics list (#1271) * first attempt at a table #1049 * 2nd attempt, smaller, different formatting table #1049 * trying to get rid of git error message by adding this to the index #1049 * alignment #1049 * trying to text wrap in a table #1049 * attempting to create text wrapping in tables #1049 * removing role, trying to wrap text #1049 * removing html break, trying to wrap text #1049 * adding code to wrap text #1049 * removing code to wrap text #1049 * fixing naming typo #1049 * hitting returns in table #1049 * removing typo file, removing line breaks #1049 * attempting a simple table for text wrapping #1049 * attempting to bold header row in simple table #1049 * simple table line break attempt #1049 * simple table line break attempt #2 #1049 * simple table line break attempt #3 #1049 * trying another | #1049 * 2 trying another | #1049 * 3 trying another | #1049 * trying a blank line #1049 * trying reformating grid #1049 * 2 trying reformating grid #1049 * 3 trying reformating grid #1049 * 4 trying reformating grid #1049 * adding in one more table row #1049 * adding more text #1049 * trying glossary format #1049 * 2 trying glossary format #1049 * 3 trying glossary format #1049 * 4 trying glossary format #1049 * adding glossary format 2D objects #1049 * fixing glossary format 2D objects #1049 * adding glossary items #1049 * fixing glossary format #1049 * hopefully all examples are working #1049 * changing reference to tool * removing 2D object examples * removing the unwrapped table * adding ACC into the example * fixing formatting #1049 * fixing formatting take 2 #1049 * fixing formatting take 3 #1049 * bolding and language change #1049 * fixing spacing #1049 * fixing spacing again #1049 * changing title #1049 * trying superscript #1049 * superscript glossary #1049 * removing quotes #1049 * fixing spacing #1049 * Tara has decided to go with the glossary format. Removing table example. #1049 * Adding 2D objects #1049 * 2D objects fix #1049 * 2D objects fix attempt 2 #1049 * 2D objects fix attempt 3 #1049 * 2D objects fix attempt 4 #1049 * 2D objects fix attempt 5 #1049 * 2D objects fix attempt 7 #1049 * 2D objects fix attempt 8 #1049 * ABR added #1049 * adding remaining AB items to list #1049 * adding remaining ACC_ items to list #1049 * ADLAND & AFSS entries #1049 * A entries #1049 * fixing formatting #1049 * fixing formatting #2 #1049 * removing new entries to make it run #1049 * adding a couple back in #1049 * adding a couple more back in #1049 * tool key at the bottom #1049 * spacing changes #1049 * spacing changes another try #1049 * fixing spacing #1049 * fixing spacing attempt 2 #1049 * fixing spacing attempt 3 #1049 * through AREA items #1049 * AMODEL listed twice but it's not #1049 * AMODEL listed twice but it's not #2 #1049 * AMODEL added question marks so it will run #1049 * final A entries #1049 * naming the glossary statistics in hopes of not conflicting with original glossary #1049 * removing glossary statistics space #1049 * Testing multiple glossary names * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, removing code with attempts at mutiple glossaries * Per #1067, adding back three question marks due to duplicate term * paring down list, adding a table for review * problems with table * still trying to get it to publish * still trying to get a second line in the table * still trying to get a second line in the table 2 * testing different formatting * testing different formatting * trying to comment out the glossary inner workings * formatting again * line breaks in table #1049 * adding in some more for an example #1049 * adding new entries in through AREA #1049 * fixing line breaks #1049 * fixing line breaks #1049 #2 * fixing line breaks and warning messages #1049 * fixing line breaks #1049 * adding AREA_RATIO through ASPECT_DIFF * fixing typo #1049 * fixing typo #1049 * fixing typo #1049 take 2 * adding AXIS_ANG to BCMSE #1049 * adding spacing #1049 * adding spacing removing commas #1049 * removing comma #1049 * BOUNDARY_DIST thru BSS_SMPL #1049 * BOUNDARY_DIST splitting across 2 lines #1049 * cleaning up typos #1049 * calibration thru centriod_dist #1049 * centriod_lat thru centroid_y #1049 * fixing spacing #1049 * removing test glossary #1049 * climo_mean thru crtk_err #1049 * fixing crtk_err spacing #1049 * fixing spacing #1049 * CSI to CURVATURE_Y #1049 * CURVATURE_X & Y spacing #1049 * DEV_CAT to DURATION_DIFF #1049 * EC_VALUE to F #1049 * F_RATE TO FBS #1049 * Fixing spacing #1049 * fcst_clus thru fcst_conv_radius #1049 * removing CTOP_PRS #1049 * fixing the order of tools for FBAR and FBIAS #1049 * fixing spacing #1049 * adding grid-stat to all point-stat entries #1049 * adding fixing spacing #1049 * adding fixing spacing take 2 #1049 * adding fixing spacing take 3 #1049 * adding fixing spacing take 4 #1049 * adding fixing spacing take 5 #1049 * adding fixing spacing take 6 #1049 * adding fixing spacing take 7 #1049 * fixing spacing with a period take 7 #1049 * first attempt fcst_ #1049 * fixing typos #1049 * thru FOBAR #1049 * thru end of F #1049 * g thru h #1049 * i thru intensity #s #1049 * fixing typos #1049 * capturing example for Julie #1049 * thru k #1049 * fixing typos #1049 * thru L #1049 * thru MG #1049 * thru N_ENS #1049 * thru all N #1049 * fixing FBIAS alignment #1049 * fixing ME and MSE alignment #1049 * fixing ME alignment take 2 #1049 * thru OBS_E #1049 * fixing alignment #1049 * fixing alignment n_thresh #1049 * thru OBS_thresh #1049 * thru O #1049 * fixing OOBAR formating #1049 * thru PR_CORR #1049 * commented lines out with line total info #1049 * commented lines out with line total info take 2 #1049 * thru R #1049 * thru SPEED #1049 * thru S #1049 * thru T #1049 * thru U #1049 * thru V #1049 * thru V #1049 * thru Z #1049 * Update statistics_list.rst Tara is testing editing in UI * Update statistics_list.rst Updates through the A's * Update statistics_list.rst Cleaning up the A's * Update statistics_list.rst Standardizing MODE and MTD entries * Update statistics_list.rst Updating B's and C's * Update statistics_list.rst Testing adding TC-Stat and TCST to an entry * Update statistics_list.rst Clean up of a few A-Cs and then update of Ds * Update statistics_list.rst A few clean-ups and Es * Update statistics_list.rst Halfway through Fs... * Update statistics_list.rst A little clean up and the rest of Fs * Update statistics_list.rst G, H, I, Ks * Update statistics_list.rst L, M, Ns * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst Rs * Update statistics_list.rst S and Ts * Update statistics_list.rst The rest of the list * Update statistics_list.rst Removed Attr from Stat Type thru E * Update statistics_list.rst Remove Attr from Statistics Type through Gs * Update statistics_list.rst Remove Attr from Statistic Type through Rs * Update statistics_list.rst Remove Attr from Stat Type to the end * Update statistics_list.rst Cleaned up some Line Type typos * Update statistics_list.rst Still more Attr cleanup Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> * Feature 1263 v4.1.0 beta4 (#1277) * update version to note development towards beta5 * Feature 934 release stage doc (#1235) * Per #934 add stages of the METplus release cycle. * Per #934, adding link to descriptions of the release cycle in the User's Guide. * Per #934, made corrections * Per #934, changed Beta and Release Candidate (rc) from bold to subsubsections. * Update index.rst Co-authored-by: Julie Prestopnik <[email protected]> * Feature 344 met util refactor (#1292) * removed deprecated sections from config examples * minor change to METplus release guide to add a link to the PDF of the User's Guide instead of downloading it and attaching it to the release * Feature 1285 extract tiles mtd times (#1315) * Feature 896 more met config (#1322) * removed incorrect search keyword * added workflow_dispatch event so workflow can be triggered by an external repository such as MET to test to ensure that changes from that repo will break anything in METplus * added another input argument for workflow_dispatch event * added job with name that shows the event name or the repository name if triggered by an external repository such as MET * GHA: add username that triggered external event to event info job name * added required input argument for external trigger that contains the commit hash of the push event that triggered in the other repo * change event info to show commit hash instead of username that merged the PR * changed input names to match names of event in repository that triggered workflow * feature 1320 OMP_NUM_THREADS (#1338) * Feature 1183 memory documentation (#1340) Co-authored-by: George McCabe <[email protected]> * add email address of user who triggered push event to job name * Feature 1166 series analysis field info (#1353) * Feature 1116 usecase smos (#1348) Co-authored-by: Mrinal Biswas <[email protected]> * turn off new use case from every push * feature 1236 Control Members in EnsembleStat and GenEnsProd (#1357) * added optional argument to change the directory to untar new input data into so the same Dockerfile can be used to add data for other METplus components such as MET * feature 1358 v4.1.0-beta5 release (#1359) * update version for next development cycle * Feature 1216 usecase smap (#1361) * Adding a conf file for SMAP * Adding a directory to host the read file * Removing temp file * Updated the valid dates to match Todd's code * Adding documentation for SMAP case * Updates the valis dates to match Todd's code * Removing a tmp file * Typo in file name * Updating the input RTOFS to have the init time instead of the valid time ii the file name * updated file paths, tesing * updated use case descriptions, rearranged use case group testing * put new use case into its own group so that the diff logic can evaluate marine_and_cryosphere:3. The truth data for 3-4 does not exist yet so the diff fails. Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * updated marine_and_cryo grouping Co-authored-by: George McCabe <[email protected]> Co-authored-by: johnhg <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: lisagoodrich <[email protected]> Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: j-opatz <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: j-opatz <[email protected]>

* Feature 1091 extent cycloneplotter (#1218) * Github #1091 add support to define plot's extent * Github #1091 Add support for defining plot's extent * Github #1091 add support for defining plot extent * GitHub #1091 Provide support to define plotting a bounding box of interest and replace deprecated cartopy attributes. * Github Issue #1091 Add configuration support to define the bounding box defining the area of interest to plot. * Github Issue #1091 provide support to define bounding box defining the area of interest to plot. * Github Issue #1091 Removed unused shapely imports * GItHub Issue #1091 Removed duplicate CYCLONE_PLOTTER_GLOBAL_PLOT * Github #1091 Include config settings needed to indicate the region of interest to be plotted. * Github Issue #1091 Keep only the North Hemisphere lon and lats in the config file to serve as example for plotting a specific area of the map. * convert file back to unix format via dos2unix and added change that was lost in merge * return None from function instead of exiting so that METplus clean up functionality will still run * fixed indentation * fixed indentation - for loop should not be nested inside other for loop * fixed check for failure in retrieve_data function * feature 1223 error if file not found (#1238) * feature 1252 allow dictionary value for time_summary.width (#1253) * feature 1213 obs_quality_inc/exc (#1260) * Feature 1203 ioda2nc (#1262) * Add default title for the new use case issue template. * Feature 1019 harmonic preprocessing (#1272) Co-authored-by: George McCabe <[email protected]> * Feature 1266 gen ens prod missing ensembles (#1275) * Feature 1049 statistics list (#1271) * first attempt at a table #1049 * 2nd attempt, smaller, different formatting table #1049 * trying to get rid of git error message by adding this to the index #1049 * alignment #1049 * trying to text wrap in a table #1049 * attempting to create text wrapping in tables #1049 * removing role, trying to wrap text #1049 * removing html break, trying to wrap text #1049 * adding code to wrap text #1049 * removing code to wrap text #1049 * fixing naming typo #1049 * hitting returns in table #1049 * removing typo file, removing line breaks #1049 * attempting a simple table for text wrapping #1049 * attempting to bold header row in simple table #1049 * simple table line break attempt #1049 * simple table line break attempt #2 #1049 * simple table line break attempt #3 #1049 * trying another | #1049 * 2 trying another | #1049 * 3 trying another | #1049 * trying a blank line #1049 * trying reformating grid #1049 * 2 trying reformating grid #1049 * 3 trying reformating grid #1049 * 4 trying reformating grid #1049 * adding in one more table row #1049 * adding more text #1049 * trying glossary format #1049 * 2 trying glossary format #1049 * 3 trying glossary format #1049 * 4 trying glossary format #1049 * adding glossary format 2D objects #1049 * fixing glossary format 2D objects #1049 * adding glossary items #1049 * fixing glossary format #1049 * hopefully all examples are working #1049 * changing reference to tool * removing 2D object examples * removing the unwrapped table * adding ACC into the example * fixing formatting #1049 * fixing formatting take 2 #1049 * fixing formatting take 3 #1049 * bolding and language change #1049 * fixing spacing #1049 * fixing spacing again #1049 * changing title #1049 * trying superscript #1049 * superscript glossary #1049 * removing quotes #1049 * fixing spacing #1049 * Tara has decided to go with the glossary format. Removing table example. #1049 * Adding 2D objects #1049 * 2D objects fix #1049 * 2D objects fix attempt 2 #1049 * 2D objects fix attempt 3 #1049 * 2D objects fix attempt 4 #1049 * 2D objects fix attempt 5 #1049 * 2D objects fix attempt 7 #1049 * 2D objects fix attempt 8 #1049 * ABR added #1049 * adding remaining AB items to list #1049 * adding remaining ACC_ items to list #1049 * ADLAND & AFSS entries #1049 * A entries #1049 * fixing formatting #1049 * fixing formatting #2 #1049 * removing new entries to make it run #1049 * adding a couple back in #1049 * adding a couple more back in #1049 * tool key at the bottom #1049 * spacing changes #1049 * spacing changes another try #1049 * fixing spacing #1049 * fixing spacing attempt 2 #1049 * fixing spacing attempt 3 #1049 * through AREA items #1049 * AMODEL listed twice but it's not #1049 * AMODEL listed twice but it's not #2 #1049 * AMODEL added question marks so it will run #1049 * final A entries #1049 * naming the glossary statistics in hopes of not conflicting with original glossary #1049 * removing glossary statistics space #1049 * Testing multiple glossary names * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, removing code with attempts at mutiple glossaries * Per #1067, adding back three question marks due to duplicate term * paring down list, adding a table for review * problems with table * still trying to get it to publish * still trying to get a second line in the table * still trying to get a second line in the table 2 * testing different formatting * testing different formatting * trying to comment out the glossary inner workings * formatting again * line breaks in table #1049 * adding in some more for an example #1049 * adding new entries in through AREA #1049 * fixing line breaks #1049 * fixing line breaks #1049 #2 * fixing line breaks and warning messages #1049 * fixing line breaks #1049 * adding AREA_RATIO through ASPECT_DIFF * fixing typo #1049 * fixing typo #1049 * fixing typo #1049 take 2 * adding AXIS_ANG to BCMSE #1049 * adding spacing #1049 * adding spacing removing commas #1049 * removing comma #1049 * BOUNDARY_DIST thru BSS_SMPL #1049 * BOUNDARY_DIST splitting across 2 lines #1049 * cleaning up typos #1049 * calibration thru centriod_dist #1049 * centriod_lat thru centroid_y #1049 * fixing spacing #1049 * removing test glossary #1049 * climo_mean thru crtk_err #1049 * fixing crtk_err spacing #1049 * fixing spacing #1049 * CSI to CURVATURE_Y #1049 * CURVATURE_X & Y spacing #1049 * DEV_CAT to DURATION_DIFF #1049 * EC_VALUE to F #1049 * F_RATE TO FBS #1049 * Fixing spacing #1049 * fcst_clus thru fcst_conv_radius #1049 * removing CTOP_PRS #1049 * fixing the order of tools for FBAR and FBIAS #1049 * fixing spacing #1049 * adding grid-stat to all point-stat entries #1049 * adding fixing spacing #1049 * adding fixing spacing take 2 #1049 * adding fixing spacing take 3 #1049 * adding fixing spacing take 4 #1049 * adding fixing spacing take 5 #1049 * adding fixing spacing take 6 #1049 * adding fixing spacing take 7 #1049 * fixing spacing with a period take 7 #1049 * first attempt fcst_ #1049 * fixing typos #1049 * thru FOBAR #1049 * thru end of F #1049 * g thru h #1049 * i thru intensity #s #1049 * fixing typos #1049 * capturing example for Julie #1049 * thru k #1049 * fixing typos #1049 * thru L #1049 * thru MG #1049 * thru N_ENS #1049 * thru all N #1049 * fixing FBIAS alignment #1049 * fixing ME and MSE alignment #1049 * fixing ME alignment take 2 #1049 * thru OBS_E #1049 * fixing alignment #1049 * fixing alignment n_thresh #1049 * thru OBS_thresh #1049 * thru O #1049 * fixing OOBAR formating #1049 * thru PR_CORR #1049 * commented lines out with line total info #1049 * commented lines out with line total info take 2 #1049 * thru R #1049 * thru SPEED #1049 * thru S #1049 * thru T #1049 * thru U #1049 * thru V #1049 * thru V #1049 * thru Z #1049 * Update statistics_list.rst Tara is testing editing in UI * Update statistics_list.rst Updates through the A's * Update statistics_list.rst Cleaning up the A's * Update statistics_list.rst Standardizing MODE and MTD entries * Update statistics_list.rst Updating B's and C's * Update statistics_list.rst Testing adding TC-Stat and TCST to an entry * Update statistics_list.rst Clean up of a few A-Cs and then update of Ds * Update statistics_list.rst A few clean-ups and Es * Update statistics_list.rst Halfway through Fs... * Update statistics_list.rst A little clean up and the rest of Fs * Update statistics_list.rst G, H, I, Ks * Update statistics_list.rst L, M, Ns * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst Rs * Update statistics_list.rst S and Ts * Update statistics_list.rst The rest of the list * Update statistics_list.rst Removed Attr from Stat Type thru E * Update statistics_list.rst Remove Attr from Statistics Type through Gs * Update statistics_list.rst Remove Attr from Statistic Type through Rs * Update statistics_list.rst Remove Attr from Stat Type to the end * Update statistics_list.rst Cleaned up some Line Type typos * Update statistics_list.rst Still more Attr cleanup Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> * Feature 1263 v4.1.0 beta4 (#1277) * update version to note development towards beta5 * Feature 934 release stage doc (#1235) * Per #934 add stages of the METplus release cycle. * Per #934, adding link to descriptions of the release cycle in the User's Guide. * Per #934, made corrections * Per #934, changed Beta and Release Candidate (rc) from bold to subsubsections. * Update index.rst Co-authored-by: Julie Prestopnik <[email protected]> * Feature 344 met util refactor (#1292) * removed deprecated sections from config examples * minor change to METplus release guide to add a link to the PDF of the User's Guide instead of downloading it and attaching it to the release * Feature 1285 extract tiles mtd times (#1315) * Feature 896 more met config (#1322) * removed incorrect search keyword * added workflow_dispatch event so workflow can be triggered by an external repository such as MET to test to ensure that changes from that repo will break anything in METplus * added another input argument for workflow_dispatch event * added job with name that shows the event name or the repository name if triggered by an external repository such as MET * GHA: add username that triggered external event to event info job name * added required input argument for external trigger that contains the commit hash of the push event that triggered in the other repo * change event info to show commit hash instead of username that merged the PR * changed input names to match names of event in repository that triggered workflow * feature 1320 OMP_NUM_THREADS (#1338) * Feature 1183 memory documentation (#1340) Co-authored-by: George McCabe <[email protected]> * add email address of user who triggered push event to job name * Feature 1166 series analysis field info (#1353) * Feature 1116 usecase smos (#1348) Co-authored-by: Mrinal Biswas <[email protected]> * turn off new use case from every push * feature 1236 Control Members in EnsembleStat and GenEnsProd (#1357) * added optional argument to change the directory to untar new input data into so the same Dockerfile can be used to add data for other METplus components such as MET * feature 1358 v4.1.0-beta5 release (#1359) * update version for next development cycle * Feature 1216 usecase smap (#1361) * Adding a conf file for SMAP * Adding a directory to host the read file * Removing temp file * Updated the valid dates to match Todd's code * Adding documentation for SMAP case * Updates the valis dates to match Todd's code * Removing a tmp file * Typo in file name * Updating the input RTOFS to have the init time instead of the valid time ii the file name * updated file paths, tesing * updated use case descriptions, rearranged use case group testing * put new use case into its own group so that the diff logic can evaluate marine_and_cryosphere:3. The truth data for 3-4 does not exist yet so the diff fails. Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * updated marine_and_cryo grouping * Feature 1230 stratosphere metrics (#1354) * Initial checkin for Meridial Mean use case * Changed the name and directories * Added some documentation * Fixed config variables * Issue 1230 Stratospheric metrics (zonal/meridional use case) put INPUT_BASE in the user_env_vars so the user doesn't need to set the INPUT_BASE environment in the current working shell * Issue #1230 remove import of metplotpy-this isn't called * Issue #1230_stratosphere_metric Add use case to the list of use cases to be run for testing * Issue 1230 Remove the INPUT_BASE from the user_env_vars section * Issue #1230 redundant files * Issue #1230 redundant files * Issue #1230 redundant files * Issue #120 replace INPUT_BASE with INPUT_FILE_NAME * Issue #1230 remove entry for INPUT_FILE_NAME, this goes in the system.conf * Update all_use_cases.txt Copy and pasted #11 from s2s use case for the Stratosphere use case but forgot to update the index to 12. * Update use_case_groups.json added use case #12 from s2s to test stratosphere use case * Issue #1230 put INPUT_FILE_NAME back under the user_env_vars * Issue #1230 forgot to include the filename * Issue #1230 clean up config file, remove uneccessary comments, group related entries in the user_env_vars * Issue #1230 remove extraneous and incorrect path to the input_filename setting * Removed pingouin dependency * Issue #1230 another cut and paste error fixed for the Stratosphere use case * Issue #1230 type in name, obs_Only should be obsOnly * Issue #1230 added the metdatadb to the env, code imports metdatadb and may require some of these dependencies * issue #1230 removed pingouin dependency from comment to reduce any confusion * issue #1230 turn off the test for the Stratosphere metrics use case * Added use case image Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: George McCabe <[email protected]> Co-authored-by: johnhg <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: lisagoodrich <[email protected]> Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: j-opatz <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: j-opatz <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: Christina Kalb <[email protected]>

* Feature 1091 extent cycloneplotter (#1218) * Github #1091 add support to define plot's extent * Github #1091 Add support for defining plot's extent * Github #1091 add support for defining plot extent * GitHub #1091 Provide support to define plotting a bounding box of interest and replace deprecated cartopy attributes. * Github Issue #1091 Add configuration support to define the bounding box defining the area of interest to plot. * Github Issue #1091 provide support to define bounding box defining the area of interest to plot. * Github Issue #1091 Removed unused shapely imports * GItHub Issue #1091 Removed duplicate CYCLONE_PLOTTER_GLOBAL_PLOT * Github #1091 Include config settings needed to indicate the region of interest to be plotted. * Github Issue #1091 Keep only the North Hemisphere lon and lats in the config file to serve as example for plotting a specific area of the map. * convert file back to unix format via dos2unix and added change that was lost in merge * return None from function instead of exiting so that METplus clean up functionality will still run * fixed indentation * fixed indentation - for loop should not be nested inside other for loop * fixed check for failure in retrieve_data function * feature 1223 error if file not found (#1238) * feature 1252 allow dictionary value for time_summary.width (#1253) * feature 1213 obs_quality_inc/exc (#1260) * Feature 1203 ioda2nc (#1262) * Add default title for the new use case issue template. * Feature 1019 harmonic preprocessing (#1272) Co-authored-by: George McCabe <[email protected]> * Feature 1266 gen ens prod missing ensembles (#1275) * Feature 1049 statistics list (#1271) * first attempt at a table #1049 * 2nd attempt, smaller, different formatting table #1049 * trying to get rid of git error message by adding this to the index #1049 * alignment #1049 * trying to text wrap in a table #1049 * attempting to create text wrapping in tables #1049 * removing role, trying to wrap text #1049 * removing html break, trying to wrap text #1049 * adding code to wrap text #1049 * removing code to wrap text #1049 * fixing naming typo #1049 * hitting returns in table #1049 * removing typo file, removing line breaks #1049 * attempting a simple table for text wrapping #1049 * attempting to bold header row in simple table #1049 * simple table line break attempt #1049 * simple table line break attempt #2 #1049 * simple table line break attempt #3 #1049 * trying another | #1049 * 2 trying another | #1049 * 3 trying another | #1049 * trying a blank line #1049 * trying reformating grid #1049 * 2 trying reformating grid #1049 * 3 trying reformating grid #1049 * 4 trying reformating grid #1049 * adding in one more table row #1049 * adding more text #1049 * trying glossary format #1049 * 2 trying glossary format #1049 * 3 trying glossary format #1049 * 4 trying glossary format #1049 * adding glossary format 2D objects #1049 * fixing glossary format 2D objects #1049 * adding glossary items #1049 * fixing glossary format #1049 * hopefully all examples are working #1049 * changing reference to tool * removing 2D object examples * removing the unwrapped table * adding ACC into the example * fixing formatting #1049 * fixing formatting take 2 #1049 * fixing formatting take 3 #1049 * bolding and language change #1049 * fixing spacing #1049 * fixing spacing again #1049 * changing title #1049 * trying superscript #1049 * superscript glossary #1049 * removing quotes #1049 * fixing spacing #1049 * Tara has decided to go with the glossary format. Removing table example. #1049 * Adding 2D objects #1049 * 2D objects fix #1049 * 2D objects fix attempt 2 #1049 * 2D objects fix attempt 3 #1049 * 2D objects fix attempt 4 #1049 * 2D objects fix attempt 5 #1049 * 2D objects fix attempt 7 #1049 * 2D objects fix attempt 8 #1049 * ABR added #1049 * adding remaining AB items to list #1049 * adding remaining ACC_ items to list #1049 * ADLAND & AFSS entries #1049 * A entries #1049 * fixing formatting #1049 * fixing formatting #2 #1049 * removing new entries to make it run #1049 * adding a couple back in #1049 * adding a couple more back in #1049 * tool key at the bottom #1049 * spacing changes #1049 * spacing changes another try #1049 * fixing spacing #1049 * fixing spacing attempt 2 #1049 * fixing spacing attempt 3 #1049 * through AREA items #1049 * AMODEL listed twice but it's not #1049 * AMODEL listed twice but it's not #2 #1049 * AMODEL added question marks so it will run #1049 * final A entries #1049 * naming the glossary statistics in hopes of not conflicting with original glossary #1049 * removing glossary statistics space #1049 * Testing multiple glossary names * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, removing code with attempts at mutiple glossaries * Per #1067, adding back three question marks due to duplicate term * paring down list, adding a table for review * problems with table * still trying to get it to publish * still trying to get a second line in the table * still trying to get a second line in the table 2 * testing different formatting * testing different formatting * trying to comment out the glossary inner workings * formatting again * line breaks in table #1049 * adding in some more for an example #1049 * adding new entries in through AREA #1049 * fixing line breaks #1049 * fixing line breaks #1049 #2 * fixing line breaks and warning messages #1049 * fixing line breaks #1049 * adding AREA_RATIO through ASPECT_DIFF * fixing typo #1049 * fixing typo #1049 * fixing typo #1049 take 2 * adding AXIS_ANG to BCMSE #1049 * adding spacing #1049 * adding spacing removing commas #1049 * removing comma #1049 * BOUNDARY_DIST thru BSS_SMPL #1049 * BOUNDARY_DIST splitting across 2 lines #1049 * cleaning up typos #1049 * calibration thru centriod_dist #1049 * centriod_lat thru centroid_y #1049 * fixing spacing #1049 * removing test glossary #1049 * climo_mean thru crtk_err #1049 * fixing crtk_err spacing #1049 * fixing spacing #1049 * CSI to CURVATURE_Y #1049 * CURVATURE_X & Y spacing #1049 * DEV_CAT to DURATION_DIFF #1049 * EC_VALUE to F #1049 * F_RATE TO FBS #1049 * Fixing spacing #1049 * fcst_clus thru fcst_conv_radius #1049 * removing CTOP_PRS #1049 * fixing the order of tools for FBAR and FBIAS #1049 * fixing spacing #1049 * adding grid-stat to all point-stat entries #1049 * adding fixing spacing #1049 * adding fixing spacing take 2 #1049 * adding fixing spacing take 3 #1049 * adding fixing spacing take 4 #1049 * adding fixing spacing take 5 #1049 * adding fixing spacing take 6 #1049 * adding fixing spacing take 7 #1049 * fixing spacing with a period take 7 #1049 * first attempt fcst_ #1049 * fixing typos #1049 * thru FOBAR #1049 * thru end of F #1049 * g thru h #1049 * i thru intensity #s #1049 * fixing typos #1049 * capturing example for Julie #1049 * thru k #1049 * fixing typos #1049 * thru L #1049 * thru MG #1049 * thru N_ENS #1049 * thru all N #1049 * fixing FBIAS alignment #1049 * fixing ME and MSE alignment #1049 * fixing ME alignment take 2 #1049 * thru OBS_E #1049 * fixing alignment #1049 * fixing alignment n_thresh #1049 * thru OBS_thresh #1049 * thru O #1049 * fixing OOBAR formating #1049 * thru PR_CORR #1049 * commented lines out with line total info #1049 * commented lines out with line total info take 2 #1049 * thru R #1049 * thru SPEED #1049 * thru S #1049 * thru T #1049 * thru U #1049 * thru V #1049 * thru V #1049 * thru Z #1049 * Update statistics_list.rst Tara is testing editing in UI * Update statistics_list.rst Updates through the A's * Update statistics_list.rst Cleaning up the A's * Update statistics_list.rst Standardizing MODE and MTD entries * Update statistics_list.rst Updating B's and C's * Update statistics_list.rst Testing adding TC-Stat and TCST to an entry * Update statistics_list.rst Clean up of a few A-Cs and then update of Ds * Update statistics_list.rst A few clean-ups and Es * Update statistics_list.rst Halfway through Fs... * Update statistics_list.rst A little clean up and the rest of Fs * Update statistics_list.rst G, H, I, Ks * Update statistics_list.rst L, M, Ns * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst Rs * Update statistics_list.rst S and Ts * Update statistics_list.rst The rest of the list * Update statistics_list.rst Removed Attr from Stat Type thru E * Update statistics_list.rst Remove Attr from Statistics Type through Gs * Update statistics_list.rst Remove Attr from Statistic Type through Rs * Update statistics_list.rst Remove Attr from Stat Type to the end * Update statistics_list.rst Cleaned up some Line Type typos * Update statistics_list.rst Still more Attr cleanup Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> * Feature 1263 v4.1.0 beta4 (#1277) * update version to note development towards beta5 * Feature 934 release stage doc (#1235) * Per #934 add stages of the METplus release cycle. * Per #934, adding link to descriptions of the release cycle in the User's Guide. * Per #934, made corrections * Per #934, changed Beta and Release Candidate (rc) from bold to subsubsections. * Update index.rst Co-authored-by: Julie Prestopnik <[email protected]> * Feature 344 met util refactor (#1292) * removed deprecated sections from config examples * minor change to METplus release guide to add a link to the PDF of the User's Guide instead of downloading it and attaching it to the release * Feature 1285 extract tiles mtd times (#1315) * Feature 896 more met config (#1322) * removed incorrect search keyword * added workflow_dispatch event so workflow can be triggered by an external repository such as MET to test to ensure that changes from that repo will break anything in METplus * added another input argument for workflow_dispatch event * added job with name that shows the event name or the repository name if triggered by an external repository such as MET * GHA: add username that triggered external event to event info job name * added required input argument for external trigger that contains the commit hash of the push event that triggered in the other repo * change event info to show commit hash instead of username that merged the PR * changed input names to match names of event in repository that triggered workflow * feature 1320 OMP_NUM_THREADS (#1338) * Feature 1183 memory documentation (#1340) Co-authored-by: George McCabe <[email protected]> * add email address of user who triggered push event to job name * Feature 1166 series analysis field info (#1353) * Feature 1116 usecase smos (#1348) Co-authored-by: Mrinal Biswas <[email protected]> * turn off new use case from every push * feature 1236 Control Members in EnsembleStat and GenEnsProd (#1357) * added optional argument to change the directory to untar new input data into so the same Dockerfile can be used to add data for other METplus components such as MET * feature 1358 v4.1.0-beta5 release (#1359) * update version for next development cycle * Feature 1216 usecase smap (#1361) * Adding a conf file for SMAP * Adding a directory to host the read file * Removing temp file * Updated the valid dates to match Todd's code * Adding documentation for SMAP case * Updates the valis dates to match Todd's code * Removing a tmp file * Typo in file name * Updating the input RTOFS to have the init time instead of the valid time ii the file name * updated file paths, tesing * updated use case descriptions, rearranged use case group testing * put new use case into its own group so that the diff logic can evaluate marine_and_cryosphere:3. The truth data for 3-4 does not exist yet so the diff fails. Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * updated marine_and_cryo grouping * Feature 1230 stratosphere metrics (#1354) * Initial checkin for Meridial Mean use case * Changed the name and directories * Added some documentation * Fixed config variables * Issue 1230 Stratospheric metrics (zonal/meridional use case) put INPUT_BASE in the user_env_vars so the user doesn't need to set the INPUT_BASE environment in the current working shell * Issue #1230 remove import of metplotpy-this isn't called * Issue #1230_stratosphere_metric Add use case to the list of use cases to be run for testing * Issue 1230 Remove the INPUT_BASE from the user_env_vars section * Issue #1230 redundant files * Issue #1230 redundant files * Issue #1230 redundant files * Issue #120 replace INPUT_BASE with INPUT_FILE_NAME * Issue #1230 remove entry for INPUT_FILE_NAME, this goes in the system.conf * Update all_use_cases.txt Copy and pasted #11 from s2s use case for the Stratosphere use case but forgot to update the index to 12. * Update use_case_groups.json added use case #12 from s2s to test stratosphere use case * Issue #1230 put INPUT_FILE_NAME back under the user_env_vars * Issue #1230 forgot to include the filename * Issue #1230 clean up config file, remove uneccessary comments, group related entries in the user_env_vars * Issue #1230 remove extraneous and incorrect path to the input_filename setting * Removed pingouin dependency * Issue #1230 another cut and paste error fixed for the Stratosphere use case * Issue #1230 type in name, obs_Only should be obsOnly * Issue #1230 added the metdatadb to the env, code imports metdatadb and may require some of these dependencies * issue #1230 removed pingouin dependency from comment to reduce any confusion * issue #1230 turn off the test for the Stratosphere metrics use case * Added use case image Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: Christina Kalb <[email protected]> * Corrected spelling of occurrence in two places * Fixed misspelling of occurrence * Found and fixed two more misspellings of occurence * Changed version specific information to by X.Y.Z moved text indicating to click save to the bottom of the list. * Feature 1374 python packages (#1378) * Added documentation about updating spreadsheet of Python requirements * Changed references to master_metplus.py to run_metplus.py * Fixed formatting of section with run_metplus.py commands * Updated text for updating the spreadsheet * Removed a newly added section that wasn't needed Co-authored-by: Julie Prestopnik <[email protected]> * feature 1368 PCPCombine use zero accum (#1381) * feature 1369 grid_weight_flag in EnsembleStat (#1379) * Per #1356, change how wrappers create instances of other wrappers to ensure that config settings for the created instance do not change values in METplusConfig used by the rest of the wrappers, ci-run-all-diff * Per #1356, remove config_overrides functionality in favor of using instances -- this prevents the issue where overrides for a given tool affect the global config settings. it also forces the configs to override for an instance to be put into another config section so that it will be available in the final conf, ci-run-all-diff * Per #1356, update documentation to reflect changes * Per #1356, fixed unit test to no longer use deprecated approach to overriding config variables * feature 1247 climatology settings (#1385) * feature 1356 isolate config (#1386) * Feature 675 Continuous Integration Documentation (#1409) Co-authored-by: johnhg <[email protected]> * feature 1382 Filename templates in MODEL (#1414) * Feature 1289 explicit file list (#1387) * Per PR #1387 review, fix typo * Bugfix 1421 EnsembleStat use fcst fields in ens dictionary if ens is unset (#1422) * fixed missing path change for docker file path * removed variables that are not used * Feature 1408 use case ptype (#1425) * adding imagery, config file, updating internal tests * updated lists, auto testing * corrected old reference * Update use_case_groups.json Co-authored-by: bikegeek <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: George McCabe <[email protected]> Co-authored-by: johnhg <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: lisagoodrich <[email protected]> Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: Christina Kalb <[email protected]>

* Feature 1091 extent cycloneplotter (#1218) * Github #1091 add support to define plot's extent * Github #1091 Add support for defining plot's extent * Github #1091 add support for defining plot extent * GitHub #1091 Provide support to define plotting a bounding box of interest and replace deprecated cartopy attributes. * Github Issue #1091 Add configuration support to define the bounding box defining the area of interest to plot. * Github Issue #1091 provide support to define bounding box defining the area of interest to plot. * Github Issue #1091 Removed unused shapely imports * GItHub Issue #1091 Removed duplicate CYCLONE_PLOTTER_GLOBAL_PLOT * Github #1091 Include config settings needed to indicate the region of interest to be plotted. * Github Issue #1091 Keep only the North Hemisphere lon and lats in the config file to serve as example for plotting a specific area of the map. * convert file back to unix format via dos2unix and added change that was lost in merge * return None from function instead of exiting so that METplus clean up functionality will still run * fixed indentation * fixed indentation - for loop should not be nested inside other for loop * fixed check for failure in retrieve_data function * feature 1223 error if file not found (#1238) * feature 1252 allow dictionary value for time_summary.width (#1253) * feature 1213 obs_quality_inc/exc (#1260) * Feature 1203 ioda2nc (#1262) * Add default title for the new use case issue template. * Feature 1019 harmonic preprocessing (#1272) Co-authored-by: George McCabe <[email protected]> * Feature 1266 gen ens prod missing ensembles (#1275) * Feature 1049 statistics list (#1271) * first attempt at a table #1049 * 2nd attempt, smaller, different formatting table #1049 * trying to get rid of git error message by adding this to the index #1049 * alignment #1049 * trying to text wrap in a table #1049 * attempting to create text wrapping in tables #1049 * removing role, trying to wrap text #1049 * removing html break, trying to wrap text #1049 * adding code to wrap text #1049 * removing code to wrap text #1049 * fixing naming typo #1049 * hitting returns in table #1049 * removing typo file, removing line breaks #1049 * attempting a simple table for text wrapping #1049 * attempting to bold header row in simple table #1049 * simple table line break attempt #1049 * simple table line break attempt #2 #1049 * simple table line break attempt #3 #1049 * trying another | #1049 * 2 trying another | #1049 * 3 trying another | #1049 * trying a blank line #1049 * trying reformating grid #1049 * 2 trying reformating grid #1049 * 3 trying reformating grid #1049 * 4 trying reformating grid #1049 * adding in one more table row #1049 * adding more text #1049 * trying glossary format #1049 * 2 trying glossary format #1049 * 3 trying glossary format #1049 * 4 trying glossary format #1049 * adding glossary format 2D objects #1049 * fixing glossary format 2D objects #1049 * adding glossary items #1049 * fixing glossary format #1049 * hopefully all examples are working #1049 * changing reference to tool * removing 2D object examples * removing the unwrapped table * adding ACC into the example * fixing formatting #1049 * fixing formatting take 2 #1049 * fixing formatting take 3 #1049 * bolding and language change #1049 * fixing spacing #1049 * fixing spacing again #1049 * changing title #1049 * trying superscript #1049 * superscript glossary #1049 * removing quotes #1049 * fixing spacing #1049 * Tara has decided to go with the glossary format. Removing table example. #1049 * Adding 2D objects #1049 * 2D objects fix #1049 * 2D objects fix attempt 2 #1049 * 2D objects fix attempt 3 #1049 * 2D objects fix attempt 4 #1049 * 2D objects fix attempt 5 #1049 * 2D objects fix attempt 7 #1049 * 2D objects fix attempt 8 #1049 * ABR added #1049 * adding remaining AB items to list #1049 * adding remaining ACC_ items to list #1049 * ADLAND & AFSS entries #1049 * A entries #1049 * fixing formatting #1049 * fixing formatting #2 #1049 * removing new entries to make it run #1049 * adding a couple back in #1049 * adding a couple more back in #1049 * tool key at the bottom #1049 * spacing changes #1049 * spacing changes another try #1049 * fixing spacing #1049 * fixing spacing attempt 2 #1049 * fixing spacing attempt 3 #1049 * through AREA items #1049 * AMODEL listed twice but it's not #1049 * AMODEL listed twice but it's not #2 #1049 * AMODEL added question marks so it will run #1049 * final A entries #1049 * naming the glossary statistics in hopes of not conflicting with original glossary #1049 * removing glossary statistics space #1049 * Testing multiple glossary names * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, removing code with attempts at mutiple glossaries * Per #1067, adding back three question marks due to duplicate term * paring down list, adding a table for review * problems with table * still trying to get it to publish * still trying to get a second line in the table * still trying to get a second line in the table 2 * testing different formatting * testing different formatting * trying to comment out the glossary inner workings * formatting again * line breaks in table #1049 * adding in some more for an example #1049 * adding new entries in through AREA #1049 * fixing line breaks #1049 * fixing line breaks #1049 #2 * fixing line breaks and warning messages #1049 * fixing line breaks #1049 * adding AREA_RATIO through ASPECT_DIFF * fixing typo #1049 * fixing typo #1049 * fixing typo #1049 take 2 * adding AXIS_ANG to BCMSE #1049 * adding spacing #1049 * adding spacing removing commas #1049 * removing comma #1049 * BOUNDARY_DIST thru BSS_SMPL #1049 * BOUNDARY_DIST splitting across 2 lines #1049 * cleaning up typos #1049 * calibration thru centriod_dist #1049 * centriod_lat thru centroid_y #1049 * fixing spacing #1049 * removing test glossary #1049 * climo_mean thru crtk_err #1049 * fixing crtk_err spacing #1049 * fixing spacing #1049 * CSI to CURVATURE_Y #1049 * CURVATURE_X & Y spacing #1049 * DEV_CAT to DURATION_DIFF #1049 * EC_VALUE to F #1049 * F_RATE TO FBS #1049 * Fixing spacing #1049 * fcst_clus thru fcst_conv_radius #1049 * removing CTOP_PRS #1049 * fixing the order of tools for FBAR and FBIAS #1049 * fixing spacing #1049 * adding grid-stat to all point-stat entries #1049 * adding fixing spacing #1049 * adding fixing spacing take 2 #1049 * adding fixing spacing take 3 #1049 * adding fixing spacing take 4 #1049 * adding fixing spacing take 5 #1049 * adding fixing spacing take 6 #1049 * adding fixing spacing take 7 #1049 * fixing spacing with a period take 7 #1049 * first attempt fcst_ #1049 * fixing typos #1049 * thru FOBAR #1049 * thru end of F #1049 * g thru h #1049 * i thru intensity #s #1049 * fixing typos #1049 * capturing example for Julie #1049 * thru k #1049 * fixing typos #1049 * thru L #1049 * thru MG #1049 * thru N_ENS #1049 * thru all N #1049 * fixing FBIAS alignment #1049 * fixing ME and MSE alignment #1049 * fixing ME alignment take 2 #1049 * thru OBS_E #1049 * fixing alignment #1049 * fixing alignment n_thresh #1049 * thru OBS_thresh #1049 * thru O #1049 * fixing OOBAR formating #1049 * thru PR_CORR #1049 * commented lines out with line total info #1049 * commented lines out with line total info take 2 #1049 * thru R #1049 * thru SPEED #1049 * thru S #1049 * thru T #1049 * thru U #1049 * thru V #1049 * thru V #1049 * thru Z #1049 * Update statistics_list.rst Tara is testing editing in UI * Update statistics_list.rst Updates through the A's * Update statistics_list.rst Cleaning up the A's * Update statistics_list.rst Standardizing MODE and MTD entries * Update statistics_list.rst Updating B's and C's * Update statistics_list.rst Testing adding TC-Stat and TCST to an entry * Update statistics_list.rst Clean up of a few A-Cs and then update of Ds * Update statistics_list.rst A few clean-ups and Es * Update statistics_list.rst Halfway through Fs... * Update statistics_list.rst A little clean up and the rest of Fs * Update statistics_list.rst G, H, I, Ks * Update statistics_list.rst L, M, Ns * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst Rs * Update statistics_list.rst S and Ts * Update statistics_list.rst The rest of the list * Update statistics_list.rst Removed Attr from Stat Type thru E * Update statistics_list.rst Remove Attr from Statistics Type through Gs * Update statistics_list.rst Remove Attr from Statistic Type through Rs * Update statistics_list.rst Remove Attr from Stat Type to the end * Update statistics_list.rst Cleaned up some Line Type typos * Update statistics_list.rst Still more Attr cleanup Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> * Feature 1263 v4.1.0 beta4 (#1277) * update version to note development towards beta5 * Feature 934 release stage doc (#1235) * Per #934 add stages of the METplus release cycle. * Per #934, adding link to descriptions of the release cycle in the User's Guide. * Per #934, made corrections * Per #934, changed Beta and Release Candidate (rc) from bold to subsubsections. * Update index.rst Co-authored-by: Julie Prestopnik <[email protected]> * Feature 344 met util refactor (#1292) * removed deprecated sections from config examples * minor change to METplus release guide to add a link to the PDF of the User's Guide instead of downloading it and attaching it to the release * Feature 1285 extract tiles mtd times (#1315) * Feature 896 more met config (#1322) * removed incorrect search keyword * added workflow_dispatch event so workflow can be triggered by an external repository such as MET to test to ensure that changes from that repo will break anything in METplus * added another input argument for workflow_dispatch event * added job with name that shows the event name or the repository name if triggered by an external repository such as MET * GHA: add username that triggered external event to event info job name * added required input argument for external trigger that contains the commit hash of the push event that triggered in the other repo * change event info to show commit hash instead of username that merged the PR * changed input names to match names of event in repository that triggered workflow * feature 1320 OMP_NUM_THREADS (#1338) * Feature 1183 memory documentation (#1340) Co-authored-by: George McCabe <[email protected]> * add email address of user who triggered push event to job name * Feature 1166 series analysis field info (#1353) * Feature 1116 usecase smos (#1348) Co-authored-by: Mrinal Biswas <[email protected]> * turn off new use case from every push * feature 1236 Control Members in EnsembleStat and GenEnsProd (#1357) * added optional argument to change the directory to untar new input data into so the same Dockerfile can be used to add data for other METplus components such as MET * feature 1358 v4.1.0-beta5 release (#1359) * update version for next development cycle * Feature 1216 usecase smap (#1361) * Adding a conf file for SMAP * Adding a directory to host the read file * Removing temp file * Updated the valid dates to match Todd's code * Adding documentation for SMAP case * Updates the valis dates to match Todd's code * Removing a tmp file * Typo in file name * Updating the input RTOFS to have the init time instead of the valid time ii the file name * updated file paths, tesing * updated use case descriptions, rearranged use case group testing * put new use case into its own group so that the diff logic can evaluate marine_and_cryosphere:3. The truth data for 3-4 does not exist yet so the diff fails. Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * updated marine_and_cryo grouping * Feature 1230 stratosphere metrics (#1354) * Initial checkin for Meridial Mean use case * Changed the name and directories * Added some documentation * Fixed config variables * Issue 1230 Stratospheric metrics (zonal/meridional use case) put INPUT_BASE in the user_env_vars so the user doesn't need to set the INPUT_BASE environment in the current working shell * Issue #1230 remove import of metplotpy-this isn't called * Issue #1230_stratosphere_metric Add use case to the list of use cases to be run for testing * Issue 1230 Remove the INPUT_BASE from the user_env_vars section * Issue #1230 redundant files * Issue #1230 redundant files * Issue #1230 redundant files * Issue #120 replace INPUT_BASE with INPUT_FILE_NAME * Issue #1230 remove entry for INPUT_FILE_NAME, this goes in the system.conf * Update all_use_cases.txt Copy and pasted #11 from s2s use case for the Stratosphere use case but forgot to update the index to 12. * Update use_case_groups.json added use case #12 from s2s to test stratosphere use case * Issue #1230 put INPUT_FILE_NAME back under the user_env_vars * Issue #1230 forgot to include the filename * Issue #1230 clean up config file, remove uneccessary comments, group related entries in the user_env_vars * Issue #1230 remove extraneous and incorrect path to the input_filename setting * Removed pingouin dependency * Issue #1230 another cut and paste error fixed for the Stratosphere use case * Issue #1230 type in name, obs_Only should be obsOnly * Issue #1230 added the metdatadb to the env, code imports metdatadb and may require some of these dependencies * issue #1230 removed pingouin dependency from comment to reduce any confusion * issue #1230 turn off the test for the Stratosphere metrics use case * Added use case image Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: Christina Kalb <[email protected]> * Corrected spelling of occurrence in two places * Fixed misspelling of occurrence * Found and fixed two more misspellings of occurence * Changed version specific information to by X.Y.Z moved text indicating to click save to the bottom of the list. * Feature 1374 python packages (#1378) * Added documentation about updating spreadsheet of Python requirements * Changed references to master_metplus.py to run_metplus.py * Fixed formatting of section with run_metplus.py commands * Updated text for updating the spreadsheet * Removed a newly added section that wasn't needed Co-authored-by: Julie Prestopnik <[email protected]> * feature 1368 PCPCombine use zero accum (#1381) * feature 1369 grid_weight_flag in EnsembleStat (#1379) * Per #1356, change how wrappers create instances of other wrappers to ensure that config settings for the created instance do not change values in METplusConfig used by the rest of the wrappers, ci-run-all-diff * Per #1356, remove config_overrides functionality in favor of using instances -- this prevents the issue where overrides for a given tool affect the global config settings. it also forces the configs to override for an instance to be put into another config section so that it will be available in the final conf, ci-run-all-diff * Per #1356, update documentation to reflect changes * Per #1356, fixed unit test to no longer use deprecated approach to overriding config variables * feature 1247 climatology settings (#1385) * feature 1356 isolate config (#1386) * Feature 675 Continuous Integration Documentation (#1409) Co-authored-by: johnhg <[email protected]> * feature 1382 Filename templates in MODEL (#1414) * Feature 1289 explicit file list (#1387) * Per PR #1387 review, fix typo * Bugfix 1421 EnsembleStat use fcst fields in ens dictionary if ens is unset (#1422) * fixed missing path change for docker file path * removed variables that are not used * Feature 1408 use case ptype (#1425) * adding imagery, config file, updating internal tests * updated lists, auto testing * corrected old reference * Update use_case_groups.json * only trigger testing workflow for pull requests that are going into develop or main_* branches * Feature 1371 blocking wr (#1426) * turn off use cases for push events * Feature 1392 climo_cdf.direct_prob and SeriesAnalysis (#1396) * Changed event triggering rules for documentation workflow so that it will run for pull requests even if no files in the docs directory have changed. A recent pull request passed but develop failed after it was approved because files that were referenced in the use case documentation files had been deleted as part of the PR. * feature 1273 TCGen -edeck and -shape (#1424) * Feature 966 Fix mask.poly logic (#1430) * Feature 1030 total seconds documentation (#1428) Co-authored-by: George McCabe <[email protected]> * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * get branch name in job controls job * replace logic to update input data volumes with call to dtcenter/metplus-action-data-update action * feature 1431 remove obs_prepbufr_map (#1433) * Per #675, added images and updated content * added statsmodel package to metplotpy docker conda environment * Added fcst_file_type to two wrappers (#1437) Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: George McCabe <[email protected]> * added release notes for tasks that have already been completed for beta6 * updated env var for METplotpy base to match changes made to the METplotpy repository * Updated METplotpy base env var to preemptively match changes from PR dtcenter/METplotpy#197. Once that PR is merged into develop, a test workflow will run that should fix this issue. * removed commented code that is no longer used * update the instructions for updating test input data to ensure volume_mount_directories file is updated in the upcoming version directory, i.e. v4.1, so that the Docker data volumes from the release will be created properly * removed duplicate script * updated script to copy volume_mount_directories file into new METplus version directory * added logic to skip files that are temporary copies of tar files that should not be added to new version directory * updated release guide instructions for creating METplus data directory for next development cycle * Feature 911 tc trackint (#1439) Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * added new use cases that are currently being reviewed to release notes * updated event info in contributor's guide * Feature 626 fix hovmoeller documentation (#1444) * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py replace hovmoeller_diagram.conf with UserScript_obsPrecip_obsOnly_Hovmoeller.conf * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py Removed a typo: UserScript This uses data from... Now: This uses data from... * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py * Modified the PR template to add review of the source issue metadata ci-skip-all * feature_1403_python_components_table (#1446) * creating new METplus Components Python Requirements * adding a return to test * testing use case links * testing use case links #2 * testing use case links #3 * testing use case links #4 * Filling in some use cases * fixing indentation #1403 * fixing indentation #1403 take 2 * fixing indentation #1403 take 3 * cleaning up anything after # in the links #1403 * entries thru D #1403 * entries thru H #1403 * fixing indents #1403 * fixing indents #1403 take 2 * fixing web name #1403 * entries thru L #1403 * entry matplotlib #1403 * entry matplotlib #1403 marking duplicate names * entry metcalcpy #1403 * entry metcalcpy fix indents #1403 * entry matplotlib #1403 fixing duplicate names * entry metplotpy #1403 * entry metplotpy #1403 marking duplicate names * entry metplotpy #1403 fixing * entry thru ps #1403 * double names #1403 * character fix #1403 * adding in missing links #1403 * thru pyproj #1403 * problem with line 353 #1403 * thru pyproj #1403 typos * more duplicate names #1403 * line break #1403 * loose ends #1403 * adding >= to some of the versions #1403 * adding name to the Use Cases to avoid duplicate naming issues TEST #1403 * adding name to the Use Cases to avoid duplicate naming issues thru pyproj #1403 * adding thru pyresample #1403 * fixing pyresample typo #1403 * adding thru scikit-image #1403 * fixing line breaks#1403 * web links thru scikit-learn #1403 * web links thru scipy #1403 * web links thru sklearn #1403 * Duplicate web names. temp. fix #1403 * Duplicate web names fixed #1403 * Duplicate web names fixing #3 #1403 * Duplicate web names fixing #4 #1403 * Duplicate web names fixing #5 #1403 * table end #1403 * table loose ends #1403 * Removed unnecessary comma * table loose ends after pull request #1403 * Minor modifications to METplus Component column * Changed description of pyproj * Added link for last referenced use case * Attempting to fix doc warning Co-authored-by: jprestop <[email protected]> * Feature 1318 usecase aviso (#1440) * Correct small typo. * Update pull_request_template.md * Added files for ReadTheDocs (#864) * Updated documentation links for ReadtheDocs * Update pull_request_template.md Added entry for completion date for pull request review. * Adding python embedding script to read AVISO, RTOFS, HYCOM and OSTIA data * Updated the script - working but only RMSE values match * Adding documentation for the AVISO case * Adding a conf file for the AVISO case * Updates to the codel to include model3, obs, climo3 but also has many extra print statements, netcdf output files which needs to be cleaned * Cleaning up the conf file and adding SAL1L2 to get the model_climo sdev and obs_climo sdev * Updating the file to fix the standard names and add the model3, climo3 and obs3 * Updating the file to fix the standard names and add the model3, climo3 and obs3 * Adding an image to the use case. Image Courtesy: EMC/NOAA website * added to use case list * updated case description * removed merge issues * caught second docs issue * removed other doc merge issues, turned off auto test * more merge comments sneaking in Co-authored-by: johnhg <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: George McCabe <[email protected]> Co-authored-by: johnhg <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: lisagoodrich <[email protected]> Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]>

* Feature 1091 extent cycloneplotter (#1218) * Github #1091 add support to define plot's extent * Github #1091 Add support for defining plot's extent * Github #1091 add support for defining plot extent * GitHub #1091 Provide support to define plotting a bounding box of interest and replace deprecated cartopy attributes. * Github Issue #1091 Add configuration support to define the bounding box defining the area of interest to plot. * Github Issue #1091 provide support to define bounding box defining the area of interest to plot. * Github Issue #1091 Removed unused shapely imports * GItHub Issue #1091 Removed duplicate CYCLONE_PLOTTER_GLOBAL_PLOT * Github #1091 Include config settings needed to indicate the region of interest to be plotted. * Github Issue #1091 Keep only the North Hemisphere lon and lats in the config file to serve as example for plotting a specific area of the map. * convert file back to unix format via dos2unix and added change that was lost in merge * return None from function instead of exiting so that METplus clean up functionality will still run * fixed indentation * fixed indentation - for loop should not be nested inside other for loop * fixed check for failure in retrieve_data function * feature 1223 error if file not found (#1238) * feature 1252 allow dictionary value for time_summary.width (#1253) * feature 1213 obs_quality_inc/exc (#1260) * Feature 1203 ioda2nc (#1262) * Add default title for the new use case issue template. * Feature 1019 harmonic preprocessing (#1272) Co-authored-by: George McCabe <[email protected]> * Feature 1266 gen ens prod missing ensembles (#1275) * Feature 1049 statistics list (#1271) * first attempt at a table #1049 * 2nd attempt, smaller, different formatting table #1049 * trying to get rid of git error message by adding this to the index #1049 * alignment #1049 * trying to text wrap in a table #1049 * attempting to create text wrapping in tables #1049 * removing role, trying to wrap text #1049 * removing html break, trying to wrap text #1049 * adding code to wrap text #1049 * removing code to wrap text #1049 * fixing naming typo #1049 * hitting returns in table #1049 * removing typo file, removing line breaks #1049 * attempting a simple table for text wrapping #1049 * attempting to bold header row in simple table #1049 * simple table line break attempt #1049 * simple table line break attempt #2 #1049 * simple table line break attempt #3 #1049 * trying another | #1049 * 2 trying another | #1049 * 3 trying another | #1049 * trying a blank line #1049 * trying reformating grid #1049 * 2 trying reformating grid #1049 * 3 trying reformating grid #1049 * 4 trying reformating grid #1049 * adding in one more table row #1049 * adding more text #1049 * trying glossary format #1049 * 2 trying glossary format #1049 * 3 trying glossary format #1049 * 4 trying glossary format #1049 * adding glossary format 2D objects #1049 * fixing glossary format 2D objects #1049 * adding glossary items #1049 * fixing glossary format #1049 * hopefully all examples are working #1049 * changing reference to tool * removing 2D object examples * removing the unwrapped table * adding ACC into the example * fixing formatting #1049 * fixing formatting take 2 #1049 * fixing formatting take 3 #1049 * bolding and language change #1049 * fixing spacing #1049 * fixing spacing again #1049 * changing title #1049 * trying superscript #1049 * superscript glossary #1049 * removing quotes #1049 * fixing spacing #1049 * Tara has decided to go with the glossary format. Removing table example. #1049 * Adding 2D objects #1049 * 2D objects fix #1049 * 2D objects fix attempt 2 #1049 * 2D objects fix attempt 3 #1049 * 2D objects fix attempt 4 #1049 * 2D objects fix attempt 5 #1049 * 2D objects fix attempt 7 #1049 * 2D objects fix attempt 8 #1049 * ABR added #1049 * adding remaining AB items to list #1049 * adding remaining ACC_ items to list #1049 * ADLAND & AFSS entries #1049 * A entries #1049 * fixing formatting #1049 * fixing formatting #2 #1049 * removing new entries to make it run #1049 * adding a couple back in #1049 * adding a couple more back in #1049 * tool key at the bottom #1049 * spacing changes #1049 * spacing changes another try #1049 * fixing spacing #1049 * fixing spacing attempt 2 #1049 * fixing spacing attempt 3 #1049 * through AREA items #1049 * AMODEL listed twice but it's not #1049 * AMODEL listed twice but it's not #2 #1049 * AMODEL added question marks so it will run #1049 * final A entries #1049 * naming the glossary statistics in hopes of not conflicting with original glossary #1049 * removing glossary statistics space #1049 * Testing multiple glossary names * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, removing code with attempts at mutiple glossaries * Per #1067, adding back three question marks due to duplicate term * paring down list, adding a table for review * problems with table * still trying to get it to publish * still trying to get a second line in the table * still trying to get a second line in the table 2 * testing different formatting * testing different formatting * trying to comment out the glossary inner workings * formatting again * line breaks in table #1049 * adding in some more for an example #1049 * adding new entries in through AREA #1049 * fixing line breaks #1049 * fixing line breaks #1049 #2 * fixing line breaks and warning messages #1049 * fixing line breaks #1049 * adding AREA_RATIO through ASPECT_DIFF * fixing typo #1049 * fixing typo #1049 * fixing typo #1049 take 2 * adding AXIS_ANG to BCMSE #1049 * adding spacing #1049 * adding spacing removing commas #1049 * removing comma #1049 * BOUNDARY_DIST thru BSS_SMPL #1049 * BOUNDARY_DIST splitting across 2 lines #1049 * cleaning up typos #1049 * calibration thru centriod_dist #1049 * centriod_lat thru centroid_y #1049 * fixing spacing #1049 * removing test glossary #1049 * climo_mean thru crtk_err #1049 * fixing crtk_err spacing #1049 * fixing spacing #1049 * CSI to CURVATURE_Y #1049 * CURVATURE_X & Y spacing #1049 * DEV_CAT to DURATION_DIFF #1049 * EC_VALUE to F #1049 * F_RATE TO FBS #1049 * Fixing spacing #1049 * fcst_clus thru fcst_conv_radius #1049 * removing CTOP_PRS #1049 * fixing the order of tools for FBAR and FBIAS #1049 * fixing spacing #1049 * adding grid-stat to all point-stat entries #1049 * adding fixing spacing #1049 * adding fixing spacing take 2 #1049 * adding fixing spacing take 3 #1049 * adding fixing spacing take 4 #1049 * adding fixing spacing take 5 #1049 * adding fixing spacing take 6 #1049 * adding fixing spacing take 7 #1049 * fixing spacing with a period take 7 #1049 * first attempt fcst_ #1049 * fixing typos #1049 * thru FOBAR #1049 * thru end of F #1049 * g thru h #1049 * i thru intensity #s #1049 * fixing typos #1049 * capturing example for Julie #1049 * thru k #1049 * fixing typos #1049 * thru L #1049 * thru MG #1049 * thru N_ENS #1049 * thru all N #1049 * fixing FBIAS alignment #1049 * fixing ME and MSE alignment #1049 * fixing ME alignment take 2 #1049 * thru OBS_E #1049 * fixing alignment #1049 * fixing alignment n_thresh #1049 * thru OBS_thresh #1049 * thru O #1049 * fixing OOBAR formating #1049 * thru PR_CORR #1049 * commented lines out with line total info #1049 * commented lines out with line total info take 2 #1049 * thru R #1049 * thru SPEED #1049 * thru S #1049 * thru T #1049 * thru U #1049 * thru V #1049 * thru V #1049 * thru Z #1049 * Update statistics_list.rst Tara is testing editing in UI * Update statistics_list.rst Updates through the A's * Update statistics_list.rst Cleaning up the A's * Update statistics_list.rst Standardizing MODE and MTD entries * Update statistics_list.rst Updating B's and C's * Update statistics_list.rst Testing adding TC-Stat and TCST to an entry * Update statistics_list.rst Clean up of a few A-Cs and then update of Ds * Update statistics_list.rst A few clean-ups and Es * Update statistics_list.rst Halfway through Fs... * Update statistics_list.rst A little clean up and the rest of Fs * Update statistics_list.rst G, H, I, Ks * Update statistics_list.rst L, M, Ns * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst Rs * Update statistics_list.rst S and Ts * Update statistics_list.rst The rest of the list * Update statistics_list.rst Removed Attr from Stat Type thru E * Update statistics_list.rst Remove Attr from Statistics Type through Gs * Update statistics_list.rst Remove Attr from Statistic Type through Rs * Update statistics_list.rst Remove Attr from Stat Type to the end * Update statistics_list.rst Cleaned up some Line Type typos * Update statistics_list.rst Still more Attr cleanup Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> * Feature 1263 v4.1.0 beta4 (#1277) * update version to note development towards beta5 * Feature 934 release stage doc (#1235) * Per #934 add stages of the METplus release cycle. * Per #934, adding link to descriptions of the release cycle in the User's Guide. * Per #934, made corrections * Per #934, changed Beta and Release Candidate (rc) from bold to subsubsections. * Update index.rst Co-authored-by: Julie Prestopnik <[email protected]> * Feature 344 met util refactor (#1292) * removed deprecated sections from config examples * minor change to METplus release guide to add a link to the PDF of the User's Guide instead of downloading it and attaching it to the release * Feature 1285 extract tiles mtd times (#1315) * Feature 896 more met config (#1322) * removed incorrect search keyword * added workflow_dispatch event so workflow can be triggered by an external repository such as MET to test to ensure that changes from that repo will break anything in METplus * added another input argument for workflow_dispatch event * added job with name that shows the event name or the repository name if triggered by an external repository such as MET * GHA: add username that triggered external event to event info job name * added required input argument for external trigger that contains the commit hash of the push event that triggered in the other repo * change event info to show commit hash instead of username that merged the PR * changed input names to match names of event in repository that triggered workflow * feature 1320 OMP_NUM_THREADS (#1338) * Feature 1183 memory documentation (#1340) Co-authored-by: George McCabe <[email protected]> * add email address of user who triggered push event to job name * Feature 1166 series analysis field info (#1353) * Feature 1116 usecase smos (#1348) Co-authored-by: Mrinal Biswas <[email protected]> * turn off new use case from every push * feature 1236 Control Members in EnsembleStat and GenEnsProd (#1357) * added optional argument to change the directory to untar new input data into so the same Dockerfile can be used to add data for other METplus components such as MET * feature 1358 v4.1.0-beta5 release (#1359) * update version for next development cycle * Feature 1216 usecase smap (#1361) * Adding a conf file for SMAP * Adding a directory to host the read file * Removing temp file * Updated the valid dates to match Todd's code * Adding documentation for SMAP case * Updates the valis dates to match Todd's code * Removing a tmp file * Typo in file name * Updating the input RTOFS to have the init time instead of the valid time ii the file name * updated file paths, tesing * updated use case descriptions, rearranged use case group testing * put new use case into its own group so that the diff logic can evaluate marine_and_cryosphere:3. The truth data for 3-4 does not exist yet so the diff fails. Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * updated marine_and_cryo grouping * Feature 1230 stratosphere metrics (#1354) * Initial checkin for Meridial Mean use case * Changed the name and directories * Added some documentation * Fixed config variables * Issue 1230 Stratospheric metrics (zonal/meridional use case) put INPUT_BASE in the user_env_vars so the user doesn't need to set the INPUT_BASE environment in the current working shell * Issue #1230 remove import of metplotpy-this isn't called * Issue #1230_stratosphere_metric Add use case to the list of use cases to be run for testing * Issue 1230 Remove the INPUT_BASE from the user_env_vars section * Issue #1230 redundant files * Issue #1230 redundant files * Issue #1230 redundant files * Issue #120 replace INPUT_BASE with INPUT_FILE_NAME * Issue #1230 remove entry for INPUT_FILE_NAME, this goes in the system.conf * Update all_use_cases.txt Copy and pasted #11 from s2s use case for the Stratosphere use case but forgot to update the index to 12. * Update use_case_groups.json added use case #12 from s2s to test stratosphere use case * Issue #1230 put INPUT_FILE_NAME back under the user_env_vars * Issue #1230 forgot to include the filename * Issue #1230 clean up config file, remove uneccessary comments, group related entries in the user_env_vars * Issue #1230 remove extraneous and incorrect path to the input_filename setting * Removed pingouin dependency * Issue #1230 another cut and paste error fixed for the Stratosphere use case * Issue #1230 type in name, obs_Only should be obsOnly * Issue #1230 added the metdatadb to the env, code imports metdatadb and may require some of these dependencies * issue #1230 removed pingouin dependency from comment to reduce any confusion * issue #1230 turn off the test for the Stratosphere metrics use case * Added use case image Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: Christina Kalb <[email protected]> * Corrected spelling of occurrence in two places * Fixed misspelling of occurrence * Found and fixed two more misspellings of occurence * Changed version specific information to by X.Y.Z moved text indicating to click save to the bottom of the list. * Feature 1374 python packages (#1378) * Added documentation about updating spreadsheet of Python requirements * Changed references to master_metplus.py to run_metplus.py * Fixed formatting of section with run_metplus.py commands * Updated text for updating the spreadsheet * Removed a newly added section that wasn't needed Co-authored-by: Julie Prestopnik <[email protected]> * feature 1368 PCPCombine use zero accum (#1381) * feature 1369 grid_weight_flag in EnsembleStat (#1379) * Per #1356, change how wrappers create instances of other wrappers to ensure that config settings for the created instance do not change values in METplusConfig used by the rest of the wrappers, ci-run-all-diff * Per #1356, remove config_overrides functionality in favor of using instances -- this prevents the issue where overrides for a given tool affect the global config settings. it also forces the configs to override for an instance to be put into another config section so that it will be available in the final conf, ci-run-all-diff * Per #1356, update documentation to reflect changes * Per #1356, fixed unit test to no longer use deprecated approach to overriding config variables * feature 1247 climatology settings (#1385) * feature 1356 isolate config (#1386) * Feature 675 Continuous Integration Documentation (#1409) Co-authored-by: johnhg <[email protected]> * feature 1382 Filename templates in MODEL (#1414) * Feature 1289 explicit file list (#1387) * Per PR #1387 review, fix typo * Bugfix 1421 EnsembleStat use fcst fields in ens dictionary if ens is unset (#1422) * fixed missing path change for docker file path * removed variables that are not used * Feature 1408 use case ptype (#1425) * adding imagery, config file, updating internal tests * updated lists, auto testing * corrected old reference * Update use_case_groups.json * only trigger testing workflow for pull requests that are going into develop or main_* branches * Feature 1371 blocking wr (#1426) * turn off use cases for push events * Feature 1392 climo_cdf.direct_prob and SeriesAnalysis (#1396) * Changed event triggering rules for documentation workflow so that it will run for pull requests even if no files in the docs directory have changed. A recent pull request passed but develop failed after it was approved because files that were referenced in the use case documentation files had been deleted as part of the PR. * feature 1273 TCGen -edeck and -shape (#1424) * Feature 966 Fix mask.poly logic (#1430) * Feature 1030 total seconds documentation (#1428) Co-authored-by: George McCabe <[email protected]> * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * get branch name in job controls job * replace logic to update input data volumes with call to dtcenter/metplus-action-data-update action * feature 1431 remove obs_prepbufr_map (#1433) * Per #675, added images and updated content * added statsmodel package to metplotpy docker conda environment * Added fcst_file_type to two wrappers (#1437) Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: George McCabe <[email protected]> * added release notes for tasks that have already been completed for beta6 * updated env var for METplotpy base to match changes made to the METplotpy repository * Updated METplotpy base env var to preemptively match changes from PR dtcenter/METplotpy#197. Once that PR is merged into develop, a test workflow will run that should fix this issue. * removed commented code that is no longer used * update the instructions for updating test input data to ensure volume_mount_directories file is updated in the upcoming version directory, i.e. v4.1, so that the Docker data volumes from the release will be created properly * removed duplicate script * updated script to copy volume_mount_directories file into new METplus version directory * added logic to skip files that are temporary copies of tar files that should not be added to new version directory * updated release guide instructions for creating METplus data directory for next development cycle * Feature 911 tc trackint (#1439) Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * added new use cases that are currently being reviewed to release notes * updated event info in contributor's guide * Feature 626 fix hovmoeller documentation (#1444) * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py replace hovmoeller_diagram.conf with UserScript_obsPrecip_obsOnly_Hovmoeller.conf * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py Removed a typo: UserScript This uses data from... Now: This uses data from... * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py * Modified the PR template to add review of the source issue metadata ci-skip-all * feature_1403_python_components_table (#1446) * creating new METplus Components Python Requirements * adding a return to test * testing use case links * testing use case links #2 * testing use case links #3 * testing use case links #4 * Filling in some use cases * fixing indentation #1403 * fixing indentation #1403 take 2 * fixing indentation #1403 take 3 * cleaning up anything after # in the links #1403 * entries thru D #1403 * entries thru H #1403 * fixing indents #1403 * fixing indents #1403 take 2 * fixing web name #1403 * entries thru L #1403 * entry matplotlib #1403 * entry matplotlib #1403 marking duplicate names * entry metcalcpy #1403 * entry metcalcpy fix indents #1403 * entry matplotlib #1403 fixing duplicate names * entry metplotpy #1403 * entry metplotpy #1403 marking duplicate names * entry metplotpy #1403 fixing * entry thru ps #1403 * double names #1403 * character fix #1403 * adding in missing links #1403 * thru pyproj #1403 * problem with line 353 #1403 * thru pyproj #1403 typos * more duplicate names #1403 * line break #1403 * loose ends #1403 * adding >= to some of the versions #1403 * adding name to the Use Cases to avoid duplicate naming issues TEST #1403 * adding name to the Use Cases to avoid duplicate naming issues thru pyproj #1403 * adding thru pyresample #1403 * fixing pyresample typo #1403 * adding thru scikit-image #1403 * fixing line breaks#1403 * web links thru scikit-learn #1403 * web links thru scipy #1403 * web links thru sklearn #1403 * Duplicate web names. temp. fix #1403 * Duplicate web names fixed #1403 * Duplicate web names fixing #3 #1403 * Duplicate web names fixing #4 #1403 * Duplicate web names fixing #5 #1403 * table end #1403 * table loose ends #1403 * Removed unnecessary comma * table loose ends after pull request #1403 * Minor modifications to METplus Component column * Changed description of pyproj * Added link for last referenced use case * Attempting to fix doc warning Co-authored-by: jprestop <[email protected]> * Feature 1318 usecase aviso (#1440) * Correct small typo. * Update pull_request_template.md * Added files for ReadTheDocs (#864) * Updated documentation links for ReadtheDocs * Update pull_request_template.md Added entry for completion date for pull request review. * Adding python embedding script to read AVISO, RTOFS, HYCOM and OSTIA data * Updated the script - working but only RMSE values match * Adding documentation for the AVISO case * Adding a conf file for the AVISO case * Updates to the codel to include model3, obs, climo3 but also has many extra print statements, netcdf output files which needs to be cleaned * Cleaning up the conf file and adding SAL1L2 to get the model_climo sdev and obs_climo sdev * Updating the file to fix the standard names and add the model3, climo3 and obs3 * Updating the file to fix the standard names and add the model3, climo3 and obs3 * Adding an image to the use case. Image Courtesy: EMC/NOAA website * added to use case list * updated case description * removed merge issues * caught second docs issue * removed other doc merge issues, turned off auto test * more merge comments sneaking in Co-authored-by: johnhg <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: George McCabe <[email protected]> Co-authored-by: johnhg <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: lisagoodrich <[email protected]> Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> Co-authored-by: j-opatz <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: j-opatz <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]>

* Feature 1091 extent cycloneplotter (#1218) * Github #1091 add support to define plot's extent * Github #1091 Add support for defining plot's extent * Github #1091 add support for defining plot extent * GitHub #1091 Provide support to define plotting a bounding box of interest and replace deprecated cartopy attributes. * Github Issue #1091 Add configuration support to define the bounding box defining the area of interest to plot. * Github Issue #1091 provide support to define bounding box defining the area of interest to plot. * Github Issue #1091 Removed unused shapely imports * GItHub Issue #1091 Removed duplicate CYCLONE_PLOTTER_GLOBAL_PLOT * Github #1091 Include config settings needed to indicate the region of interest to be plotted. * Github Issue #1091 Keep only the North Hemisphere lon and lats in the config file to serve as example for plotting a specific area of the map. * convert file back to unix format via dos2unix and added change that was lost in merge * return None from function instead of exiting so that METplus clean up functionality will still run * fixed indentation * fixed indentation - for loop should not be nested inside other for loop * fixed check for failure in retrieve_data function * feature 1223 error if file not found (#1238) * feature 1252 allow dictionary value for time_summary.width (#1253) * feature 1213 obs_quality_inc/exc (#1260) * Feature 1203 ioda2nc (#1262) * Add default title for the new use case issue template. * Feature 1019 harmonic preprocessing (#1272) Co-authored-by: George McCabe <[email protected]> * Feature 1266 gen ens prod missing ensembles (#1275) * Feature 1049 statistics list (#1271) * first attempt at a table #1049 * 2nd attempt, smaller, different formatting table #1049 * trying to get rid of git error message by adding this to the index #1049 * alignment #1049 * trying to text wrap in a table #1049 * attempting to create text wrapping in tables #1049 * removing role, trying to wrap text #1049 * removing html break, trying to wrap text #1049 * adding code to wrap text #1049 * removing code to wrap text #1049 * fixing naming typo #1049 * hitting returns in table #1049 * removing typo file, removing line breaks #1049 * attempting a simple table for text wrapping #1049 * attempting to bold header row in simple table #1049 * simple table line break attempt #1049 * simple table line break attempt #2 #1049 * simple table line break attempt #3 #1049 * trying another | #1049 * 2 trying another | #1049 * 3 trying another | #1049 * trying a blank line #1049 * trying reformating grid #1049 * 2 trying reformating grid #1049 * 3 trying reformating grid #1049 * 4 trying reformating grid #1049 * adding in one more table row #1049 * adding more text #1049 * trying glossary format #1049 * 2 trying glossary format #1049 * 3 trying glossary format #1049 * 4 trying glossary format #1049 * adding glossary format 2D objects #1049 * fixing glossary format 2D objects #1049 * adding glossary items #1049 * fixing glossary format #1049 * hopefully all examples are working #1049 * changing reference to tool * removing 2D object examples * removing the unwrapped table * adding ACC into the example * fixing formatting #1049 * fixing formatting take 2 #1049 * fixing formatting take 3 #1049 * bolding and language change #1049 * fixing spacing #1049 * fixing spacing again #1049 * changing title #1049 * trying superscript #1049 * superscript glossary #1049 * removing quotes #1049 * fixing spacing #1049 * Tara has decided to go with the glossary format. Removing table example. #1049 * Adding 2D objects #1049 * 2D objects fix #1049 * 2D objects fix attempt 2 #1049 * 2D objects fix attempt 3 #1049 * 2D objects fix attempt 4 #1049 * 2D objects fix attempt 5 #1049 * 2D objects fix attempt 7 #1049 * 2D objects fix attempt 8 #1049 * ABR added #1049 * adding remaining AB items to list #1049 * adding remaining ACC_ items to list #1049 * ADLAND & AFSS entries #1049 * A entries #1049 * fixing formatting #1049 * fixing formatting #2 #1049 * removing new entries to make it run #1049 * adding a couple back in #1049 * adding a couple more back in #1049 * tool key at the bottom #1049 * spacing changes #1049 * spacing changes another try #1049 * fixing spacing #1049 * fixing spacing attempt 2 #1049 * fixing spacing attempt 3 #1049 * through AREA items #1049 * AMODEL listed twice but it's not #1049 * AMODEL listed twice but it's not #2 #1049 * AMODEL added question marks so it will run #1049 * final A entries #1049 * naming the glossary statistics in hopes of not conflicting with original glossary #1049 * removing glossary statistics space #1049 * Testing multiple glossary names * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, removing code with attempts at mutiple glossaries * Per #1067, adding back three question marks due to duplicate term * paring down list, adding a table for review * problems with table * still trying to get it to publish * still trying to get a second line in the table * still trying to get a second line in the table 2 * testing different formatting * testing different formatting * trying to comment out the glossary inner workings * formatting again * line breaks in table #1049 * adding in some more for an example #1049 * adding new entries in through AREA #1049 * fixing line breaks #1049 * fixing line breaks #1049 #2 * fixing line breaks and warning messages #1049 * fixing line breaks #1049 * adding AREA_RATIO through ASPECT_DIFF * fixing typo #1049 * fixing typo #1049 * fixing typo #1049 take 2 * adding AXIS_ANG to BCMSE #1049 * adding spacing #1049 * adding spacing removing commas #1049 * removing comma #1049 * BOUNDARY_DIST thru BSS_SMPL #1049 * BOUNDARY_DIST splitting across 2 lines #1049 * cleaning up typos #1049 * calibration thru centriod_dist #1049 * centriod_lat thru centroid_y #1049 * fixing spacing #1049 * removing test glossary #1049 * climo_mean thru crtk_err #1049 * fixing crtk_err spacing #1049 * fixing spacing #1049 * CSI to CURVATURE_Y #1049 * CURVATURE_X & Y spacing #1049 * DEV_CAT to DURATION_DIFF #1049 * EC_VALUE to F #1049 * F_RATE TO FBS #1049 * Fixing spacing #1049 * fcst_clus thru fcst_conv_radius #1049 * removing CTOP_PRS #1049 * fixing the order of tools for FBAR and FBIAS #1049 * fixing spacing #1049 * adding grid-stat to all point-stat entries #1049 * adding fixing spacing #1049 * adding fixing spacing take 2 #1049 * adding fixing spacing take 3 #1049 * adding fixing spacing take 4 #1049 * adding fixing spacing take 5 #1049 * adding fixing spacing take 6 #1049 * adding fixing spacing take 7 #1049 * fixing spacing with a period take 7 #1049 * first attempt fcst_ #1049 * fixing typos #1049 * thru FOBAR #1049 * thru end of F #1049 * g thru h #1049 * i thru intensity #s #1049 * fixing typos #1049 * capturing example for Julie #1049 * thru k #1049 * fixing typos #1049 * thru L #1049 * thru MG #1049 * thru N_ENS #1049 * thru all N #1049 * fixing FBIAS alignment #1049 * fixing ME and MSE alignment #1049 * fixing ME alignment take 2 #1049 * thru OBS_E #1049 * fixing alignment #1049 * fixing alignment n_thresh #1049 * thru OBS_thresh #1049 * thru O #1049 * fixing OOBAR formating #1049 * thru PR_CORR #1049 * commented lines out with line total info #1049 * commented lines out with line total info take 2 #1049 * thru R #1049 * thru SPEED #1049 * thru S #1049 * thru T #1049 * thru U #1049 * thru V #1049 * thru V #1049 * thru Z #1049 * Update statistics_list.rst Tara is testing editing in UI * Update statistics_list.rst Updates through the A's * Update statistics_list.rst Cleaning up the A's * Update statistics_list.rst Standardizing MODE and MTD entries * Update statistics_list.rst Updating B's and C's * Update statistics_list.rst Testing adding TC-Stat and TCST to an entry * Update statistics_list.rst Clean up of a few A-Cs and then update of Ds * Update statistics_list.rst A few clean-ups and Es * Update statistics_list.rst Halfway through Fs... * Update statistics_list.rst A little clean up and the rest of Fs * Update statistics_list.rst G, H, I, Ks * Update statistics_list.rst L, M, Ns * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst Rs * Update statistics_list.rst S and Ts * Update statistics_list.rst The rest of the list * Update statistics_list.rst Removed Attr from Stat Type thru E * Update statistics_list.rst Remove Attr from Statistics Type through Gs * Update statistics_list.rst Remove Attr from Statistic Type through Rs * Update statistics_list.rst Remove Attr from Stat Type to the end * Update statistics_list.rst Cleaned up some Line Type typos * Update statistics_list.rst Still more Attr cleanup Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> * Feature 1263 v4.1.0 beta4 (#1277) * update version to note development towards beta5 * Feature 934 release stage doc (#1235) * Per #934 add stages of the METplus release cycle. * Per #934, adding link to descriptions of the release cycle in the User's Guide. * Per #934, made corrections * Per #934, changed Beta and Release Candidate (rc) from bold to subsubsections. * Update index.rst Co-authored-by: Julie Prestopnik <[email protected]> * Feature 344 met util refactor (#1292) * removed deprecated sections from config examples * minor change to METplus release guide to add a link to the PDF of the User's Guide instead of downloading it and attaching it to the release * Feature 1285 extract tiles mtd times (#1315) * Feature 896 more met config (#1322) * removed incorrect search keyword * added workflow_dispatch event so workflow can be triggered by an external repository such as MET to test to ensure that changes from that repo will break anything in METplus * added another input argument for workflow_dispatch event * added job with name that shows the event name or the repository name if triggered by an external repository such as MET * GHA: add username that triggered external event to event info job name * added required input argument for external trigger that contains the commit hash of the push event that triggered in the other repo * change event info to show commit hash instead of username that merged the PR * changed input names to match names of event in repository that triggered workflow * feature 1320 OMP_NUM_THREADS (#1338) * Feature 1183 memory documentation (#1340) Co-authored-by: George McCabe <[email protected]> * add email address of user who triggered push event to job name * Feature 1166 series analysis field info (#1353) * Feature 1116 usecase smos (#1348) Co-authored-by: Mrinal Biswas <[email protected]> * turn off new use case from every push * feature 1236 Control Members in EnsembleStat and GenEnsProd (#1357) * added optional argument to change the directory to untar new input data into so the same Dockerfile can be used to add data for other METplus components such as MET * feature 1358 v4.1.0-beta5 release (#1359) * update version for next development cycle * Feature 1216 usecase smap (#1361) * Adding a conf file for SMAP * Adding a directory to host the read file * Removing temp file * Updated the valid dates to match Todd's code * Adding documentation for SMAP case * Updates the valis dates to match Todd's code * Removing a tmp file * Typo in file name * Updating the input RTOFS to have the init time instead of the valid time ii the file name * updated file paths, tesing * updated use case descriptions, rearranged use case group testing * put new use case into its own group so that the diff logic can evaluate marine_and_cryosphere:3. The truth data for 3-4 does not exist yet so the diff fails. Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * updated marine_and_cryo grouping * Feature 1230 stratosphere metrics (#1354) * Initial checkin for Meridial Mean use case * Changed the name and directories * Added some documentation * Fixed config variables * Issue 1230 Stratospheric metrics (zonal/meridional use case) put INPUT_BASE in the user_env_vars so the user doesn't need to set the INPUT_BASE environment in the current working shell * Issue #1230 remove import of metplotpy-this isn't called * Issue #1230_stratosphere_metric Add use case to the list of use cases to be run for testing * Issue 1230 Remove the INPUT_BASE from the user_env_vars section * Issue #1230 redundant files * Issue #1230 redundant files * Issue #1230 redundant files * Issue #120 replace INPUT_BASE with INPUT_FILE_NAME * Issue #1230 remove entry for INPUT_FILE_NAME, this goes in the system.conf * Update all_use_cases.txt Copy and pasted #11 from s2s use case for the Stratosphere use case but forgot to update the index to 12. * Update use_case_groups.json added use case #12 from s2s to test stratosphere use case * Issue #1230 put INPUT_FILE_NAME back under the user_env_vars * Issue #1230 forgot to include the filename * Issue #1230 clean up config file, remove uneccessary comments, group related entries in the user_env_vars * Issue #1230 remove extraneous and incorrect path to the input_filename setting * Removed pingouin dependency * Issue #1230 another cut and paste error fixed for the Stratosphere use case * Issue #1230 type in name, obs_Only should be obsOnly * Issue #1230 added the metdatadb to the env, code imports metdatadb and may require some of these dependencies * issue #1230 removed pingouin dependency from comment to reduce any confusion * issue #1230 turn off the test for the Stratosphere metrics use case * Added use case image Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: Christina Kalb <[email protected]> * Corrected spelling of occurrence in two places * Fixed misspelling of occurrence * Found and fixed two more misspellings of occurence * Changed version specific information to by X.Y.Z moved text indicating to click save to the bottom of the list. * Feature 1374 python packages (#1378) * Added documentation about updating spreadsheet of Python requirements * Changed references to master_metplus.py to run_metplus.py * Fixed formatting of section with run_metplus.py commands * Updated text for updating the spreadsheet * Removed a newly added section that wasn't needed Co-authored-by: Julie Prestopnik <[email protected]> * feature 1368 PCPCombine use zero accum (#1381) * feature 1369 grid_weight_flag in EnsembleStat (#1379) * Per #1356, change how wrappers create instances of other wrappers to ensure that config settings for the created instance do not change values in METplusConfig used by the rest of the wrappers, ci-run-all-diff * Per #1356, remove config_overrides functionality in favor of using instances -- this prevents the issue where overrides for a given tool affect the global config settings. it also forces the configs to override for an instance to be put into another config section so that it will be available in the final conf, ci-run-all-diff * Per #1356, update documentation to reflect changes * Per #1356, fixed unit test to no longer use deprecated approach to overriding config variables * feature 1247 climatology settings (#1385) * feature 1356 isolate config (#1386) * Feature 675 Continuous Integration Documentation (#1409) Co-authored-by: johnhg <[email protected]> * feature 1382 Filename templates in MODEL (#1414) * Feature 1289 explicit file list (#1387) * Per PR #1387 review, fix typo * Bugfix 1421 EnsembleStat use fcst fields in ens dictionary if ens is unset (#1422) * fixed missing path change for docker file path * removed variables that are not used * Feature 1408 use case ptype (#1425) * adding imagery, config file, updating internal tests * updated lists, auto testing * corrected old reference * Update use_case_groups.json * only trigger testing workflow for pull requests that are going into develop or main_* branches * Feature 1371 blocking wr (#1426) * turn off use cases for push events * Feature 1392 climo_cdf.direct_prob and SeriesAnalysis (#1396) * Changed event triggering rules for documentation workflow so that it will run for pull requests even if no files in the docs directory have changed. A recent pull request passed but develop failed after it was approved because files that were referenced in the use case documentation files had been deleted as part of the PR. * feature 1273 TCGen -edeck and -shape (#1424) * Feature 966 Fix mask.poly logic (#1430) * Feature 1030 total seconds documentation (#1428) Co-authored-by: George McCabe <[email protected]> * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * get branch name in job controls job * replace logic to update input data volumes with call to dtcenter/metplus-action-data-update action * feature 1431 remove obs_prepbufr_map (#1433) * Per #675, added images and updated content * added statsmodel package to metplotpy docker conda environment * Added fcst_file_type to two wrappers (#1437) Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: George McCabe <[email protected]> * added release notes for tasks that have already been completed for beta6 * updated env var for METplotpy base to match changes made to the METplotpy repository * Updated METplotpy base env var to preemptively match changes from PR dtcenter/METplotpy#197. Once that PR is merged into develop, a test workflow will run that should fix this issue. * removed commented code that is no longer used * update the instructions for updating test input data to ensure volume_mount_directories file is updated in the upcoming version directory, i.e. v4.1, so that the Docker data volumes from the release will be created properly * removed duplicate script * updated script to copy volume_mount_directories file into new METplus version directory * added logic to skip files that are temporary copies of tar files that should not be added to new version directory * updated release guide instructions for creating METplus data directory for next development cycle * Feature 911 tc trackint (#1439) Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * added new use cases that are currently being reviewed to release notes * updated event info in contributor's guide * Feature 626 fix hovmoeller documentation (#1444) * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py replace hovmoeller_diagram.conf with UserScript_obsPrecip_obsOnly_Hovmoeller.conf * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py Removed a typo: UserScript This uses data from... Now: This uses data from... * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py * Modified the PR template to add review of the source issue metadata ci-skip-all * feature_1403_python_components_table (#1446) * creating new METplus Components Python Requirements * adding a return to test * testing use case links * testing use case links #2 * testing use case links #3 * testing use case links #4 * Filling in some use cases * fixing indentation #1403 * fixing indentation #1403 take 2 * fixing indentation #1403 take 3 * cleaning up anything after # in the links #1403 * entries thru D #1403 * entries thru H #1403 * fixing indents #1403 * fixing indents #1403 take 2 * fixing web name #1403 * entries thru L #1403 * entry matplotlib #1403 * entry matplotlib #1403 marking duplicate names * entry metcalcpy #1403 * entry metcalcpy fix indents #1403 * entry matplotlib #1403 fixing duplicate names * entry metplotpy #1403 * entry metplotpy #1403 marking duplicate names * entry metplotpy #1403 fixing * entry thru ps #1403 * double names #1403 * character fix #1403 * adding in missing links #1403 * thru pyproj #1403 * problem with line 353 #1403 * thru pyproj #1403 typos * more duplicate names #1403 * line break #1403 * loose ends #1403 * adding >= to some of the versions #1403 * adding name to the Use Cases to avoid duplicate naming issues TEST #1403 * adding name to the Use Cases to avoid duplicate naming issues thru pyproj #1403 * adding thru pyresample #1403 * fixing pyresample typo #1403 * adding thru scikit-image #1403 * fixing line breaks#1403 * web links thru scikit-learn #1403 * web links thru scipy #1403 * web links thru sklearn #1403 * Duplicate web names. temp. fix #1403 * Duplicate web names fixed #1403 * Duplicate web names fixing #3 #1403 * Duplicate web names fixing #4 #1403 * Duplicate web names fixing #5 #1403 * table end #1403 * table loose ends #1403 * Removed unnecessary comma * table loose ends after pull request #1403 * Minor modifications to METplus Component column * Changed description of pyproj * Added link for last referenced use case * Attempting to fix doc warning Co-authored-by: jprestop <[email protected]> * Feature 1318 usecase aviso (#1440) * Correct small typo. * Update pull_request_template.md * Added files for ReadTheDocs (#864) * Updated documentation links for ReadtheDocs * Update pull_request_template.md Added entry for completion date for pull request review. * Adding python embedding script to read AVISO, RTOFS, HYCOM and OSTIA data * Updated the script - working but only RMSE values match * Adding documentation for the AVISO case * Adding a conf file for the AVISO case * Updates to the codel to include model3, obs, climo3 but also has many extra print statements, netcdf output files which needs to be cleaned * Cleaning up the conf file and adding SAL1L2 to get the model_climo sdev and obs_climo sdev * Updating the file to fix the standard names and add the model3, climo3 and obs3 * Updating the file to fix the standard names and add the model3, climo3 and obs3 * Adding an image to the use case. Image Courtesy: EMC/NOAA website * added to use case list * updated case description * removed merge issues * caught second docs issue * removed other doc merge issues, turned off auto test * more merge comments sneaking in Co-authored-by: johnhg <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> * Revert "Feature 1318 usecase aviso (#1440)" (#1459) This reverts commit d3021de. Co-authored-by: bikegeek <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: George McCabe <[email protected]> Co-authored-by: johnhg <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: lisagoodrich <[email protected]> Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]>

* thru O #1049 * fixing OOBAR formating #1049 * thru PR_CORR #1049 * commented lines out with line total info #1049 * commented lines out with line total info take 2 #1049 * thru R #1049 * thru SPEED #1049 * thru S #1049 * thru T #1049 * thru U #1049 * thru V #1049 * thru V #1049 * thru Z #1049 * Update statistics_list.rst Tara is testing editing in UI * Update statistics_list.rst Updates through the A's * Update statistics_list.rst Cleaning up the A's * Update statistics_list.rst Standardizing MODE and MTD entries * Update statistics_list.rst Updating B's and C's * Update statistics_list.rst Testing adding TC-Stat and TCST to an entry * feature 1252 allow dictionary value for time_summary.width (#1253) * Update statistics_list.rst Clean up of a few A-Cs and then update of Ds * Update statistics_list.rst A few clean-ups and Es * Update statistics_list.rst Halfway through Fs... * feature 1213 obs_quality_inc/exc (#1260) * Feature 1203 ioda2nc (#1262) * Update statistics_list.rst A little clean up and the rest of Fs * Update statistics_list.rst G, H, I, Ks * Update statistics_list.rst L, M, Ns * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst Rs * Update statistics_list.rst S and Ts * Update statistics_list.rst The rest of the list * Update statistics_list.rst Removed Attr from Stat Type thru E * Update statistics_list.rst Remove Attr from Statistics Type through Gs * Update statistics_list.rst Remove Attr from Statistic Type through Rs * Update statistics_list.rst Remove Attr from Stat Type to the end * Update statistics_list.rst Cleaned up some Line Type typos * Update statistics_list.rst Still more Attr cleanup * Add default title for the new use case issue template. * Feature 1019 harmonic preprocessing (#1272) Co-authored-by: George McCabe <[email protected]> * Feature 1266 gen ens prod missing ensembles (#1275) * Feature 1049 statistics list (#1271) * first attempt at a table #1049 * 2nd attempt, smaller, different formatting table #1049 * trying to get rid of git error message by adding this to the index #1049 * alignment #1049 * trying to text wrap in a table #1049 * attempting to create text wrapping in tables #1049 * removing role, trying to wrap text #1049 * removing html break, trying to wrap text #1049 * adding code to wrap text #1049 * removing code to wrap text #1049 * fixing naming typo #1049 * hitting returns in table #1049 * removing typo file, removing line breaks #1049 * attempting a simple table for text wrapping #1049 * attempting to bold header row in simple table #1049 * simple table line break attempt #1049 * simple table line break attempt #2 #1049 * simple table line break attempt #3 #1049 * trying another | #1049 * 2 trying another | #1049 * 3 trying another | #1049 * trying a blank line #1049 * trying reformating grid #1049 * 2 trying reformating grid #1049 * 3 trying reformating grid #1049 * 4 trying reformating grid #1049 * adding in one more table row #1049 * adding more text #1049 * trying glossary format #1049 * 2 trying glossary format #1049 * 3 trying glossary format #1049 * 4 trying glossary format #1049 * adding glossary format 2D objects #1049 * fixing glossary format 2D objects #1049 * adding glossary items #1049 * fixing glossary format #1049 * hopefully all examples are working #1049 * changing reference to tool * removing 2D object examples * removing the unwrapped table * adding ACC into the example * fixing formatting #1049 * fixing formatting take 2 #1049 * fixing formatting take 3 #1049 * bolding and language change #1049 * fixing spacing #1049 * fixing spacing again #1049 * changing title #1049 * trying superscript #1049 * superscript glossary #1049 * removing quotes #1049 * fixing spacing #1049 * Tara has decided to go with the glossary format. Removing table example. #1049 * Adding 2D objects #1049 * 2D objects fix #1049 * 2D objects fix attempt 2 #1049 * 2D objects fix attempt 3 #1049 * 2D objects fix attempt 4 #1049 * 2D objects fix attempt 5 #1049 * 2D objects fix attempt 7 #1049 * 2D objects fix attempt 8 #1049 * ABR added #1049 * adding remaining AB items to list #1049 * adding remaining ACC_ items to list #1049 * ADLAND & AFSS entries #1049 * A entries #1049 * fixing formatting #1049 * fixing formatting #2 #1049 * removing new entries to make it run #1049 * adding a couple back in #1049 * adding a couple more back in #1049 * tool key at the bottom #1049 * spacing changes #1049 * spacing changes another try #1049 * fixing spacing #1049 * fixing spacing attempt 2 #1049 * fixing spacing attempt 3 #1049 * through AREA items #1049 * AMODEL listed twice but it's not #1049 * AMODEL listed twice but it's not #2 #1049 * AMODEL added question marks so it will run #1049 * final A entries #1049 * naming the glossary statistics in hopes of not conflicting with original glossary #1049 * removing glossary statistics space #1049 * Testing multiple glossary names * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, removing code with attempts at mutiple glossaries * Per #1067, adding back three question marks due to duplicate term * paring down list, adding a table for review * problems with table * still trying to get it to publish * still trying to get a second line in the table * still trying to get a second line in the table 2 * testing different formatting * testing different formatting * trying to comment out the glossary inner workings * formatting again * line breaks in table #1049 * adding in some more for an example #1049 * adding new entries in through AREA #1049 * fixing line breaks #1049 * fixing line breaks #1049 #2 * fixing line breaks and warning messages #1049 * fixing line breaks #1049 * adding AREA_RATIO through ASPECT_DIFF * fixing typo #1049 * fixing typo #1049 * fixing typo #1049 take 2 * adding AXIS_ANG to BCMSE #1049 * adding spacing #1049 * adding spacing removing commas #1049 * removing comma #1049 * BOUNDARY_DIST thru BSS_SMPL #1049 * BOUNDARY_DIST splitting across 2 lines #1049 * cleaning up typos #1049 * calibration thru centriod_dist #1049 * centriod_lat thru centroid_y #1049 * fixing spacing #1049 * removing test glossary #1049 * climo_mean thru crtk_err #1049 * fixing crtk_err spacing #1049 * fixing spacing #1049 * CSI to CURVATURE_Y #1049 * CURVATURE_X & Y spacing #1049 * DEV_CAT to DURATION_DIFF #1049 * EC_VALUE to F #1049 * F_RATE TO FBS #1049 * Fixing spacing #1049 * fcst_clus thru fcst_conv_radius #1049 * removing CTOP_PRS #1049 * fixing the order of tools for FBAR and FBIAS #1049 * fixing spacing #1049 * adding grid-stat to all point-stat entries #1049 * adding fixing spacing #1049 * adding fixing spacing take 2 #1049 * adding fixing spacing take 3 #1049 * adding fixing spacing take 4 #1049 * adding fixing spacing take 5 #1049 * adding fixing spacing take 6 #1049 * adding fixing spacing take 7 #1049 * fixing spacing with a period take 7 #1049 * first attempt fcst_ #1049 * fixing typos #1049 * thru FOBAR #1049 * thru end of F #1049 * g thru h #1049 * i thru intensity #s #1049 * fixing typos #1049 * capturing example for Julie #1049 * thru k #1049 * fixing typos #1049 * thru L #1049 * thru MG #1049 * thru N_ENS #1049 * thru all N #1049 * fixing FBIAS alignment #1049 * fixing ME and MSE alignment #1049 * fixing ME alignment take 2 #1049 * thru OBS_E #1049 * fixing alignment #1049 * fixing alignment n_thresh #1049 * thru OBS_thresh #1049 * thru O #1049 * fixing OOBAR formating #1049 * thru PR_CORR #1049 * commented lines out with line total info #1049 * commented lines out with line total info take 2 #1049 * thru R #1049 * thru SPEED #1049 * thru S #1049 * thru T #1049 * thru U #1049 * thru V #1049 * thru V #1049 * thru Z #1049 * Update statistics_list.rst Tara is testing editing in UI * Update statistics_list.rst Updates through the A's * Update statistics_list.rst Cleaning up the A's * Update statistics_list.rst Standardizing MODE and MTD entries * Update statistics_list.rst Updating B's and C's * Update statistics_list.rst Testing adding TC-Stat and TCST to an entry * Update statistics_list.rst Clean up of a few A-Cs and then update of Ds * Update statistics_list.rst A few clean-ups and Es * Update statistics_list.rst Halfway through Fs... * Update statistics_list.rst A little clean up and the rest of Fs * Update statistics_list.rst G, H, I, Ks * Update statistics_list.rst L, M, Ns * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst Rs * Update statistics_list.rst S and Ts * Update statistics_list.rst The rest of the list * Update statistics_list.rst Removed Attr from Stat Type thru E * Update statistics_list.rst Remove Attr from Statistics Type through Gs * Update statistics_list.rst Remove Attr from Statistic Type through Rs * Update statistics_list.rst Remove Attr from Stat Type to the end * Update statistics_list.rst Cleaned up some Line Type typos * Update statistics_list.rst Still more Attr cleanup Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> * creating a separate list for diagnostics, in progesss * removing diagnostics from statistics, in progess #1049 * adding diagnostics_list to TOC #1049 * separating lists * fixing typos #1049 * moving blank statistic type from statistics to diagnostics #1049 * Feature 1263 v4.1.0 beta4 (#1277) * update version to note development towards beta5 * fixing errors #1049 * updating table name #1049 * fixing typo #1049 * Feature 934 release stage doc (#1235) * Per #934 add stages of the METplus release cycle. * Per #934, adding link to descriptions of the release cycle in the User's Guide. * Per #934, made corrections * Per #934, changed Beta and Release Candidate (rc) from bold to subsubsections. * Update index.rst Co-authored-by: Julie Prestopnik <[email protected]> * putting both tables into one chapter * removing diagnostics since it's going back into the statistics chapter #1049 * Delete diagnostics_list.rst This table will be added to the statistics page as a separate table. #1049 * test. breaking out directories A-B and C-D #1049 * testing with section names #1049 * alphbetical sections for statistics #1049 * making U-Z statistics list #1049 * alpabetical breaks for diagnostics list #1049 * trying to fix errors #1049 * trying to fix errors take 2 #1049 * trying to fix errors take 3 #1049 * Removed a unnecessary space #1049 * Making all METplus Names CAPITAL LETTERS #1049 * Feature 344 met util refactor (#1292) * removed deprecated sections from config examples * minor change to METplus release guide to add a link to the PDF of the User's Guide instead of downloading it and attaching it to the release * Feature 1285 extract tiles mtd times (#1315) * Feature 896 more met config (#1322) * removed incorrect search keyword * added workflow_dispatch event so workflow can be triggered by an external repository such as MET to test to ensure that changes from that repo will break anything in METplus * added another input argument for workflow_dispatch event * added job with name that shows the event name or the repository name if triggered by an external repository such as MET * GHA: add username that triggered external event to event info job name * added required input argument for external trigger that contains the commit hash of the push event that triggered in the other repo * change event info to show commit hash instead of username that merged the PR * changed input names to match names of event in repository that triggered workflow * feature 1320 OMP_NUM_THREADS (#1338) * Feature 1183 memory documentation (#1340) Co-authored-by: George McCabe <[email protected]> * add email address of user who triggered push event to job name * Feature 1166 series analysis field info (#1353) * Feature 1116 usecase smos (#1348) Co-authored-by: Mrinal Biswas <[email protected]> * turn off new use case from every push * feature 1236 Control Members in EnsembleStat and GenEnsProd (#1357) * added optional argument to change the directory to untar new input data into so the same Dockerfile can be used to add data for other METplus components such as MET * feature 1358 v4.1.0-beta5 release (#1359) * update version for next development cycle * Feature 1216 usecase smap (#1361) * Adding a conf file for SMAP * Adding a directory to host the read file * Removing temp file * Updated the valid dates to match Todd's code * Adding documentation for SMAP case * Updates the valis dates to match Todd's code * Removing a tmp file * Typo in file name * Updating the input RTOFS to have the init time instead of the valid time ii the file name * updated file paths, tesing * updated use case descriptions, rearranged use case group testing * put new use case into its own group so that the diff logic can evaluate marine_and_cryosphere:3. The truth data for 3-4 does not exist yet so the diff fails. Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * updated marine_and_cryo grouping * Feature 1230 stratosphere metrics (#1354) * Initial checkin for Meridial Mean use case * Changed the name and directories * Added some documentation * Fixed config variables * Issue 1230 Stratospheric metrics (zonal/meridional use case) put INPUT_BASE in the user_env_vars so the user doesn't need to set the INPUT_BASE environment in the current working shell * Issue #1230 remove import of metplotpy-this isn't called * Issue #1230_stratosphere_metric Add use case to the list of use cases to be run for testing * Issue 1230 Remove the INPUT_BASE from the user_env_vars section * Issue #1230 redundant files * Issue #1230 redundant files * Issue #1230 redundant files * Issue #120 replace INPUT_BASE with INPUT_FILE_NAME * Issue #1230 remove entry for INPUT_FILE_NAME, this goes in the system.conf * Update all_use_cases.txt Copy and pasted #11 from s2s use case for the Stratosphere use case but forgot to update the index to 12. * Update use_case_groups.json added use case #12 from s2s to test stratosphere use case * Issue #1230 put INPUT_FILE_NAME back under the user_env_vars * Issue #1230 forgot to include the filename * Issue #1230 clean up config file, remove uneccessary comments, group related entries in the user_env_vars * Issue #1230 remove extraneous and incorrect path to the input_filename setting * Removed pingouin dependency * Issue #1230 another cut and paste error fixed for the Stratosphere use case * Issue #1230 type in name, obs_Only should be obsOnly * Issue #1230 added the metdatadb to the env, code imports metdatadb and may require some of these dependencies * issue #1230 removed pingouin dependency from comment to reduce any confusion * issue #1230 turn off the test for the Stratosphere metrics use case * Added use case image Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: Christina Kalb <[email protected]> * Corrected spelling of occurrence in two places * Fixed misspelling of occurrence * Found and fixed two more misspellings of occurence * Changed version specific information to by X.Y.Z moved text indicating to click save to the bottom of the list. * Feature 1374 python packages (#1378) * Added documentation about updating spreadsheet of Python requirements * Changed references to master_metplus.py to run_metplus.py * Fixed formatting of section with run_metplus.py commands * Updated text for updating the spreadsheet * Removed a newly added section that wasn't needed Co-authored-by: Julie Prestopnik <[email protected]> * feature 1368 PCPCombine use zero accum (#1381) * feature 1369 grid_weight_flag in EnsembleStat (#1379) * Per #1356, change how wrappers create instances of other wrappers to ensure that config settings for the created instance do not change values in METplusConfig used by the rest of the wrappers, ci-run-all-diff * Per #1356, remove config_overrides functionality in favor of using instances -- this prevents the issue where overrides for a given tool affect the global config settings. it also forces the configs to override for an instance to be put into another config section so that it will be available in the final conf, ci-run-all-diff * Per #1356, update documentation to reflect changes * Per #1356, fixed unit test to no longer use deprecated approach to overriding config variables * feature 1247 climatology settings (#1385) * feature 1356 isolate config (#1386) * Feature 675 Continuous Integration Documentation (#1409) Co-authored-by: johnhg <[email protected]> * feature 1382 Filename templates in MODEL (#1414) * Feature 1289 explicit file list (#1387) * Per PR #1387 review, fix typo * Bugfix 1421 EnsembleStat use fcst fields in ens dictionary if ens is unset (#1422) * fixed missing path change for docker file path * removed variables that are not used * Feature 1408 use case ptype (#1425) * adding imagery, config file, updating internal tests * updated lists, auto testing * corrected old reference * Update use_case_groups.json * only trigger testing workflow for pull requests that are going into develop or main_* branches * Feature 1371 blocking wr (#1426) * turn off use cases for push events * Feature 1392 climo_cdf.direct_prob and SeriesAnalysis (#1396) * Changed event triggering rules for documentation workflow so that it will run for pull requests even if no files in the docs directory have changed. A recent pull request passed but develop failed after it was approved because files that were referenced in the use case documentation files had been deleted as part of the PR. * feature 1273 TCGen -edeck and -shape (#1424) * Feature 966 Fix mask.poly logic (#1430) * Feature 1030 total seconds documentation (#1428) Co-authored-by: George McCabe <[email protected]> * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * get branch name in job controls job * replace logic to update input data volumes with call to dtcenter/metplus-action-data-update action * feature 1431 remove obs_prepbufr_map (#1433) * Per #675, added images and updated content * added statsmodel package to metplotpy docker conda environment * Added fcst_file_type to two wrappers (#1437) Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: George McCabe <[email protected]> * added release notes for tasks that have already been completed for beta6 * updated env var for METplotpy base to match changes made to the METplotpy repository * Updated METplotpy base env var to preemptively match changes from PR dtcenter/METplotpy#197. Once that PR is merged into develop, a test workflow will run that should fix this issue. * removed commented code that is no longer used * update the instructions for updating test input data to ensure volume_mount_directories file is updated in the upcoming version directory, i.e. v4.1, so that the Docker data volumes from the release will be created properly * removed duplicate script * updated script to copy volume_mount_directories file into new METplus version directory * added logic to skip files that are temporary copies of tar files that should not be added to new version directory * updated release guide instructions for creating METplus data directory for next development cycle * Feature 911 tc trackint (#1439) Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * added new use cases that are currently being reviewed to release notes * updated event info in contributor's guide * Feature 626 fix hovmoeller documentation (#1444) * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py replace hovmoeller_diagram.conf with UserScript_obsPrecip_obsOnly_Hovmoeller.conf * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py Removed a typo: UserScript This uses data from... Now: This uses data from... * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py * Modified the PR template to add review of the source issue metadata ci-skip-all * feature_1403_python_components_table (#1446) * creating new METplus Components Python Requirements * adding a return to test * testing use case links * testing use case links #2 * testing use case links #3 * testing use case links #4 * Filling in some use cases * fixing indentation #1403 * fixing indentation #1403 take 2 * fixing indentation #1403 take 3 * cleaning up anything after # in the links #1403 * entries thru D #1403 * entries thru H #1403 * fixing indents #1403 * fixing indents #1403 take 2 * fixing web name #1403 * entries thru L #1403 * entry matplotlib #1403 * entry matplotlib #1403 marking duplicate names * entry metcalcpy #1403 * entry metcalcpy fix indents #1403 * entry matplotlib #1403 fixing duplicate names * entry metplotpy #1403 * entry metplotpy #1403 marking duplicate names * entry metplotpy #1403 fixing * entry thru ps #1403 * double names #1403 * character fix #1403 * adding in missing links #1403 * thru pyproj #1403 * problem with line 353 #1403 * thru pyproj #1403 typos * more duplicate names #1403 * line break #1403 * loose ends #1403 * adding >= to some of the versions #1403 * adding name to the Use Cases to avoid duplicate naming issues TEST #1403 * adding name to the Use Cases to avoid duplicate naming issues thru pyproj #1403 * adding thru pyresample #1403 * fixing pyresample typo #1403 * adding thru scikit-image #1403 * fixing line breaks#1403 * web links thru scikit-learn #1403 * web links thru scipy #1403 * web links thru sklearn #1403 * Duplicate web names. temp. fix #1403 * Duplicate web names fixed #1403 * Duplicate web names fixing #3 #1403 * Duplicate web names fixing #4 #1403 * Duplicate web names fixing #5 #1403 * table end #1403 * table loose ends #1403 * Removed unnecessary comma * table loose ends after pull request #1403 * Minor modifications to METplus Component column * Changed description of pyproj * Added link for last referenced use case * Attempting to fix doc warning Co-authored-by: jprestop <[email protected]> * Feature 1318 usecase aviso (#1440) * Correct small typo. * Update pull_request_template.md * Added files for ReadTheDocs (#864) * Updated documentation links for ReadtheDocs * Update pull_request_template.md Added entry for completion date for pull request review. * Adding python embedding script to read AVISO, RTOFS, HYCOM and OSTIA data * Updated the script - working but only RMSE values match * Adding documentation for the AVISO case * Adding a conf file for the AVISO case * Updates to the codel to include model3, obs, climo3 but also has many extra print statements, netcdf output files which needs to be cleaned * Cleaning up the conf file and adding SAL1L2 to get the model_climo sdev and obs_climo sdev * Updating the file to fix the standard names and add the model3, climo3 and obs3 * Updating the file to fix the standard names and add the model3, climo3 and obs3 * Adding an image to the use case. Image Courtesy: EMC/NOAA website * added to use case list * updated case description * removed merge issues * caught second docs issue * removed other doc merge issues, turned off auto test * more merge comments sneaking in Co-authored-by: johnhg <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> * Revert "Feature 1318 usecase aviso (#1440)" (#1459) This reverts commit d3021de. * Feature 1449 v4.1.0 beta6 (#1461) * Per #1449, updated version as specified in the Release Guide * Per #1449, updated release notes * Updated release date for beta6 release Co-authored-by: Julie Prestopnik <[email protected]> * updated version number after beta release * Feature 1318 usecase aviso attempt 2 (#1473) Co-authored-by: johnhg <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: j-opatz <[email protected]> * error if no input template is set for GridDiag wrapper * added comment for new logic, ci-run-diff * updated old section name to config, ci-run-all-diff * fixed bug with replacing wildcard characters with 'all' in output file paths * Revert "fixed bug with replacing wildcard characters with 'all' in output file paths" This reverts commit c11711e. * fixed bug with replacing wildcard characters with 'all' in output file paths, ci-run-all-diff * feature 1102 -out for TCStat and StatAnalysis (#1477) * feature 1464 EnsembleStat probabilistic additions (#1479) * feature 1445 normalize in GenEnsProd (#1478) * added logic to add quotes around grid value if they are not set in conf variable value, moved error check for unset grid to init step, ci-run-all-diff * bugfix 1486 PB2NC file window variables not read (#1487) * added build arg to override MET DockerHub repository so development version tests can use dtcenter/met-dev:develop * set MET DockerHub repo based on develop or stable version of METplus * run diff logic for push events to develop and main_v* branches so we can catch if changes to external repositories caused differences in the output * do not run diff logic on main_v* and develop branch push events -- diff logic is already running when external repository triggers workflow, which is what we need to test with the diff logic * Added description for Use Cases * updated script to set up new release input data directories -- use real path for all symbolic links for consistency, added usage statement * changed logic to error if new version directory to create already exists at all -- previous logic only errors if directory is not empty * Changed all filename template tags for datetime objects (init/valid/da_init/etc.) to use %H instead of %2H. We discovered that Python 3.6.8 interprets %2H as the same as %H (correctly substituting the hour), however Python 3.8.8 interprets %2H as 2H. The number in between the % and letter is not supported by strftime so it is not valid for these datetime values. The logic to zero-pad using this notation was written in the METplus string substitution logic and only applies to lead and level template tags. ci-run-all-diff * feature 1274 TCGen use case using -shape/-genesis (#1492) Co-authored-by: Kathryn Newman <[email protected]> Co-authored-by: j-opatz <[email protected]> * feature 1490 Python Embedding for Point Observations (#1491) * update use case groupings for automated tests * removed extra comma typo, ci-run-all-diff * added missing characters in commmands * Feature 1403 python components table (#1497) Co-authored-by: Lisa Goodrich <[email protected]> Co-authored-by: Julie Prestopnik <[email protected]> * testing new statistics provided by Tara #1049 * testing new statistics #2 #1049 * New statistics from Tara #1049 * New statistics from Tara small corrections #1049 * New statistics from Tara small corrections #2 #1049 * New statistics from Tara small moving to Diagnostics table #1049 * Fixing space-time, like that's possible #1049 * Fixing spacing #1049 * feature 1471 fix current field info in output_prefix (#1500) * Feature 1503 v4.1.0 rc1 (#1504) * update version after rc1 release to reflect development towards 4.1.0 release * Corrected documentation title name * bugfix 1486 pb2nc window variables (#1507) * feature 1443 document probabilistic field settings (#1508) Co-authored-by: johnhg <[email protected]> * feature 943 document referencing current level (#1509) * feature 1510 v4.1.0 Release (#1511) * updated release guide to reflect updated process to trigger reference branch GHA workflow for an official release * update version to reflect development towards 5.0.0-beta1 release * Feature 673 MET (#1517) * set MET docker repo to dtcenter/met if forcing a specific MET tag to use since that is where the stable release versions are stored on DockerHub * pass variable to tell docker build script that an external workflow triggered the run and use -lite if so to ensure that MET docker image that was created by the triggering MET workflow is used * echo output variables from job control job so they can be reviewed in the GHA log output * minor fix for consistency in logs, ci-skip-all * Feature 1399 usecase cable (#1538) * Adding initial code to read AOML Cable Transport and RTOFS cable transport variables * Updating the conf file (no stat analysis yet) and updates to the read file * Updating the conf file to add STAT_ANALYSIS * Updates to the code- adding stats * Updated code--runs now but needs to follow User Script conventions * Changed directory name based on MET conventions * Updating the read file and adding a conf file * Added logic to count the number of directories in the RTOFS dir * Updated to run with UserScript conf * update version for official release * Adding a yaml file to specify start_date * Added the yaml file and cleaned up the conf file * Added logic to read the yaml file, cleaned up the script * Updating the confile to add output dir, input template and clean up * Removing yaml, getting vDate from conf file, adding output file * Adding the doc file for the cable use case * Adding image courtesy NOAA * Adding more documentation * Adding more info * Updated docs * Correcting the python embedding script * Updating to see if the error goes away..no clue * Fixing the doc error * modifed docs, testing * updated permissions for script * turned off always testing * Updated version number * Update use_case_groups.json * Removed erroneous file * Apply suggestions from code review * Update .github/parm/use_case_groups.json Co-authored-by: George McCabe <[email protected]> * Update docs/use_cases/model_applications/marine_and_cryosphere/UserScript_fcstRTOFS_obsAOML_calcTransport.py Co-authored-by: George McCabe <[email protected]> * Update docs/use_cases/model_applications/marine_and_cryosphere/UserScript_fcstRTOFS_obsAOML_calcTransport.py Co-authored-by: George McCabe <[email protected]> * Update docs/use_cases/model_applications/marine_and_cryosphere/UserScript_fcstRTOFS_obsAOML_calcTransport.py Co-authored-by: George McCabe <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> Co-authored-by: Lisa Goodrich <[email protected]> Co-authored-by: TaraJensen <[email protected]> Co-authored-by: George McCabe <[email protected]> Co-authored-by: johnhg <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: lisagoodrich <[email protected]> Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: Kathryn Newman <[email protected]>

* thru SPEED #1049 * thru S #1049 * thru T #1049 * thru U #1049 * thru V #1049 * thru V #1049 * thru Z #1049 * Update statistics_list.rst Tara is testing editing in UI * Update statistics_list.rst Updates through the A's * Update statistics_list.rst Cleaning up the A's * Update statistics_list.rst Standardizing MODE and MTD entries * Update statistics_list.rst Updating B's and C's * Update statistics_list.rst Testing adding TC-Stat and TCST to an entry * feature 1252 allow dictionary value for time_summary.width (#1253) * Update statistics_list.rst Clean up of a few A-Cs and then update of Ds * Update statistics_list.rst A few clean-ups and Es * Update statistics_list.rst Halfway through Fs... * feature 1213 obs_quality_inc/exc (#1260) * Feature 1203 ioda2nc (#1262) * Update statistics_list.rst A little clean up and the rest of Fs * Update statistics_list.rst G, H, I, Ks * Update statistics_list.rst L, M, Ns * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst Rs * Update statistics_list.rst S and Ts * Update statistics_list.rst The rest of the list * Update statistics_list.rst Removed Attr from Stat Type thru E * Update statistics_list.rst Remove Attr from Statistics Type through Gs * Update statistics_list.rst Remove Attr from Statistic Type through Rs * Update statistics_list.rst Remove Attr from Stat Type to the end * Update statistics_list.rst Cleaned up some Line Type typos * Update statistics_list.rst Still more Attr cleanup * Add default title for the new use case issue template. * Feature 1019 harmonic preprocessing (#1272) Co-authored-by: George McCabe <[email protected]> * Feature 1266 gen ens prod missing ensembles (#1275) * Feature 1049 statistics list (#1271) * first attempt at a table #1049 * 2nd attempt, smaller, different formatting table #1049 * trying to get rid of git error message by adding this to the index #1049 * alignment #1049 * trying to text wrap in a table #1049 * attempting to create text wrapping in tables #1049 * removing role, trying to wrap text #1049 * removing html break, trying to wrap text #1049 * adding code to wrap text #1049 * removing code to wrap text #1049 * fixing naming typo #1049 * hitting returns in table #1049 * removing typo file, removing line breaks #1049 * attempting a simple table for text wrapping #1049 * attempting to bold header row in simple table #1049 * simple table line break attempt #1049 * simple table line break attempt #2 #1049 * simple table line break attempt #3 #1049 * trying another | #1049 * 2 trying another | #1049 * 3 trying another | #1049 * trying a blank line #1049 * trying reformating grid #1049 * 2 trying reformating grid #1049 * 3 trying reformating grid #1049 * 4 trying reformating grid #1049 * adding in one more table row #1049 * adding more text #1049 * trying glossary format #1049 * 2 trying glossary format #1049 * 3 trying glossary format #1049 * 4 trying glossary format #1049 * adding glossary format 2D objects #1049 * fixing glossary format 2D objects #1049 * adding glossary items #1049 * fixing glossary format #1049 * hopefully all examples are working #1049 * changing reference to tool * removing 2D object examples * removing the unwrapped table * adding ACC into the example * fixing formatting #1049 * fixing formatting take 2 #1049 * fixing formatting take 3 #1049 * bolding and language change #1049 * fixing spacing #1049 * fixing spacing again #1049 * changing title #1049 * trying superscript #1049 * superscript glossary #1049 * removing quotes #1049 * fixing spacing #1049 * Tara has decided to go with the glossary format. Removing table example. #1049 * Adding 2D objects #1049 * 2D objects fix #1049 * 2D objects fix attempt 2 #1049 * 2D objects fix attempt 3 #1049 * 2D objects fix attempt 4 #1049 * 2D objects fix attempt 5 #1049 * 2D objects fix attempt 7 #1049 * 2D objects fix attempt 8 #1049 * ABR added #1049 * adding remaining AB items to list #1049 * adding remaining ACC_ items to list #1049 * ADLAND & AFSS entries #1049 * A entries #1049 * fixing formatting #1049 * fixing formatting #2 #1049 * removing new entries to make it run #1049 * adding a couple back in #1049 * adding a couple more back in #1049 * tool key at the bottom #1049 * spacing changes #1049 * spacing changes another try #1049 * fixing spacing #1049 * fixing spacing attempt 2 #1049 * fixing spacing attempt 3 #1049 * through AREA items #1049 * AMODEL listed twice but it's not #1049 * AMODEL listed twice but it's not #2 #1049 * AMODEL added question marks so it will run #1049 * final A entries #1049 * naming the glossary statistics in hopes of not conflicting with original glossary #1049 * removing glossary statistics space #1049 * Testing multiple glossary names * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, working on attempts to have multiple glossaries with the same term * Per #1067, removing code with attempts at mutiple glossaries * Per #1067, adding back three question marks due to duplicate term * paring down list, adding a table for review * problems with table * still trying to get it to publish * still trying to get a second line in the table * still trying to get a second line in the table 2 * testing different formatting * testing different formatting * trying to comment out the glossary inner workings * formatting again * line breaks in table #1049 * adding in some more for an example #1049 * adding new entries in through AREA #1049 * fixing line breaks #1049 * fixing line breaks #1049 #2 * fixing line breaks and warning messages #1049 * fixing line breaks #1049 * adding AREA_RATIO through ASPECT_DIFF * fixing typo #1049 * fixing typo #1049 * fixing typo #1049 take 2 * adding AXIS_ANG to BCMSE #1049 * adding spacing #1049 * adding spacing removing commas #1049 * removing comma #1049 * BOUNDARY_DIST thru BSS_SMPL #1049 * BOUNDARY_DIST splitting across 2 lines #1049 * cleaning up typos #1049 * calibration thru centriod_dist #1049 * centriod_lat thru centroid_y #1049 * fixing spacing #1049 * removing test glossary #1049 * climo_mean thru crtk_err #1049 * fixing crtk_err spacing #1049 * fixing spacing #1049 * CSI to CURVATURE_Y #1049 * CURVATURE_X & Y spacing #1049 * DEV_CAT to DURATION_DIFF #1049 * EC_VALUE to F #1049 * F_RATE TO FBS #1049 * Fixing spacing #1049 * fcst_clus thru fcst_conv_radius #1049 * removing CTOP_PRS #1049 * fixing the order of tools for FBAR and FBIAS #1049 * fixing spacing #1049 * adding grid-stat to all point-stat entries #1049 * adding fixing spacing #1049 * adding fixing spacing take 2 #1049 * adding fixing spacing take 3 #1049 * adding fixing spacing take 4 #1049 * adding fixing spacing take 5 #1049 * adding fixing spacing take 6 #1049 * adding fixing spacing take 7 #1049 * fixing spacing with a period take 7 #1049 * first attempt fcst_ #1049 * fixing typos #1049 * thru FOBAR #1049 * thru end of F #1049 * g thru h #1049 * i thru intensity #s #1049 * fixing typos #1049 * capturing example for Julie #1049 * thru k #1049 * fixing typos #1049 * thru L #1049 * thru MG #1049 * thru N_ENS #1049 * thru all N #1049 * fixing FBIAS alignment #1049 * fixing ME and MSE alignment #1049 * fixing ME alignment take 2 #1049 * thru OBS_E #1049 * fixing alignment #1049 * fixing alignment n_thresh #1049 * thru OBS_thresh #1049 * thru O #1049 * fixing OOBAR formating #1049 * thru PR_CORR #1049 * commented lines out with line total info #1049 * commented lines out with line total info take 2 #1049 * thru R #1049 * thru SPEED #1049 * thru S #1049 * thru T #1049 * thru U #1049 * thru V #1049 * thru V #1049 * thru Z #1049 * Update statistics_list.rst Tara is testing editing in UI * Update statistics_list.rst Updates through the A's * Update statistics_list.rst Cleaning up the A's * Update statistics_list.rst Standardizing MODE and MTD entries * Update statistics_list.rst Updating B's and C's * Update statistics_list.rst Testing adding TC-Stat and TCST to an entry * Update statistics_list.rst Clean up of a few A-Cs and then update of Ds * Update statistics_list.rst A few clean-ups and Es * Update statistics_list.rst Halfway through Fs... * Update statistics_list.rst A little clean up and the rest of Fs * Update statistics_list.rst G, H, I, Ks * Update statistics_list.rst L, M, Ns * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst A little clean-up and Os * Update statistics_list.rst Rs * Update statistics_list.rst S and Ts * Update statistics_list.rst The rest of the list * Update statistics_list.rst Removed Attr from Stat Type thru E * Update statistics_list.rst Remove Attr from Statistics Type through Gs * Update statistics_list.rst Remove Attr from Statistic Type through Rs * Update statistics_list.rst Remove Attr from Stat Type to the end * Update statistics_list.rst Cleaned up some Line Type typos * Update statistics_list.rst Still more Attr cleanup Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: TaraJensen <[email protected]> * creating a separate list for diagnostics, in progesss * removing diagnostics from statistics, in progess #1049 * adding diagnostics_list to TOC #1049 * separating lists * fixing typos #1049 * moving blank statistic type from statistics to diagnostics #1049 * Feature 1263 v4.1.0 beta4 (#1277) * update version to note development towards beta5 * fixing errors #1049 * updating table name #1049 * fixing typo #1049 * Feature 934 release stage doc (#1235) * Per #934 add stages of the METplus release cycle. * Per #934, adding link to descriptions of the release cycle in the User's Guide. * Per #934, made corrections * Per #934, changed Beta and Release Candidate (rc) from bold to subsubsections. * Update index.rst Co-authored-by: Julie Prestopnik <[email protected]> * putting both tables into one chapter * removing diagnostics since it's going back into the statistics chapter #1049 * Delete diagnostics_list.rst This table will be added to the statistics page as a separate table. #1049 * test. breaking out directories A-B and C-D #1049 * testing with section names #1049 * alphbetical sections for statistics #1049 * making U-Z statistics list #1049 * alpabetical breaks for diagnostics list #1049 * trying to fix errors #1049 * trying to fix errors take 2 #1049 * trying to fix errors take 3 #1049 * Removed a unnecessary space #1049 * Making all METplus Names CAPITAL LETTERS #1049 * Feature 344 met util refactor (#1292) * removed deprecated sections from config examples * minor change to METplus release guide to add a link to the PDF of the User's Guide instead of downloading it and attaching it to the release * Feature 1285 extract tiles mtd times (#1315) * Feature 896 more met config (#1322) * removed incorrect search keyword * added workflow_dispatch event so workflow can be triggered by an external repository such as MET to test to ensure that changes from that repo will break anything in METplus * added another input argument for workflow_dispatch event * added job with name that shows the event name or the repository name if triggered by an external repository such as MET * GHA: add username that triggered external event to event info job name * added required input argument for external trigger that contains the commit hash of the push event that triggered in the other repo * change event info to show commit hash instead of username that merged the PR * changed input names to match names of event in repository that triggered workflow * feature 1320 OMP_NUM_THREADS (#1338) * Feature 1183 memory documentation (#1340) Co-authored-by: George McCabe <[email protected]> * add email address of user who triggered push event to job name * Feature 1166 series analysis field info (#1353) * Feature 1116 usecase smos (#1348) Co-authored-by: Mrinal Biswas <[email protected]> * turn off new use case from every push * feature 1236 Control Members in EnsembleStat and GenEnsProd (#1357) * added optional argument to change the directory to untar new input data into so the same Dockerfile can be used to add data for other METplus components such as MET * feature 1358 v4.1.0-beta5 release (#1359) * update version for next development cycle * Feature 1216 usecase smap (#1361) * Adding a conf file for SMAP * Adding a directory to host the read file * Removing temp file * Updated the valid dates to match Todd's code * Adding documentation for SMAP case * Updates the valis dates to match Todd's code * Removing a tmp file * Typo in file name * Updating the input RTOFS to have the init time instead of the valid time ii the file name * updated file paths, tesing * updated use case descriptions, rearranged use case group testing * put new use case into its own group so that the diff logic can evaluate marine_and_cryosphere:3. The truth data for 3-4 does not exist yet so the diff fails. Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * updated marine_and_cryo grouping * Feature 1230 stratosphere metrics (#1354) * Initial checkin for Meridial Mean use case * Changed the name and directories * Added some documentation * Fixed config variables * Issue 1230 Stratospheric metrics (zonal/meridional use case) put INPUT_BASE in the user_env_vars so the user doesn't need to set the INPUT_BASE environment in the current working shell * Issue #1230 remove import of metplotpy-this isn't called * Issue #1230_stratosphere_metric Add use case to the list of use cases to be run for testing * Issue 1230 Remove the INPUT_BASE from the user_env_vars section * Issue #1230 redundant files * Issue #1230 redundant files * Issue #1230 redundant files * Issue #120 replace INPUT_BASE with INPUT_FILE_NAME * Issue #1230 remove entry for INPUT_FILE_NAME, this goes in the system.conf * Update all_use_cases.txt Copy and pasted #11 from s2s use case for the Stratosphere use case but forgot to update the index to 12. * Update use_case_groups.json added use case #12 from s2s to test stratosphere use case * Issue #1230 put INPUT_FILE_NAME back under the user_env_vars * Issue #1230 forgot to include the filename * Issue #1230 clean up config file, remove uneccessary comments, group related entries in the user_env_vars * Issue #1230 remove extraneous and incorrect path to the input_filename setting * Removed pingouin dependency * Issue #1230 another cut and paste error fixed for the Stratosphere use case * Issue #1230 type in name, obs_Only should be obsOnly * Issue #1230 added the metdatadb to the env, code imports metdatadb and may require some of these dependencies * issue #1230 removed pingouin dependency from comment to reduce any confusion * issue #1230 turn off the test for the Stratosphere metrics use case * Added use case image Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: Christina Kalb <[email protected]> * Corrected spelling of occurrence in two places * Fixed misspelling of occurrence * Found and fixed two more misspellings of occurence * Changed version specific information to by X.Y.Z moved text indicating to click save to the bottom of the list. * Feature 1374 python packages (#1378) * Added documentation about updating spreadsheet of Python requirements * Changed references to master_metplus.py to run_metplus.py * Fixed formatting of section with run_metplus.py commands * Updated text for updating the spreadsheet * Removed a newly added section that wasn't needed Co-authored-by: Julie Prestopnik <[email protected]> * feature 1368 PCPCombine use zero accum (#1381) * feature 1369 grid_weight_flag in EnsembleStat (#1379) * Per #1356, change how wrappers create instances of other wrappers to ensure that config settings for the created instance do not change values in METplusConfig used by the rest of the wrappers, ci-run-all-diff * Per #1356, remove config_overrides functionality in favor of using instances -- this prevents the issue where overrides for a given tool affect the global config settings. it also forces the configs to override for an instance to be put into another config section so that it will be available in the final conf, ci-run-all-diff * Per #1356, update documentation to reflect changes * Per #1356, fixed unit test to no longer use deprecated approach to overriding config variables * feature 1247 climatology settings (#1385) * feature 1356 isolate config (#1386) * Feature 675 Continuous Integration Documentation (#1409) Co-authored-by: johnhg <[email protected]> * feature 1382 Filename templates in MODEL (#1414) * Feature 1289 explicit file list (#1387) * Per PR #1387 review, fix typo * Bugfix 1421 EnsembleStat use fcst fields in ens dictionary if ens is unset (#1422) * fixed missing path change for docker file path * removed variables that are not used * Feature 1408 use case ptype (#1425) * adding imagery, config file, updating internal tests * updated lists, auto testing * corrected old reference * Update use_case_groups.json * only trigger testing workflow for pull requests that are going into develop or main_* branches * Feature 1371 blocking wr (#1426) * turn off use cases for push events * Feature 1392 climo_cdf.direct_prob and SeriesAnalysis (#1396) * Changed event triggering rules for documentation workflow so that it will run for pull requests even if no files in the docs directory have changed. A recent pull request passed but develop failed after it was approved because files that were referenced in the use case documentation files had been deleted as part of the PR. * feature 1273 TCGen -edeck and -shape (#1424) * Feature 966 Fix mask.poly logic (#1430) * Feature 1030 total seconds documentation (#1428) Co-authored-by: George McCabe <[email protected]> * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_WeatherRegime.py * Update UserScript_fcstGFS_obsERA_Blocking.py * Update UserScript_obsERA_obsOnly_Blocking.py * Update UserScript_obsERA_obsOnly_WeatherRegime.py * get branch name in job controls job * replace logic to update input data volumes with call to dtcenter/metplus-action-data-update action * feature 1431 remove obs_prepbufr_map (#1433) * Per #675, added images and updated content * added statsmodel package to metplotpy docker conda environment * Added fcst_file_type to two wrappers (#1437) Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: George McCabe <[email protected]> * added release notes for tasks that have already been completed for beta6 * updated env var for METplotpy base to match changes made to the METplotpy repository * Updated METplotpy base env var to preemptively match changes from PR dtcenter/METplotpy#197. Once that PR is merged into develop, a test workflow will run that should fix this issue. * removed commented code that is no longer used * update the instructions for updating test input data to ensure volume_mount_directories file is updated in the upcoming version directory, i.e. v4.1, so that the Docker data volumes from the release will be created properly * removed duplicate script * updated script to copy volume_mount_directories file into new METplus version directory * added logic to skip files that are temporary copies of tar files that should not be added to new version directory * updated release guide instructions for creating METplus data directory for next development cycle * Feature 911 tc trackint (#1439) Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * added new use cases that are currently being reviewed to release notes * updated event info in contributor's guide * Feature 626 fix hovmoeller documentation (#1444) * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py replace hovmoeller_diagram.conf with UserScript_obsPrecip_obsOnly_Hovmoeller.conf * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py Removed a typo: UserScript This uses data from... Now: This uses data from... * Update UserScript_obsPrecip_obsOnly_Hovmoeller.py * Modified the PR template to add review of the source issue metadata ci-skip-all * feature_1403_python_components_table (#1446) * creating new METplus Components Python Requirements * adding a return to test * testing use case links * testing use case links #2 * testing use case links #3 * testing use case links #4 * Filling in some use cases * fixing indentation #1403 * fixing indentation #1403 take 2 * fixing indentation #1403 take 3 * cleaning up anything after # in the links #1403 * entries thru D #1403 * entries thru H #1403 * fixing indents #1403 * fixing indents #1403 take 2 * fixing web name #1403 * entries thru L #1403 * entry matplotlib #1403 * entry matplotlib #1403 marking duplicate names * entry metcalcpy #1403 * entry metcalcpy fix indents #1403 * entry matplotlib #1403 fixing duplicate names * entry metplotpy #1403 * entry metplotpy #1403 marking duplicate names * entry metplotpy #1403 fixing * entry thru ps #1403 * double names #1403 * character fix #1403 * adding in missing links #1403 * thru pyproj #1403 * problem with line 353 #1403 * thru pyproj #1403 typos * more duplicate names #1403 * line break #1403 * loose ends #1403 * adding >= to some of the versions #1403 * adding name to the Use Cases to avoid duplicate naming issues TEST #1403 * adding name to the Use Cases to avoid duplicate naming issues thru pyproj #1403 * adding thru pyresample #1403 * fixing pyresample typo #1403 * adding thru scikit-image #1403 * fixing line breaks#1403 * web links thru scikit-learn #1403 * web links thru scipy #1403 * web links thru sklearn #1403 * Duplicate web names. temp. fix #1403 * Duplicate web names fixed #1403 * Duplicate web names fixing #3 #1403 * Duplicate web names fixing #4 #1403 * Duplicate web names fixing #5 #1403 * table end #1403 * table loose ends #1403 * Removed unnecessary comma * table loose ends after pull request #1403 * Minor modifications to METplus Component column * Changed description of pyproj * Added link for last referenced use case * Attempting to fix doc warning Co-authored-by: jprestop <[email protected]> * Feature 1318 usecase aviso (#1440) * Correct small typo. * Update pull_request_template.md * Added files for ReadTheDocs (#864) * Updated documentation links for ReadtheDocs * Update pull_request_template.md Added entry for completion date for pull request review. * Adding python embedding script to read AVISO, RTOFS, HYCOM and OSTIA data * Updated the script - working but only RMSE values match * Adding documentation for the AVISO case * Adding a conf file for the AVISO case * Updates to the codel to include model3, obs, climo3 but also has many extra print statements, netcdf output files which needs to be cleaned * Cleaning up the conf file and adding SAL1L2 to get the model_climo sdev and obs_climo sdev * Updating the file to fix the standard names and add the model3, climo3 and obs3 * Updating the file to fix the standard names and add the model3, climo3 and obs3 * Adding an image to the use case. Image Courtesy: EMC/NOAA website * added to use case list * updated case description * removed merge issues * caught second docs issue * removed other doc merge issues, turned off auto test * more merge comments sneaking in Co-authored-by: johnhg <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> * Revert "Feature 1318 usecase aviso (#1440)" (#1459) This reverts commit d3021de. * Feature 1449 v4.1.0 beta6 (#1461) * Per #1449, updated version as specified in the Release Guide * Per #1449, updated release notes * Updated release date for beta6 release Co-authored-by: Julie Prestopnik <[email protected]> * updated version number after beta release * Feature 1318 usecase aviso attempt 2 (#1473) Co-authored-by: johnhg <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: j-opatz <[email protected]> * error if no input template is set for GridDiag wrapper * added comment for new logic, ci-run-diff * updated old section name to config, ci-run-all-diff * fixed bug with replacing wildcard characters with 'all' in output file paths * Revert "fixed bug with replacing wildcard characters with 'all' in output file paths" This reverts commit c11711e. * fixed bug with replacing wildcard characters with 'all' in output file paths, ci-run-all-diff * feature 1102 -out for TCStat and StatAnalysis (#1477) * feature 1464 EnsembleStat probabilistic additions (#1479) * feature 1445 normalize in GenEnsProd (#1478) * added logic to add quotes around grid value if they are not set in conf variable value, moved error check for unset grid to init step, ci-run-all-diff * bugfix 1486 PB2NC file window variables not read (#1487) * added build arg to override MET DockerHub repository so development version tests can use dtcenter/met-dev:develop * set MET DockerHub repo based on develop or stable version of METplus * run diff logic for push events to develop and main_v* branches so we can catch if changes to external repositories caused differences in the output * do not run diff logic on main_v* and develop branch push events -- diff logic is already running when external repository triggers workflow, which is what we need to test with the diff logic * Added description for Use Cases * updated script to set up new release input data directories -- use real path for all symbolic links for consistency, added usage statement * changed logic to error if new version directory to create already exists at all -- previous logic only errors if directory is not empty * Changed all filename template tags for datetime objects (init/valid/da_init/etc.) to use %H instead of %2H. We discovered that Python 3.6.8 interprets %2H as the same as %H (correctly substituting the hour), however Python 3.8.8 interprets %2H as 2H. The number in between the % and letter is not supported by strftime so it is not valid for these datetime values. The logic to zero-pad using this notation was written in the METplus string substitution logic and only applies to lead and level template tags. ci-run-all-diff * feature 1274 TCGen use case using -shape/-genesis (#1492) Co-authored-by: Kathryn Newman <[email protected]> Co-authored-by: j-opatz <[email protected]> * feature 1490 Python Embedding for Point Observations (#1491) * update use case groupings for automated tests * removed extra comma typo, ci-run-all-diff * added missing characters in commmands * Feature 1403 python components table (#1497) Co-authored-by: Lisa Goodrich <[email protected]> Co-authored-by: Julie Prestopnik <[email protected]> * testing new statistics provided by Tara #1049 * testing new statistics #2 #1049 * New statistics from Tara #1049 * New statistics from Tara small corrections #1049 * New statistics from Tara small corrections #2 #1049 * New statistics from Tara small moving to Diagnostics table #1049 * Fixing space-time, like that's possible #1049 * Fixing spacing #1049 * feature 1471 fix current field info in output_prefix (#1500) * Feature 1503 v4.1.0 rc1 (#1504) * update version after rc1 release to reflect development towards 4.1.0 release * Corrected documentation title name * bugfix 1486 pb2nc window variables (#1507) * feature 1443 document probabilistic field settings (#1508) Co-authored-by: johnhg <[email protected]> * feature 943 document referencing current level (#1509) * feature 1510 v4.1.0 Release (#1511) * updated release guide to reflect updated process to trigger reference branch GHA workflow for an official release * update version to reflect development towards 5.0.0-beta1 release * Feature 673 MET (#1517) * set MET docker repo to dtcenter/met if forcing a specific MET tag to use since that is where the stable release versions are stored on DockerHub * pass variable to tell docker build script that an external workflow triggered the run and use -lite if so to ensure that MET docker image that was created by the triggering MET workflow is used * echo output variables from job control job so they can be reviewed in the GHA log output * minor fix for consistency in logs, ci-skip-all * Feature 1399 usecase cable (#1538) * Adding initial code to read AOML Cable Transport and RTOFS cable transport variables * Updating the conf file (no stat analysis yet) and updates to the read file * Updating the conf file to add STAT_ANALYSIS * Updates to the code- adding stats * Updated code--runs now but needs to follow User Script conventions * Changed directory name based on MET conventions * Updating the read file and adding a conf file * Added logic to count the number of directories in the RTOFS dir * Updated to run with UserScript conf * update version for official release * Adding a yaml file to specify start_date * Added the yaml file and cleaned up the conf file * Added logic to read the yaml file, cleaned up the script * Updating the confile to add output dir, input template and clean up * Removing yaml, getting vDate from conf file, adding output file * Adding the doc file for the cable use case * Adding image courtesy NOAA * Adding more documentation * Adding more info * Updated docs * Correcting the python embedding script * Updating to see if the error goes away..no clue * Fixing the doc error * modifed docs, testing * updated permissions for script * turned off always testing * Updated version number * Update use_case_groups.json * Removed erroneous file * Apply suggestions from code review * Update .github/parm/use_case_groups.json Co-authored-by: George McCabe <[email protected]> * Update docs/use_cases/model_applications/marine_and_cryosphere/UserScript_fcstRTOFS_obsAOML_calcTransport.py Co-authored-by: George McCabe <[email protected]> * Update docs/use_cases/model_applications/marine_and_cryosphere/UserScript_fcstRTOFS_obsAOML_calcTransport.py Co-authored-by: George McCabe <[email protected]> * Update docs/use_cases/model_applications/marine_and_cryosphere/UserScript_fcstRTOFS_obsAOML_calcTransport.py Co-authored-by: George McCabe <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: George McCabe <[email protected]> * Update UserScript_fcstGFS_obsERA_WeatherRegime.conf Fixed a title typo * Feature component links (#1518) * Feature 770 3rd person (#1564) * changing from 2nd to 3rd person and fixing some spacing #770 * missed we and corrected #770 * typos, spacing etc #770 * spelling, italics and bolding #770 * removing you changing to 3rd person #770 * changing to 3rd person #770 * 3rd person already done. Fixing formatting, typos etc. #770 * 3rd person already done. Fixing formatting #770 * changing to 3rd person, fixing typos, spacing, etc. #770 * changing to 3rd person, fixing typos, grammar and punctuation. #770 * google doc corrections. #770 * clean up #770 * removing | #770 * typo #770 * Fixing TOC and coding block #770 * fixing note #770 * one 3rd person fix. Updating bolding and italics for file names and directories #770 * Modified Python Code Analyis Tools * Modified Python Code Analyis Tools * Removed link for PROCESS_LIST. If we can't have an internal link, we do not want one in this case. * fixing space #770 * fixing conflict #2 #770 * fixing typos #770 * fixing typos #2 #770 * directory formatting #2 #770 * Changed sub process to subprocess * Modified usage of user because this text is for developers. Made other minor changes. Co-authored-by: Julie Prestopnik <[email protected]> * bugfix 1555 develop clear args properly (#1562) * Fixed bug in classification text file (#1568) Co-authored-by: Lisa Goodrich <[email protected]> Co-authored-by: TaraJensen <[email protected]> Co-authored-by: johnhg <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: lisagoodrich <[email protected]> Co-authored-by: Julie Prestopnik <[email protected]> Co-authored-by: jprestop <[email protected]> Co-authored-by: j-opatz <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: j-opatz <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Minna Win <[email protected]> Co-authored-by: bikegeek <[email protected]> Co-authored-by: Christina Kalb <[email protected]> Co-authored-by: Hank Fisher <[email protected]> Co-authored-by: Mrinal Biswas <[email protected]> Co-authored-by: Kathryn Newman <[email protected]>

georgemccabe added 10 commits November 15, 2021 14:34

Per #1266, wrote function to handle finding ensemble inputs to be use…

e7ae0d8

…d in EnsembleStat and GenEnsProd wrappers

make N_MEMBERS optional and set c_dict values to allow multiple files…

2767bef

…, not all are mandatory, and fill in missing FCST files with missing identifier in file list

add comment and trigger diffs with ci-run-all-diff

63251c8

fixed bug that will only use first non-path value instead of continui…

c711474

…ng through list

remove redundant debugging output, ci-run-all-diff

17574eb

added back debug log message because it is useful in some situations …

2d49a42

…where the list isn't printed earlier

fixed handling of ens_thresh and vld_thresh

83d246f

fixed handling of ens_thresh and vld_thresh

246884e

While investigating an answer for discussion #1270, I noticed that th…

cc0a36e

…e BOTH keyword should only be used to find field info (VAR<n>) indices if searching for FCST or OBS. I made a quick fix to prevent BOTH from being considered if looking for another type

georgemccabe added this to the METplus-4.1.0 milestone Nov 16, 2021

georgemccabe requested a review from JohnHalleyGotway November 16, 2021 18:37

georgemccabe linked an issue Nov 16, 2021 that may be closed by this pull request

Enhance GenEnsProd logic to properly handle ensembles that are missing #1266

Closed

21 tasks

fix bug introduced when a non-path is specified for an input (i.e. fo…

6049533

…r regrid_data_plane or gen_vx_mask grid info) to skip check if file exists in this case

JohnHalleyGotway requested changes Nov 16, 2021

View reviewed changes

JohnHalleyGotway self-requested a review November 16, 2021 21:17

JohnHalleyGotway approved these changes Nov 16, 2021

View reviewed changes

georgemccabe merged commit 4fbb689 into develop Nov 16, 2021

georgemccabe deleted the feature_1266_gen_ens_prod_missing branch November 16, 2021 21:39

georgemccabe mentioned this pull request Nov 16, 2021

Update Develop-ref after #1272 and #1275 #1276

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature 1266 gen ens prod missing ensembles #1275

Feature 1266 gen ens prod missing ensembles #1275

georgemccabe commented Nov 16, 2021 •

edited

Loading

JohnHalleyGotway left a comment •

edited

Loading

JohnHalleyGotway commented Nov 16, 2021

Feature 1266 gen ens prod missing ensembles #1275

Feature 1266 gen ens prod missing ensembles #1275

Conversation

georgemccabe commented Nov 16, 2021 • edited Loading

Pull Request Testing

Pull Request Checklist

JohnHalleyGotway left a comment • edited Loading

Choose a reason for hiding this comment

JohnHalleyGotway commented Nov 16, 2021

georgemccabe commented Nov 16, 2021 •

edited

Loading

JohnHalleyGotway left a comment •

edited

Loading