Enhance MET to support separate climatology datasets for both the forecast and observation inputs

[JohnHalleyGotway]

Describe the Enhancement

The climo_mean and climo_stdev configuration file options define climatology input files for the MET tools. As of MET version 11.1.0, they are parsed from the top-level configuration file context, meaning that they can only be defined once. So when computing anomaly statistics, the same climo value is subtracted from both the forecast and observation data.

This issue is to enhance MET to support separate climatology inputs for the forecast and observation datasets, as discussed during the NOAA METplus User Telecon on June 24, 2024 and requested previously in this GitHub issue comment for MET#2308.

This work requires the following changes:

1. @j-opatz, create at least one METplus use case that demonstrates the use of separate forecast and observation climatologies. The input data for this use case will help drive development.
2. Update the MET tools to parse the existing climo_mean and climo_stdev config options separately within the forecast and observation dictionaries. Note that we'll need a METplus issue to update the wrappers accordingly to make this configurable via METplus. This impacts 5 MET tools: Point-Stat, Grid-Stat, Ensemble-Stat, Series-Analysis. Note that they are also present in Gen-Ens-Prod, but that tool does NOT process observation data.
3. Update the internal MET library to keep track of forecast and observation climatology data separately. What should happen if one is specified but not the other? What constitutes a warning? What constitutes an error?
4. Update the MPR and ORANK output line types. They currently contain columns for CLIMO_MEAN, CLIMO_STDEV, and CLIMO_CDF (only in MPR). How should these change? Assume that we need extra columns to report the forecast climo values?
5. Update Stat-Analysis to parse the updated MPR and ORANK line types and use that climo data in its computations.

Additional questions:

• As described here, the climo_stdev data is used to define climatology-based thresholds (e.g. >CDP75) and to apply binned climatology logic. Do we need the flexibility to use the forecast climatology data in a similar way? Or do we really only need to define the forecast climatology mean so that it can be subtracted from forecast value when computing anomaly statistics?

Stages of development:

• Update the MPR and ORANK output line types to just write duplicate existing climo values, update the header tables and MPR/ORANK documentation tables.
• Update Stat-Analysis to parse the updated line types and just set the existing climo values to the same thing twice.
• Update PairDataPoint class to process separate fcst and obs climo data.
• Restructure VxPairDataPoint and VxPairDataEnsemble to inherit from a common VxPairBase class (just like PairDataPoint and PairDataEnsemble inherit from PairBase).

Time Estimate

7 days?

Sub-Issues

Consider breaking the enhancement down into sub-issues.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

2700044

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Review default alert labels
• Select component(s)
• Select priority
• Select requestor(s)

Milestone and Projects

• Select Milestone as a MET-X.Y.Z version, Consider for Next Release, or Backlog of Development Ideas
• For a MET-X.Y.Z version, select the MET-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy
• @j-opatz, please add a new METplus use case to demonstrate this functionality.
• Enhancement: Support separate climatology datasets for both the forecast and observation inputs METplus#2622 to update the METplus wrappers to handle the config file changes.
• Enhance METdataio schema to handle new/modified MPR and ORANK columns introduced for MET 12.0.0 beta6 METdataio#320 to update the schema to handle the MPR and ORANK line type changes.

Enhancement Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding Source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Milestone as the next official version
  Select: MET-X.Y.Z Development project for development toward the next official release
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[JohnHalleyGotway]

I was torn as to whether we should introduce a single new fcst_climo_mean config entry… or just use the existing climo_mean and climo_stdev ones and change the context from which we parse them. Right now they’re both parsed from the top-level config context but we could switch to parsing them from inside the fcst and obs dictionaries instead… very similar to what we do when parsing fcst.field and obs.field for example.

Since @GwenChen-NOAA confirmed via email on June 24, 2024 that she’ll want both a forecast climo_mean and climo_stdev, I think we should just do that latter. Although it isn’t immediately clear to me if/how we’d actually use the forecast climo_stdev data yet. But hopefully @GwenChen-NOAA and @j-opatz will figure that out.

[JohnHalleyGotway]

On July 5, 2024, @JohnHalleyGotway met with @TaraJensen to clarify the desired extent of these changes. @TaraJensen confirmed that we want to make the support for forecast climo data as thorough and complete as our existing support for "obs" climo data.

So that includes adding new MPR and ORANK mean/stdev/cdf columns for this data, and also adding support for "FCDP" and "OCDP" threshold type for forecast/observation climo distribution percentile thresholds. Note that use of existing "CDP" threshold types should still be supported for backward compatibility but should be interpreted as being set as "OCDP" thresholds.

[JohnHalleyGotway]

Reopening this issue because a downstream METplus Use Case unexpectedly failed in this Testing Workflow Run.

In the Use Case Tests (s2s:4) job, series_analysis errored out with:

ERROR  : check_climo_n_vx() -> The number of climatology mean fields in "climo_mean.field" must be zero or match the number (29) in "fcst.field".

This is caused by the change to series_analysis on this line:

   // Check for consistent number of climatology fields
   check_climo_n_vx(fdict, n_fcst);
   check_climo_n_vx(odict, n_obs);

This use case provides 1 forecast field and 29 observation fields. Only one field is provided for the climo mean and standard deviation. Prior to MET#2924, we were only checking that climo and forecast field are consistent:

   // Check climatology fields
   check_climo_n_vx(&conf, n_fcst);

Need to update the logic and log messages in check_climo_n_vx(). Supplying 0 fields, 1 field, or a list of fields matching number of corresponding input field, should all work.

Note that @georgemccabe recommend adding a unit test to cover this configuration wrinkle.

[j-opatz]

The Github issue for a use case demonstrating this new capability is being tracked here: dtcenter/METplus#2723