Add a sum option to the time summaries computed by the point pre-processing tools

[CPKalb]

Describe the New Feature

Ascii2nc has has options for time summary such as max, mean, etc. However, for precipitation accumulations, sum is needed. Since support for the time_summary option is handled in the MET library code, adding the "sum" option there will make it available to all of the point pre-processing tools.

Whereas the min, max, and mean functions are less susceptible to missing data, the "sum" option is highly susceptible to it. Missing observation reports could produce very different results.

This can be mitigated by setting vld_thresh = 1.0 to require that all expected input data actually be present in order to produce a result. Recommend adding a sanity check to the code... if "sum" requested but vld_thresh is not set to 1.0, print a warning message to the user recommending the set vld_thresh = 1.0 and explaining the risks.

Acceptance Testing

List input data types and sources.
Describe tests required for new functionality.

Time Estimate

1 day.

Sub-Issues

Consider breaking the new feature down into sub-issues.
No sub-issues needed.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• Select engineer(s) or no engineer required
• Select scientist(s) or no scientist required

Labels

• Select component(s)
• Select priority
• Select requestor(s)

Projects and Milestone

• Select Repository and/or Organization level Project(s) or add alert: NEED PROJECT ASSIGNMENT label
• Select Milestone as the next official version or Future Versions

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdatadb, METviewer, METexpress, METcalcpy, METplotpy

New Feature Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Linked issues
  Select: Repository level development cycle Project for the next official release
  Select: Milestone as the next official version
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.

[JohnHalleyGotway]

@CPKalb, OK, I have some changes I'd like you to test out. Please test out seneca:/d1/personal/johnhg/MET/MET_development/MET-feature_2204_sum/bin/ascii2nc.

That version of ascii2nc (and the other point pre-processing tools) should support setting type = [ "sum" ]; in the time_summary dictionary.

Please let me know:

1. If it behaves the way you expect.
2. If you run a nice concise test with precip point data, please send me the sample data, config file, and command you ran. I can add it as a new unit test in MET for this functionality.
3. What logic you'd like to handle missing data...

I did NOT add any logic for this yet. Some options are...

• Do nothing and just keep the vld_thresh logic as is.
• If requesting "sum" with "vld_thresh != 1.0", print a single warning message about that recommending they set vld_thresh to 1.0.
• If requesting "sum" and the number of valid obs does not equal the number of expected obs, print a warning message each time this happens but write the sum of the data that is present.
• If requesting "sum" and the number of valid obs does not equal the number of expected obs, do NOT write a summary value (this is effectively like automatically resetting the vld_thresh to 1.0 for "sum").

What's the best choice here?