Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature #2395 wdir #2820

Merged
merged 9 commits into from
Feb 14, 2024
Merged

Feature #2395 wdir #2820

merged 9 commits into from
Feb 14, 2024

Conversation

JohnHalleyGotway
Copy link
Collaborator

@JohnHalleyGotway JohnHalleyGotway commented Feb 13, 2024

Expected Differences

  • Do these changes introduce new tools, command line arguments, or configuration file options? [No]

    If yes, please describe:

  • Do these changes modify the structure of existing or add new output data types (e.g. statistic line types or NetCDF variables)? [Yes]

    If yes, please describe:

  • Adds 3 new VL1L2 columns:

    • DIR_ME, DIR_MAE, and DIR_MSE
  • Adds 3 new VAL1L2 columns:

    • DIRA_ME, DIRA_MAE, DIRA_MSE
  • Adds 12 new VCNT columns:

    • DIR_ME, DIR_MAE, DIR_MSE, and DIR_RMSE plus _BCL and _BCU columns to store the bootstrap CI's.

Note however that the VCNT line type includes columns for bootstrap CI's, but MET does not actually compute them yet. Enhancing it to do so is requested in MET #963.

Pull Request Testing

  • Describe testing already performed for these changes:

    Manually ran make test and inspected the content of the new columns.

  • Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:

  • Meet to review the following details:

    • Review the code changes, paying particular attention to...
      • the number and names of the new output columns.
      • the computation of anomalous wind direction diffs
      • the handling of "missing" data when the direction is undefined for zero vectors. When zero vectors are encountered, the following type of warning message is written:
WARNING: VL1L2Info::compute_stats() -> Skipping 249 of 934 vector pairs for which the direction difference is undefined.
WARNING: Set the "wind_thresh" and "wind_logic" configuration options to exclude zero vectors.
  • Observe this PR testing workflow that ran without error but produced many, many diffs. Download/inspect the diffs to see that they're confined to VL1L2/VAL1L2/VCNT lines.

  • Review the description and equations added to the end of Appendix G for accuracy and clarity.

  • Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]
    Yes, this includes updates to the Point-Stat chapter and Appendix G.

  • Do these changes include sufficient testing updates? [Yes]
    New output is produced for many existing unit tests.

  • Will this PR result in changes to the test suite? [Yes]

    If yes, describe the new output and/or changes to the existing output:

    A total of 84 ascii output files are modified:

    • 48 files with a .stat suffix.
    • 13 files with a _vl1l2.txt suffix.
    • 9 files with a _val1l2.txt suffix.
    • 14 files with a _vcnt.txt suffix.
  • Please complete this pull request review by [Fri 2/16/24].

Pull Request Checklist

See the METplus Workflow for details.

  • Review the source issue metadata (required labels, projects, and milestone).
  • Complete the PR definition above.
  • Ensure the PR title matches the feature or bugfix branch name.
  • Define the PR metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Milestone as the version that will include these changes
    Select: Coordinated METplus-X.Y Support project for bugfix releases or MET-X.Y.Z Development project for official releases
  • After submitting the PR, select the ⚙️ icon in the Development section of the right hand sidebar. Search for the issue that this PR will close and select it, if it is not already selected.
  • After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
  • Close the linked issue and delete your feature or bugfix branch from GitHub.

@JohnHalleyGotway JohnHalleyGotway added this to the MET 12.0.0 milestone Feb 13, 2024
@JohnHalleyGotway JohnHalleyGotway linked an issue Feb 13, 2024 that may be closed by this pull request
24 tasks
@JohnHalleyGotway JohnHalleyGotway marked this pull request as ready for review February 14, 2024 16:13
@JohnHalleyGotway
Copy link
Collaborator Author

@ericgilleland and @JohnHalleyGotway met on Feb 14, 2024 to discuss this PR. Some questions:

  1. Does it make sense to add output to the VAL1L2 line type? MET Add new wind direction verification statistics for RMSE, Bias, and MAE #2395 requests additions to the VL1L2 and VCNT line types. I added VAL1L2 for consistency, but the question is whether or not wind direction anomaly statistics are useful to compute scientifically.
  2. Per @ericgilleland, it may be possible to define parametric confidence intervals, using the von Mises distribution, for these wind direction statistics. That would require the addition of _NCL and _NCU columns in the VCNT line type. But that remains an open question. Recommend delaying for now but consider for the future.

Copy link
Contributor

@ericgilleland ericgilleland left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🏁 Done
Development

Successfully merging this pull request may close these issues.

Add new wind direction verification statistics for RMSE, Bias, and MAE
2 participants