Update develop-ref after #1927 and #1928 #1931
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Commented out items and added text to description
… if latitude moves north to south
…to_laton Bugfix 1817 point2grid latlon to laton
…arsing Bugfix 1508 tc_gen file list parsing
…_double_type_lat_lon Bugfix 1838 point2grid supports the double type lat/lon
…called API handles 2D or 3D)
Feature 1903 sonarqube
* Per #1912, update logic of Met2dDataFile::process_data_plane(). Rather than returning bad status for a field of all bad data, print a warning message and continue processing. * Per #1912, fixing typo in error message that I ran into. Doesn't actually apply to this issue. * Per #1912, making an unrelated change here to fix up the MODE-Analysis log output. It was reporting (nul) for the number of lines being read/kept. Simplify the log messages to print the correct numbers.
* Per issue #1858: changed parse_conf_obs_qty to parse_conf_obs_qty_inc. Added new function parse_conf_obs_qty_exc. SL * Per issue #1858: changed conf_key_obs_qty to conf_key_obs_qty_inc. Added conf_key_obs_qty_exc. SL * Per issue #1858: modified vx_pd.set_obs_qty_filt() to use parse_conf_obs_qty_inc() instead of parse_conf_obs_qty(). SL * Per issue #1858: added back the original parse_conf_obs_qty() function for backward compatibility. SL * Per issue #1858: added back config_constants.h for backwards compatibility. SL * Per issue #1858: added code to read in obs_qty_inc instead of obs_qty. Also added in code for backwards compatibility to check for obs_qty (obs_quality) and use that for obs_qty_inc. SL * Changed obs_qty_filt to obs_qty_inc_filt. Added obs_qty_exc_filt, including relevant function. SL * Per issue #1858 added vx_pd.set_obs_qty_exc_filt. SL * In add_point_obs() added if block to apply obs_qty_exc_filt. SL * Per issue #1858, in add_point_obs(), cleaned up sections that apply obs_qty_inc and obs_qty_exc filters. Fixed spacing issue. SL * Per issue #1858 in process_config(), cleaned up sections that set obs_qty_inc_filt and obs_qty_exc_filt. Fixed spacing issue. SL * Per issue #1858 in process_obs_file() cleaned up code. SL * Per issue #1858: changed obs_qty_filt to obs_qty_inc_filt. Added obs_qty_exc_filt. Modified add_point_obs() to use obs_qty_exc_filt. SL * Per issue #1858: in process_config(): added code to set_obs_qty_inc_filt and set_obs_qty_exc_filt. Also added code to check if old obs_quality is being used, if so, set it to obs_qty_inc_filt. SL * Per issue #1858: in add_point_obs() modified section that applies the obs_qty_inc and obs_qty_exc filters. Combined the if-blocks into one concise if-block that applies both filters. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_inc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: For both the point_stat and ensemble_stat tests: modified tests that use OBS_QTY to use OBS_QTY_INC. Added unit test for OBS_QTY_EXC (obs_quality_exclude). SL * Per issue #1858: just added comment at top of file indicating what changed. SL * Per issue #1858: just added comment at top of file indicating what changed. SL * Per issue #1858: removed parse_conf_obs_qty. Modified parse_conf_obs_qty_inc to check for old 'obs_quality' entry. SL * Per issue #1858: in process_config(), modified section that calls set_obs_qty_inc_filt...this new function can now handle old 'obs_quality' entry as well as the new 'obs_quality_inc'. SL * Modified parse_conf_obs_qty_inc and parse_conf_obs_qty_exc. SL * Per issue #1858: re-updated parse_conf_obs_qty_exc and reverted back to the simplier version of this function. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_exc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc and added obs_quality_inc. SL * Per issue #1858: replaced obs_quality with obs_quality_inc. Added documentation for obs_quality_exc. SL * Per issue #1858: checking in latest version. Modified section that calls set_obs_qty_inc_filt...this new function can now handle old 'obs_quality' entry as well as the new 'obs_quality_inc'. SL * Per issue #1858: cleaned up some extra line spaces. SL * Per #1858, removed the deprecated obs_quality configuration entry from 3 EnsembleStat and 5 PointStat configuration files. * Per issue #1858: duplicated fcst/obs fields to run/test with both obs_quality_inc and obs_quality_exc. SL * Per issue #1858: duplicated fcst/obs fields to run/test with both obs_quality_inc and obs_quality_exc. SL * Per issue #1858. Deleted these files git repository. These were replaced with: PointStatConfig_qty_inc_exc. SL * Per issue #1858. Deleted these files from the git repository. These were replaced with: EnsembleStatConfig_qty_inc_exc. SL * Per issue #1858. Deleted these files from the git repository. These were replaced with: PointStatConfig_qty_inc_exc. SL * Per issue #1858: Modified: Don't need to run separate tests for obs_quality_inc and obs_quality_exc, these are now combined into one test via the updated config files: PointStatConfig_qty_inc_exc and EnsembleStatConfig_qty_inc_exc. SL * Per #1858, update the logic in parse_conf_obs_qty_inc() to use last_lookup_status(). See forthcoming GitHub issue comment about this. * Per #1858, tweak the warning message. * Per #1858, made the existing unit_quality_filter.xml tests a bit more concise. Call Point-Stat and Ensemble-Stat only once each time. But for each, test 3 things: all qty, obs_quality_inc, and obs_quality_exc. * Per #1858, remove the _ON from the job name since the _OFF has been included in the same configuration. Co-authored-by: Seth Linden <[email protected]> Co-authored-by: John Halley Gotway <[email protected]> Co-authored-by: John Halley Gotway <[email protected]>
* #1894 Added station ID 14008 to pb2nc_NDAS_var_all unit test * #1894 Check the existance of the valid data up to 16 levels Co-authored-by: Howard Soh <[email protected]>
* #1913 Changed MAX_PBL to 10000. Filter out if the PBL input data is invalid * #1912 Replaced is_eq to is_bad_data * #1913 Changed is_eq to is_bad_data Co-authored-by: Howard Soh <[email protected]>
…phasis prior to the METplus Governance meeting.
…ength (#1923) * #1922 Removed trailing spaces based on the string lenth than buffer length * #1922 Renamed cleanup_hdr_bfr to m_rstrip and moved to str_wrappers.cc Co-authored-by: Howard Soh <[email protected]>
* Per #1823, no actual code changes. Just making code header formatting consistent. This issue will require modifying these classes to wrap global grids. * Per #1823, more reformatting of headers for consistency. No code changes. * Per #1823, more formatting. * Per #1823, updates to the vx_util library. But note that this is just the mechanics, passing around the is_global boolean. I still need to actually update the logic to do the wrapping. I also need to update the rest of the codebase for these changes. Perhaps the grid template info should be a struct instead of several args? * Per #1823, move _isGlobal into the GridTemplate base class since that's where it's applied to the logic... which I still need to do! * Per #1823, this is a clear and obvious bugfix. The interp.vld_thresh setting was NOT getting set properly in Point-Stat and Ensemble-Stat. Need to add a call to set_interp_thresh instead of using the default >0 threshold. * Per #1823, update the GridTemplate member functions to wrap X for global grids. * Per #1823, update the interp_xy and interp_bilin functions to handle global grids where we wrap x. * Per #1823, add LatLonGrid isGlobal member and set it right when the grid is created. * Per #1823, update compute_interp() and get_interp_points() functions to handle the is_global flag. * Per #1823, remove the unused fractional_coverage_square() function rather than updating it to handle global grids. * Per #1823, update application code to make updated calls to DataPlane utility functions for point_stat, grid_stat, ensemble_stat, and shift_data_plane. * Per #1823, update PairDataPoint and PairDataEnsemble classes for global grids. * Per #1823, update regridding logic to only skip points off the grid in the x-direction for non-global grids. * Per #1823, include the interpolation method/width in the log message for skipping observations due to a bad interpolated forecast value. * Per #1823, added a positive_modulo(int, int) utility function to mimic the modulo functionality within python. This is needed to wrap negative X indices back to positive for global grids. * Per #1823, rename is_global related variables to wrap_lon instead. The point here is whether or not the longitudes span 0 to 360 and has nothing to do with the extent of the latitudes. Using is_global incorrectly implies that the latitudes span -90 to 90. * Per #1823, fix typo. * Per #1823, add a new call to regrid_data_plane that in met version 10.0.0 has a vertical stripe of missing data that is gone in this feature branch. * Per #1823, switch the regrid_data_plane call to use existing input data rather than having to add new inputs. In MET version 10.0.0, this results in a vertical line at the wrap longitude point. But feature 1823 fixes and that demonstrates that smoothing across the wrap point works well. * Per #1823, print a log message when a global grid has a longitude spacing that does not evenly divide 360. * Per #1823, working on the log message. * Per #1823, change to debug 4. * Per #1823, just whitespace * Per #1823, fix spacing. * Per #1823, correct halfheight to halfwidth.
* Per #1904, add gen_ens_prod tool. Note that this is a complete copy of the existing Ensemble-Stat tool, just renamed to Gen-Ens-Prod, including the newly added documentation section. * Per #1904, update the gen-ens-prod documentation labels, switching _ES_ to _GEP_ to get rid of RTD documentation warnings. * Per #1904, more renaming of EnsembleStat to GenEnsProd in the code and add the default configuration file. * Per #1904, add updated version of the MET flowchart for version 10.1.0. * Per #1904, add make directives for running a sample call to gen_ens_prod. * Per #1904, updates to the default Gen-Ens-Prod config file. * Per #1904, strip out the logic from Ensemble-Stat that does not apply to Gen-Ens-Prod. * Per #1904, updates to the Gen-Ens-Prod documentation. These are not complete. Continue edits in the configuration section. * Per #1904, fix command line in example to use -out instead of -outdir. * Per #1904, removing lots of uneeded code. Update to read climatology data for use in defining climo cdp threshold types. * Per #1904, add climo and climo_cdp options to ensemble_flag. * Per #1904, update gen_ens_prod to actually write climo and climo_cdp outputs. * Per #1904, add NumArray::set_const(double, int) to quickly call erase() followed by add_const(double, int). That saves code in gen_ens_prod. * Per #1904, add support for the ensemble control member. Exclude it from the spread. * Per #1904, update call to gen_ens_prod for make test and also update the unit tests. Still need more work to get climo working in unit tests. * Per #1904, update the fractional_coverage() function to handle climo mean/stdev for CDP type thresholds. * Per #1904, update Grid-Stat, Ensemlbe-Stat, and Gen-Ens-Prod to pass climo data to the fractional_coverage() function. * Per #1904, fix logic for handling climo data in the fractional_coverage() function. Also update the gen_ens_prod unit tests to call it once with/without a control member. I manually confirmed that the spread is the same in the output but the means differ. * Per #1904, add the make test script logic to unit_met_test_scripts.xml * Per #1904, complete initial version of docs for gen_ens_prod tool. Remove unused rng and tmp_dir config entries. Simplify variable names. * Per #1904, fix names to get it compiling and remove unneeded gsl include. * Per #1904, simplify variable names. * Per #1904, correct the logic for handling the control member. The ensemble sum is used to compute both the mean and the standard deviation. Since we include control member in the mean but not the standard deviation, we need to track two different versions of that sum. * Per #1904, in gen_ens_prod.cc correct calls to the smooth_field() and fractional_coverage() utility functions as their signatures where changed by recent enhancements in develop.
This was
linked to
issues
Oct 4, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Testing
This updates develop-ref to get past changes flagged in NB20211002.
The differences are caused by caused by 2 earlier PR's.
It also modifies 10 existing output files.
Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
None
Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes]
Do these changes include sufficient testing updates? [Yes]
Will this PR result in changes to the test suite? [Yes]
If yes, describe the new output and/or changes to the existing output:
Update reference version, as expected.
Please complete this pull request review by [Fill in date].
Pull Request Checklist
See the METplus Workflow for details.
Select: Reviewer(s)
Select: Organization level software support Project or Repository level development cycle Project
Select: Milestone as the version that will include these changes