Skip to content

Commit

Permalink
Update develop-ref after #1633 (#1635)
Browse files Browse the repository at this point in the history
* Getting rid of compiler warnings in PB2NC by replacing several instances of the NULL pointer with the nul character (\0) instead.

* Fix typo in config_options.rst.

* Feature 1408 var_name_for_grib_code (#1617)

* #1408 Added get_var_id

* #1408 Check variable name in the configuration to use the variable name instewad of grib code

* #1408 Added point2grid_ascii2nc_surfrad_DW_PSP_by_name

* Feature 1580 2d time (#1616)

* #1580 Added get_grid_from_lat_lon_vars

* #1580 Added get_grid_from_lat_lon_vars and support 2D time variable

* #1580 Support int type variable without scale_factor and add_offset attributes

* #1580 Support 2D time variable. Implemented filtering by valid_time

* #1580 Bug fix: read time with dimension 0

* #1580 Support time variable with no dimension

* #1580 Initial release

* #1580 Added point2grid_2D_time

* #1580 Check project attribute for GOES

* #1580 Changed NULL to 0 to avoid co,pilation warning

* #1580 Added point2grid_2D_time

* #1580 Added "point2grid configuration file" section

* #1580 Changed to_grid for point2grid_NCCF_UK & point2grid_2D_time

Co-authored-by: Howard Soh <[email protected]>
Co-authored-by: John Halley Gotway <[email protected]>

* feature 1580 nccf (#1619)

* #1580 Correct the precision at _apply_scale_factor

* #1580 Added unit test plot_data_plane_NCCF_time

* #1580 Changed argument type to double at _apply_scale_factor(double)

* Bugfix 1618 develop pb2nc (#1623)

Co-authored-by: Howard Soh <[email protected]>

* Feature 1624 OBS_COMMAND (#1625)

* Per #1627, add grid_data.regrid config option for PlotPointObs and update the tool to do the requested regridding. Still need to update the docs.

* Per #1627, update docs about grid_data.regrid config option for PlotPointObs.

* Per #1627, add another call to plot_point_obs to exercise the new regrid functionality.

* Feature 1624 obs_command second try (#1629)

* Per #1624, define OBS_COMMAND.

* Per #1624, unset the test-specific environment variables after completing the run.

* Per #1624, after PR #1625 merged these changes into develop, they caused 2 unexpected diffs in the NB output. These were caused by enviornment variables being unset after each test. Updating unit_netcdf.xml and unit_point2grid.xml to define more test-specific environment variables to reproduce previous NB output.

* Organizing NB climatology and point2grid output files into the appopriate directories rather than having them at the top-level directory.

* Update pull_request_template.md

* Update the point2grid unit tests to write their temp files to the point2grid subdirectory instead of the top-level test output directory.

* Update appendixC.rst

Split the definition of H_RATE and POD

* Feature 1626 tc_gen (#1633)

* Per #1448, many changes for TC-Gen. Replace the oper_genesis dictionary with the oper_technique string. Add genesis_init_diff config entry. Update config_constants.h accordingly and the tc_gen_conf_info.h/.cc to parse the updated config entries.

* Per #1448, large overhaul of the tc_gen matching logic. This work is not yet complete. Still need to compute categorical MISSES but the current version does compile.

* Per #1448, add GenesisInfoArray::has_storm_id() function and remove the unused set_dland() function.

* Per #1448, more updates. Define the best genesis events while parsing the best tracks. We need to know the best genesis events in order to count up the forecast misses.

* Per #1448, lots more changes for tc_gen. Create a PairDataGenesis class to store genesis pairs. This will be needed to write a matched pair line type.

* Per #1448, minor tweaks to log messages.

* Per #1448, update PairDataGenesis class to store the BEST track Storm ID since the forecast genesis do not have meaningful Storm ID's.

* Per #1448, in GenesisInfoArray::add(), do NOT store multiple genesis events for the same storm, but do print a useful Debug(3) log message about it.

* Per #1448, update PairDataGenesis::has_case() logic to check the storm id and initialization time but NOT require an exact forecast hour match.

* Per #1448, update the tc_gen log messages to more concisely and consistently report the storm id.

* Per #1448, update the PairDataGenesis logic a bit to have all the misses and hits in chronological order.

* Per #1448, add genesis_init_diff entry.

* Per #1448, set the default genesis_init_diff entry to 48 hours since that's what Dan H used in his examples.

* Per #1448, work on comments and log messages.

* Per #1448, reimplement TrackInfoArray as a vector instead of managing the memory myself. This makes the implmentation of TrackInfoArray::erase_storm_id() very easy. Replace n_tracks() function with n() in several places.

* Per #1448, add valid_freq and basin_file config entries. Also rename load_dland.h/.cc to load_tc_data.h/.cc and add code to read the basin file.

* Per #1448, add GenesisInfoArray::erase_storm_id().

* Per #1448, update tc_gen code to handle new config options.

* Per #1448, had my units wrong. Was processing seconds when I thought it was hours!

* Per #1448, making test TC-Gen config file consistent with the default.

* Per #1448, also track the obs valid times.

* Per #1448, switch from tech1/tech2 to dev/ops methods. Update log messages and add lots of details to the tc_gen documentation.

* Per #1430, in tc_gen enable dev_method_flag, ops_method_flag, ci_alpha, and output_flag to be specified separately for each filter. Also add nc_pairs_flag and genesis_track_points_window config options. Add config constants entries for these options and update tc_gen to handle all of these changes.

* Per #1430, consolidate the parse_grid_mask() code a bit to avoid redundancy.:

* Per #1430, just cleaning up some messy comments.

* Per #1430, adding hooks for writing NetCDF output file.

* Per #1430, update DataPlane::set_size() function to take a 3rd argument to specify how the DataPlane should be initialized.

* Per #1430, update the nc_pairs_flag options and update the code to parse them.

* Per #1430, update the TrackInfo class to track and report the min/max warm core information.

* Per #1430, current state of development. Still a work in progress. I'm getting runtime segfaults when testing and I still need to NOT overcount the BEST track hits.

* Per #1430, committing changes described by #1430 (comment)

* Per #1430, forgot to rename genesis_match_window to genesis_hit_window as it is in the code.

* Per #1430, chaning GenesisInfo to just inherit directly from TrackInfo. Frankly, I should have thought of this a LONG time ago.

* Per #1430, change the default desc setting from NA to ALL and add the best_unique_flag option.

* Per #1430, simplify the logic now that GenesisInfo is derived from TrackInfo. Also support the best_unique_flag config option.

* Per #1430, instead of storing 12 individual DataPlane objects, store them in a map to make writing their output more convenient.

* Per #1430, updating documentation and comments.

* Per #1430, more doc updates.

* Per #1430, update unit test to only write NetCDF counts for the AL_BASIN and not the other filters.

* Per #1430, fix parsing logic for nc_pairs_flag = TRUE.

* Per #1430, fix bug. Check the VxOpt.NcInfo before calling write_nc(), not the top-level one.

* Per #1430, the docker build of tc_gen failed.

* Per #1430, working on DockerHub compilation.

* Per #1430, getting DockerHub build working.

* One more try.

* Per #1597, add hooks for new GENMPR stat line type.

* Per #1597, add config file option and column definitions for the GENMPR line type.

* Per #1597, finish writing the GENMPR line type.

* Per #1597, change the default output grid from a global 5 degree to global 1 degree grid.

* Per #1597, change GENMPR output columns to GEN_TDIFF and INIT_TDIFF since they're reported in HHMMSS format instead of seconds. Also, tweak the config file for the tc-gen unit test.

* Per #1597, have to add GENMPR header columns for Stat-Analysis and test scripts to handle it.

* Per #1597, update Stat-Analysis to handle the GENMPR line type.

* Per #1597, user's guide updates for the GENMPR and NetCDF output file.

* Per #1597, add AGEN_INIT and AGEN_FHR columns.

* Per #1597, add AGEN_INIT and AGEN_FHR columns.

* Per #1597, remove the AGEN_TIME and BGEN_TIME columns from the GENMPR line type and instead write the genesis times to the FCST_VALID_BEG/END and OBS_VALID_BEG/END header columns.

* Remove some unused output column name definitions. There are a remnant from very early versions of MET which included the CTP, CFP, and COP line types.

* Per #1597, update config file options to use dev_hit_radius, dev_hit_window, and opt_hit_tdiff. Also update log message to switch from 'lead' to 'forecast hour'.

* Per #1626, add met_regrid_nearest() utility function since I'm calling it twice.

* Per #1626, update the basin_global_tenth_degree.nc basin definition file to include basin name abbreviations.

* Per #1626, update load_tc_data.h/.cc to also read the basin abbreviations from the NetCDF basin file.

* Per #1626, add TC-Gen config file options for init_inc, init_exc, and basin_mask. Updated the library and application code, and updated the user's guide.

Co-authored-by: hsoh-u <[email protected]>
Co-authored-by: Howard Soh <[email protected]>
Co-authored-by: John Halley Gotway <[email protected]>
Co-authored-by: j-opatz <[email protected]>
  • Loading branch information
5 people authored Jan 24, 2021
1 parent 3fa2b3a commit 592c476
Show file tree
Hide file tree
Showing 39 changed files with 3,599 additions and 1,346 deletions.
136 changes: 108 additions & 28 deletions met/data/config/TCGenConfig_default
Original file line number Diff line number Diff line change
Expand Up @@ -13,25 +13,30 @@

////////////////////////////////////////////////////////////////////////////////
//
// Genesis event definition criteria.
// Genesis event definition criteria
//
////////////////////////////////////////////////////////////////////////////////

//
// Model initialization frequency in hours, starting at 0.
// Model initialization frequency in hours, starting at 0
//
init_freq = 6;

//
// Lead times in hours to be searched for genesis events.
// Valid hour frequency to be analyzed in hours, starting at 0
//
lead_window = {
beg = 24;
valid_freq = 6;

//
// Forecast hours to be searched for genesis events
//
fcst_hr_window = {
beg = 6;
end = 120;
}

//
// Minimum track duration for genesis event in hours.
// Minimum track duration for genesis event in hours
//
min_duration = 12;

Expand Down Expand Up @@ -60,36 +65,27 @@ best_genesis = {
}

//
// Operational track genesis event criteria. Defined as tracks reaching the
// specified intensity category, maximum wind speed threshold, and minimum
// sea-level pressure threshold. The operational track genesis time is valid
// time of the first track point where all of these criteria are met.
// Operational track technique name
//
oper_genesis = {
technique = "CARQ";
category = [ "DB", "LO", "WV" ];
vmax_thresh = NA;
mslp_thresh = NA;
}
oper_technique = "CARQ";

////////////////////////////////////////////////////////////////////////////////
//
// Track filtering options which may be specified separately in each filter
// array entry.
// Track filtering options
// May be specified separately in each filter array entry.
//
////////////////////////////////////////////////////////////////////////////////

//
// Filter is an array of dictionaries containing the track filtering options
// listed below. If empty, a single filter is defined using the top-level
// settings.
// Array of dictionaries containing the track filtering options
// If empty, a single filter is defined using the top-level settings.
//
filter = [];

//
// Description written to output DESC column
//
desc = "NA";
desc = "ALL";

//
// Forecast ATCF ID's
Expand All @@ -109,10 +105,12 @@ storm_id = [];
storm_name = [];

//
// Forecast and operational initialization time window
// Forecast and operational initialization times to include or exclude
//
init_beg = "";
init_end = "";
init_inc = [];
init_exc = [];

//
// Forecast, BEST, and operational valid time window
Expand All @@ -135,27 +133,63 @@ lead = [];
//
vx_mask = "";

//
// Spatial masking of hurricane basin names from the basin_file
//
basin_mask = [];

//
// Distance to land threshold
//
dland_thresh = NA;

////////////////////////////////////////////////////////////////////////////////
//
// Genesis matching time window, in hours relative to the forecast genesis time
// Matching and scoring options
// May be specified separately in each filter array entry.
//
genesis_window = {
////////////////////////////////////////////////////////////////////////////////

//
// Radius in km to search for a matching genesis event
//
genesis_match_radius = 500;

//
// Radius in km for a development scoring method hit
//
dev_hit_radius = 500;

//
// Time window in hours for a development scoring method hit
//
dev_hit_window = {
beg = -24;
end = 24;
}

//
// Genesis matching search radius in km.
// Maximum Best track genesis minus model initialization time difference for an
// operational scoring method hit
//
ops_hit_tdiff = 48;

//
// Discard genesis forecasts for initializations at or after the matching
// BEST track genesis time
//
genesis_radius = 300;
discard_init_post_genesis_flag = TRUE;

//
// Scoring methods to be applied
//
dev_method_flag = TRUE;
ops_method_flag = TRUE;

////////////////////////////////////////////////////////////////////////////////
//
// Global settings.
// Output options
// May be specified separately in each filter array entry.
//
////////////////////////////////////////////////////////////////////////////////

Expand All @@ -171,14 +205,60 @@ output_flag = {
fho = NONE;
ctc = BOTH;
cts = BOTH;
genmpr = NONE;
}

//
// NetCDF genesis pair counts
//
nc_pairs_flag = {
latlon = TRUE;
fcst_genesis = TRUE;
fcst_tracks = TRUE;
fcst_fy_oy = TRUE;
fcst_fy_on = TRUE;
best_genesis = TRUE;
best_tracks = TRUE;
best_fy_oy = TRUE;
best_fn_oy = TRUE;
}

//
// Specify which track points should be counted by thresholding the track point
// valid time minus genesis time difference.
//
valid_minus_genesis_diff_thresh = NA;

//
// Count unique BEST track genesis event locations (TRUE) versus counting the
// location for all pairs (FALSE).
//
best_unique_flag = TRUE;

////////////////////////////////////////////////////////////////////////////////
//
// Global settings
// May only be specified once.
//
////////////////////////////////////////////////////////////////////////////////

//
// Specify the NetCDF output of the gen_dland tool containing a gridded
// representation of the minimum distance to land.
//
dland_file = "MET_BASE/tc_data/dland_global_tenth_degree.nc";

//
// Specify the NetCDF file containing a gridded representation of the
// global basins.
//
basin_file = "MET_BASE/tc_data/basin_global_tenth_degree.nc";

//
// NetCDF genesis pairs grid
//
nc_pairs_grid = "G003";

//
// Indicate a version number for the contents of this configuration file.
// The value should generally not be modified.
Expand Down
1 change: 1 addition & 0 deletions met/data/table_files/met_header_columns_V10.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ V10.0 : STAT : SSVAR : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID
V10.0 : STAT : VAL1L2 : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL UFABAR VFABAR UOABAR VOABAR UVFOABAR UVFFABAR UVOOABAR
V10.0 : STAT : VL1L2 : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL UFBAR VFBAR UOBAR VOBAR UVFOBAR UVFFBAR UVOOBAR F_SPEED_BAR O_SPEED_BAR
V10.0 : STAT : VCNT : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL FBAR FBAR_BCL FBAR_BCU OBAR OBAR_BCL OBAR_BCU FS_RMS FS_RMS_BCL FS_RMS_BCU OS_RMS OS_RMS_BCL OS_RMS_BCU MSVE MSVE_BCL MSVE_BCU RMSVE RMSVE_BCL RMSVE_BCU FSTDEV FSTDEV_BCL FSTDEV_BCU OSTDEV OSTDEV_BCL OSTDEV_BCU FDIR FDIR_BCL FDIR_BCU ODIR ODIR_BCL ODIR_BCU FBAR_SPEED FBAR_SPEED_BCL FBAR_SPEED_BCU OBAR_SPEED OBAR_SPEED_BCL OBAR_SPEED_BCU VDIFF_SPEED VDIFF_SPEED_BCL VDIFF_SPEED_BCU VDIFF_DIR VDIFF_DIR_BCL VDIFF_DIR_BCU SPEED_ERR SPEED_ERR_BCL SPEED_ERR_BCU SPEED_ABSERR SPEED_ABSERR_BCL SPEED_ABSERR_BCU DIR_ERR DIR_ERR_BCL DIR_ERR_BCU DIR_ABSERR DIR_ABSERR_BCL DIR_ABSERR_BCU
V10.0 : STAT : GENMPR : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL INDEX STORM_ID AGEN_INIT AGEN_FHR AGEN_LAT AGEN_LON AGEN_DLAND BGEN_LAT BGEN_LON BGEN_DLAND GEN_DIST GEN_TDIFF INIT_TDIFF DEV_CAT OPS_CAT

V10.0 : MODE : OBJ : VERSION MODEL N_VALID GRID_RES DESC FCST_LEAD FCST_VALID FCST_ACCUM OBS_LEAD OBS_VALID OBS_ACCUM FCST_RAD FCST_THR OBS_RAD OBS_THR FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE OBJECT_ID OBJECT_CAT CENTROID_X CENTROID_Y CENTROID_LAT CENTROID_LON AXIS_ANG LENGTH WIDTH AREA AREA_THRESH CURVATURE CURVATURE_X CURVATURE_Y COMPLEXITY INTENSITY_10 INTENSITY_25 INTENSITY_50 INTENSITY_75 INTENSITY_90 INTENSITY_USER INTENSITY_SUM CENTROID_DIST BOUNDARY_DIST CONVEX_HULL_DIST ANGLE_DIFF ASPECT_DIFF AREA_RATIO INTERSECTION_AREA UNION_AREA SYMMETRIC_DIFF INTERSECTION_OVER_AREA CURVATURE_RATIO COMPLEXITY_RATIO PERCENTILE_INTENSITY_RATIO INTEREST
V10.0 : MODE : CTS : VERSION MODEL N_VALID GRID_RES DESC FCST_LEAD FCST_VALID FCST_ACCUM OBS_LEAD OBS_VALID OBS_ACCUM FCST_RAD FCST_THR OBS_RAD OBS_THR FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE FIELD TOTAL FY_OY FY_ON FN_OY FN_ON BASER FMEAN ACC FBIAS PODY PODN POFD FAR CSI GSS HK HSS ODDS
Expand Down
Binary file modified met/data/tc_data/basin_global_tenth_degree.nc
Binary file not shown.
Loading

0 comments on commit 592c476

Please sign in to comment.