Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature 1788 ssidx #1892

Merged
merged 66 commits into from
Aug 30, 2021
Merged
Show file tree
Hide file tree
Changes from 57 commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
af43755
Per #1788, add initial definition of the SSIDX line type.
JohnHalleyGotway Jul 16, 2021
97f8c9e
Per #1788, add a STAT-Analysis config file for computing the CBS Scor…
JohnHalleyGotway Jul 16, 2021
fde1bb8
Per #1788, add hooks for computing -job cbs_score.
JohnHalleyGotway Jul 16, 2021
b2b3d08
Per #1788, since we're adding a new SSIDX line type, the columns for …
JohnHalleyGotway Jul 19, 2021
128c41e
Per #1788, rename cbs_score to cbs_index per UK Met Office instructio…
JohnHalleyGotway Jul 22, 2021
49d5aed
Merge branch 'develop' into feature_1788_SSIDX
JohnHalleyGotway Jul 23, 2021
3084507
Per #1788 rename STATAnalysisConfig_CBS_Score to STATAnalysisConfig_C…
JohnHalleyGotway Jul 23, 2021
df77bae
Per #1788, update Makefile to handle name change.
JohnHalleyGotway Jul 23, 2021
5a70dab
Merge branch 'develop' into feature_1788_SSIDX
JohnHalleyGotway Aug 2, 2021
2fc2880
Per #1788, add support for SSIDX stat line type.
JohnHalleyGotway Aug 2, 2021
9e59ed5
Per #1788, define 4 output SSIDX columns.
JohnHalleyGotway Aug 2, 2021
2b50bd5
Per #1788, define an SSIndexInfo struct for stashing/writing SS Index…
JohnHalleyGotway Aug 2, 2021
3c9b2e4
Per #1788, write the SSIDX columns. However we don't need to function…
JohnHalleyGotway Aug 2, 2021
7f2f729
Per #1788, create an AggrSSIndexInfo struct to keep track of unique s…
JohnHalleyGotway Aug 2, 2021
f60a2bf
Per #1788, update the stat-analysis job code to make use of the new s…
JohnHalleyGotway Aug 2, 2021
89a18d6
Merge branch 'develop' into feature_1788_SSIDX
JohnHalleyGotway Aug 20, 2021
1c3000a
Merge branch 'develop' into feature_1788_SSIDX
JohnHalleyGotway Aug 23, 2021
545c56b
Per #1788 fix typo.
JohnHalleyGotway Aug 23, 2021
3290253
Per #1788, remove GO Index defintion from the code comments. That det…
JohnHalleyGotway Aug 23, 2021
f2a17ee
Per #1788, since coding up the GO Index several CNT and CTC columns h…
JohnHalleyGotway Aug 23, 2021
d6154e5
Per #1788, update check_hdr_str() to print the name of the problemati…
JohnHalleyGotway Aug 23, 2021
f1d12b7
Per #1788, add a set(one) member function to multiple array classes. …
JohnHalleyGotway Aug 24, 2021
753a07b
Per #1788, update the compute_ss_index() function in stat_analysis to…
JohnHalleyGotway Aug 24, 2021
0a48249
Per #1788, add FCST_MODEL and REF_MODEL output columns to the SSIDX l…
JohnHalleyGotway Aug 24, 2021
0734dd8
Per #1788, add a new N_INIT column to the SSIDX line type to report t…
JohnHalleyGotway Aug 25, 2021
271d528
Per #1788, update STATAnalysisJob to explicitly set the job type dire…
JohnHalleyGotway Aug 25, 2021
d3fb644
Per #1788, major change here adding skill_score_index_job.h/.cc to ha…
JohnHalleyGotway Aug 25, 2021
c2a0796
Per #1788, update stat_analysis source code to make use of the new cl…
JohnHalleyGotway Aug 25, 2021
3ac9c67
Per #1788, write a log message listing the model initialization times…
JohnHalleyGotway Aug 25, 2021
18bea24
Per #1788, tweak log message.
JohnHalleyGotway Aug 25, 2021
b7ac565
Per #1788, compute n_term as the max lenght of the required arrays.
JohnHalleyGotway Aug 25, 2021
8bc7b45
Per #1788, drop in the ACTUAL definition of the GO index provide by t…
JohnHalleyGotway Aug 26, 2021
f717cd5
Per #1788, correct CBS Index config file. Some parameters had 40 elem…
JohnHalleyGotway Aug 26, 2021
9353062
Per #1788, do not write SSIDX output if there are no valid terms found.
JohnHalleyGotway Aug 26, 2021
0668e77
Per #1788, format the values in the skill score index log messages, r…
JohnHalleyGotway Aug 26, 2021
3b14efb
Per #1788, update the get_stat() member functions. For 0 pairs, retur…
JohnHalleyGotway Aug 26, 2021
d0ed0b4
Per #1788, working on readability of the warning and error messages.
JohnHalleyGotway Aug 26, 2021
31f3070
Per #1788, define the ss_index_name directly in the STATAnalysis conf…
JohnHalleyGotway Aug 26, 2021
75a3a49
Per #1788, store the ss_index_name configuration option in the StatJo…
JohnHalleyGotway Aug 26, 2021
41a3d05
Per #1788, change from name to ss_index_name variable to be a bit mor…
JohnHalleyGotway Aug 26, 2021
cde0be0
Per #1788, since ss_index_name is in the StatJob class we no longer n…
JohnHalleyGotway Aug 26, 2021
5354d3e
Per #1788, simply the skill score index job handling logic a bit. We …
JohnHalleyGotway Aug 26, 2021
bbfd838
Revert "Per #1788, since ss_index_name is in the StatJob class we no …
JohnHalleyGotway Aug 26, 2021
a83c355
Revert "Revert "Per #1788, since ss_index_name is in the StatJob clas…
JohnHalleyGotway Aug 26, 2021
b221070
Per #1788, I'd pruned too much code from stat_analysis.cc which cause…
JohnHalleyGotway Aug 26, 2021
eb5b147
Per #1788, switch Z0 to L0 to match the sample data for computing CBS…
JohnHalleyGotway Aug 26, 2021
22ed9c6
Per #1788, make it so that cbs_index and go_index work from config file.
JohnHalleyGotway Aug 26, 2021
bf7b1f4
Per #1788, update the documentation for the skill score index changes.
JohnHalleyGotway Aug 26, 2021
ecc6463
Per #1788, just move some variable definitions further down since the…
JohnHalleyGotway Aug 26, 2021
9b779da
Per #1788, make job output description into a numbered section.
JohnHalleyGotway Aug 26, 2021
f313660
Per #1788, little tweak.
JohnHalleyGotway Aug 26, 2021
dc0fa3e
Per #1788, update the STATAnalysisConfig files to add ss_index_name a…
JohnHalleyGotway Aug 27, 2021
5110125
Per #1788, add new unit tests to exercise recent ss_index changes and…
JohnHalleyGotway Aug 27, 2021
646948b
Merge branch 'develop' into feature_1788_SSIDX
JohnHalleyGotway Aug 27, 2021
fac383a
Update met/docs/Users_Guide/stat-analysis.rst
JohnHalleyGotway Aug 27, 2021
e486e2b
Update met/docs/Users_Guide/stat-analysis.rst
JohnHalleyGotway Aug 27, 2021
58c74b9
Update met/docs/Users_Guide/stat-analysis.rst
JohnHalleyGotway Aug 27, 2021
3f12259
Per #1788, fix typo in error message.
JohnHalleyGotway Aug 27, 2021
945040d
Merge branch 'feature_1788_SSIDX' of https://github.com/dtcenter/MET …
JohnHalleyGotway Aug 27, 2021
50d3e38
Per #1788, fix a bug for ss_index... changing 'add' to 'set'. Also se…
JohnHalleyGotway Aug 27, 2021
cb8f0c0
Per #1788, tweak the ss_index logic slightly to make it more efficien…
JohnHalleyGotway Aug 27, 2021
806f0cc
Per #1788, minor tweaks to the STAT-Analysis docs.
JohnHalleyGotway Aug 30, 2021
9b8d7dd
Per #1788, add ss_index_vld_thresh option to existing STAT-Analysis c…
JohnHalleyGotway Aug 30, 2021
736c489
Per #1788, add StatAnalysisJob ss_index_vld_thresh entry and parse it…
JohnHalleyGotway Aug 30, 2021
b532b02
Per #1788, update stat_analysis code to check ss_index_vld_thresh set…
JohnHalleyGotway Aug 30, 2021
bc5491c
Per #1788, tweak one test to exercise a non-default ss_index_vld_thre…
JohnHalleyGotway Aug 30, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions met/data/config/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ config_DATA = \
SeriesAnalysisConfig_default \
STATAnalysisConfig_default \
STATAnalysisConfig_GO_Index \
STATAnalysisConfig_CBS_Index \
TCPairsConfig_default \
TCRMWConfig_default \
RMWAnalysisConfig_default \
Expand Down
131 changes: 131 additions & 0 deletions met/data/config/STATAnalysisConfig_CBS_Index
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
////////////////////////////////////////////////////////////////////////////////
//
// STAT-Analysis configuration file used to compute the CBS Index.
//
// WARNING: This file should not be modified.
//
////////////////////////////////////////////////////////////////////////////////

//
// Filtering input STAT lines by the contents of each column
//
model = [];

fcst_lead = [ "24", "48", "72", "96", "120",
"24", "48", "72", "96", "120",
"24", "48", "72", "96", "120",
"24", "48", "72", "96", "120",
"24", "48", "72", "96", "120",
"24", "48", "72", "96", "120",
"24", "48", "72", "96", "120",
"24", "48", "72", "96", "120"
];
obs_lead = [];

fcst_valid_beg = "";
fcst_valid_end = "";
fcst_valid_inc = [];
fcst_valid_exc = [];
fcst_valid_hour = [];

obs_valid_beg = "";
obs_valid_end = "";
obs_valid_inc = [];
obs_valid_exc = [];
obs_valid_hour = [];

fcst_init_beg = "";
fcst_init_end = "";
fcst_init_inc = [];
fcst_init_exc = [];
fcst_init_hour = [];

obs_init_beg = "";
obs_init_end = "";
obs_init_inc = [];
obs_init_exc = [];
obs_init_hour = [];

fcst_var = [ "PRMSL", "PRMSL", "PRMSL", "PRMSL", "PRMSL",
"HGT", "HGT", "HGT", "HGT", "HGT",
"WIND", "WIND", "WIND", "WIND", "WIND",
"WIND", "WIND", "WIND", "WIND", "WIND",
"WIND", "WIND", "WIND", "WIND", "WIND",
"PRMSL", "PRMSL", "PRMSL", "PRMSL", "PRMSL",
"HGT", "HGT", "HGT", "HGT", "HGT",
"WIND", "WIND", "WIND", "WIND", "WIND"
];
obs_var = [];

fcst_lev = [ "L0", "L0", "L0", "L0", "L0",
"P500", "P500", "P500", "P500", "P500",
"P250", "P250", "P250", "P250", "P250",
"P850", "P850", "P850", "P850", "P850",
"P250", "P250", "P250", "P250", "P250",
"L0", "L0", "L0", "L0", "L0",
"P500", "P500", "P500", "P500", "P500",
"P250", "P250", "P250", "P250", "P250"
];
obs_lev = [];

obtype = [];

vx_mask = [ "NH", "NH", "NH", "NH", "NH",
"NH", "NH", "NH", "NH", "NH",
"NH", "NH", "NH", "NH", "NH",
"TROP", "TROP", "TROP", "TROP", "TROP",
"TROP", "TROP", "TROP", "TROP", "TROP",
"SH", "SH", "SH", "SH", "SH",
"SH", "SH", "SH", "SH", "SH",
"SH", "SH", "SH", "SH", "SH"
];

interp_mthd = [];

interp_pnts = [];

fcst_thresh = [];
obs_thresh = [];
cov_thresh = [];

alpha = [];

line_type = [ "SL1L2" ];

column = [ "RMSE" ];

weight = [ 6.4, 6.4, 6.4, 6.4, 6.4,
2.4, 2.4, 2.4, 2.4, 2.4,
2.4, 2.4, 2.4, 2.4, 2.4,
2.0, 2.0, 2.0, 2.0, 2.0,
1.2, 1.2, 1.2, 1.2, 1.2,
3.2, 3.2, 3.2, 3.2, 3.2,
1.2, 1.2, 1.2, 1.2, 1.2,
1.2, 1.2, 1.2, 1.2, 1.2
];

////////////////////////////////////////////////////////////////////////////////

//
// Confidence interval settings
//
out_alpha = 0.05;

boot = {
interval = PCTILE;
rep_prop = 1.0;
n_rep = 0;
rng = "mt19937";
seed = "";
}

////////////////////////////////////////////////////////////////////////////////

ss_index_name = "CBS_INDEX";
hss_ec_value = NA;
rank_corr_flag = FALSE;
vif_flag = FALSE;
tmp_dir = "/tmp";
version = "V10.1.0";

////////////////////////////////////////////////////////////////////////////////
77 changes: 27 additions & 50 deletions met/data/config/STATAnalysisConfig_GO_Index
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,18 @@
//
model = [];

fcst_lead = [ "12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48"
fcst_lead = [ "12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48",
"12", "24", "36", "48"
];
obs_lead = [];

Expand Down Expand Up @@ -94,46 +94,22 @@ cov_thresh = [];

alpha = [];

line_type = [ "SL1L2", "SL1L2", "SL1L2", "SL1L2",
"SL1L2", "SL1L2", "SL1L2", "SL1L2",
"SL1L2", "SL1L2", "SL1L2", "SL1L2",
"SL1L2", "SL1L2", "SL1L2", "SL1L2",
"SL1L2", "SL1L2", "SL1L2", "SL1L2",
"SL1L2", "SL1L2", "SL1L2", "SL1L2",
"SL1L2", "SL1L2", "SL1L2", "SL1L2",
"SL1L2", "SL1L2", "SL1L2", "SL1L2",
"SL1L2", "SL1L2", "SL1L2", "SL1L2",
"SL1L2", "SL1L2", "SL1L2", "SL1L2",
"SL1L2", "SL1L2", "SL1L2", "SL1L2",
"SL1L2", "SL1L2", "SL1L2", "SL1L2"
];
line_type = [ "SL1L2" ];

column = [ "RMSE", "RMSE", "RMSE", "RMSE",
"RMSE", "RMSE", "RMSE", "RMSE",
"RMSE", "RMSE", "RMSE", "RMSE",
"RMSE", "RMSE", "RMSE", "RMSE",
"RMSE", "RMSE", "RMSE", "RMSE",
"RMSE", "RMSE", "RMSE", "RMSE",
"RMSE", "RMSE", "RMSE", "RMSE",
"RMSE", "RMSE", "RMSE", "RMSE",
"RMSE", "RMSE", "RMSE", "RMSE",
"RMSE", "RMSE", "RMSE", "RMSE",
"RMSE", "RMSE", "RMSE", "RMSE",
"RMSE", "RMSE", "RMSE", "RMSE"
];
column = [ "RMSE" ];

weight = [ 4.0, 3.0, 2.0, 1.0,
4.0, 3.0, 2.0, 1.0,
4.0, 3.0, 2.0, 1.0,
8.0, 6.0, 4.0, 2.0,
8.0, 6.0, 4.0, 2.0,
8.0, 6.0, 4.0, 2.0,
8.0, 6.0, 4.0, 2.0,
8.0, 6.0, 4.0, 2.0,
4.0, 3.0, 2.0, 1.0,
8.0, 6.0, 4.0, 2.0,
4.0, 3.0, 2.0, 1.0,
8.0, 6.0, 4.0, 2.0
weight = [ 4.0, 3.0, 2.0, 1.0,
4.0, 3.0, 2.0, 1.0,
4.0, 3.0, 2.0, 1.0,
8.0, 6.0, 4.0, 2.0,
8.0, 6.0, 4.0, 2.0,
8.0, 6.0, 4.0, 2.0,
8.0, 6.0, 4.0, 2.0,
8.0, 6.0, 4.0, 2.0,
4.0, 3.0, 2.0, 1.0,
8.0, 6.0, 4.0, 2.0,
4.0, 3.0, 2.0, 1.0,
8.0, 6.0, 4.0, 2.0
];

////////////////////////////////////////////////////////////////////////////////
Expand All @@ -153,6 +129,7 @@ boot = {

////////////////////////////////////////////////////////////////////////////////

ss_index_name = "GO_INDEX";
hss_ec_value = NA;
rank_corr_flag = FALSE;
vif_flag = FALSE;
Expand Down
1 change: 1 addition & 0 deletions met/data/config/STATAnalysisConfig_default
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ wmo_fisher_stats = [ "CNT:PR_CORR", "CNT:SP_CORR",

////////////////////////////////////////////////////////////////////////////////

ss_index_name = "SS_INDEX";
hss_ec_value = NA;
rank_corr_flag = FALSE;
vif_flag = FALSE;
Expand Down
1 change: 1 addition & 0 deletions met/data/table_files/met_header_columns_V10.1.txt
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ V10.1 : STAT : VAL1L2 : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID
V10.1 : STAT : VL1L2 : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL UFBAR VFBAR UOBAR VOBAR UVFOBAR UVFFBAR UVOOBAR F_SPEED_BAR O_SPEED_BAR
V10.1 : STAT : VCNT : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL FBAR FBAR_BCL FBAR_BCU OBAR OBAR_BCL OBAR_BCU FS_RMS FS_RMS_BCL FS_RMS_BCU OS_RMS OS_RMS_BCL OS_RMS_BCU MSVE MSVE_BCL MSVE_BCU RMSVE RMSVE_BCL RMSVE_BCU FSTDEV FSTDEV_BCL FSTDEV_BCU OSTDEV OSTDEV_BCL OSTDEV_BCU FDIR FDIR_BCL FDIR_BCU ODIR ODIR_BCL ODIR_BCU FBAR_SPEED FBAR_SPEED_BCL FBAR_SPEED_BCU OBAR_SPEED OBAR_SPEED_BCL OBAR_SPEED_BCU VDIFF_SPEED VDIFF_SPEED_BCL VDIFF_SPEED_BCU VDIFF_DIR VDIFF_DIR_BCL VDIFF_DIR_BCU SPEED_ERR SPEED_ERR_BCL SPEED_ERR_BCU SPEED_ABSERR SPEED_ABSERR_BCL SPEED_ABSERR_BCU DIR_ERR DIR_ERR_BCL DIR_ERR_BCU DIR_ABSERR DIR_ABSERR_BCL DIR_ABSERR_BCU
V10.1 : STAT : GENMPR : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL INDEX STORM_ID AGEN_INIT AGEN_FHR AGEN_LAT AGEN_LON AGEN_DLAND BGEN_LAT BGEN_LON BGEN_DLAND GEN_DIST GEN_TDIFF INIT_TDIFF DEV_CAT OPS_CAT
V10.1 : STAT : SSIDX : VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE FCST_MODEL REF_MODEL N_INIT N_TERM N_VLD SS_INDEX

V10.1 : MODE : OBJ : VERSION MODEL N_VALID GRID_RES DESC FCST_LEAD FCST_VALID FCST_ACCUM OBS_LEAD OBS_VALID OBS_ACCUM FCST_RAD FCST_THR OBS_RAD OBS_THR FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE OBJECT_ID OBJECT_CAT CENTROID_X CENTROID_Y CENTROID_LAT CENTROID_LON AXIS_ANG LENGTH WIDTH AREA AREA_THRESH CURVATURE CURVATURE_X CURVATURE_Y COMPLEXITY INTENSITY_10 INTENSITY_25 INTENSITY_50 INTENSITY_75 INTENSITY_90 INTENSITY_USER INTENSITY_SUM CENTROID_DIST BOUNDARY_DIST CONVEX_HULL_DIST ANGLE_DIFF ASPECT_DIFF AREA_RATIO INTERSECTION_AREA UNION_AREA SYMMETRIC_DIFF INTERSECTION_OVER_AREA CURVATURE_RATIO COMPLEXITY_RATIO PERCENTILE_INTENSITY_RATIO INTEREST
V10.1 : MODE : CTS : VERSION MODEL N_VALID GRID_RES DESC FCST_LEAD FCST_VALID FCST_ACCUM OBS_LEAD OBS_VALID OBS_ACCUM FCST_RAD FCST_THR OBS_RAD OBS_THR FCST_VAR FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE FIELD TOTAL FY_OY FY_ON FN_OY FN_ON BASER FMEAN ACC FBIAS PODY PODN POFD FAR CSI GSS HK HSS ODDS
Expand Down
2 changes: 1 addition & 1 deletion met/docs/Users_Guide/point-stat.rst
Original file line number Diff line number Diff line change
Expand Up @@ -498,7 +498,7 @@ The first set of header columns are common to all of the output files generated

.. _table_PS_header_info_point-stat_out:

.. list-table:: Header information for each file point-stat outputs.
.. list-table:: Common STAT header columns.
:widths: auto
:header-rows: 2

Expand Down
Loading