diff --git a/met/data/config/PointStatConfig_default b/met/data/config/PointStatConfig_default index 048ff2cb4a..4cdaefaf6e 100644 --- a/met/data/config/PointStatConfig_default +++ b/met/data/config/PointStatConfig_default @@ -280,6 +280,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; // Only for HiRA. + orank = NONE; // Only for HiRA. rps = NONE; // Only for HiRA. eclv = NONE; mpr = NONE; diff --git a/met/docs/Users_Guide/grid-stat.rst b/met/docs/Users_Guide/grid-stat.rst index 264903825f..ef1a465763 100644 --- a/met/docs/Users_Guide/grid-stat.rst +++ b/met/docs/Users_Guide/grid-stat.rst @@ -388,7 +388,7 @@ The **output_flag** array controls the type of output that the Grid-Stat tool ge 10. **VAL1L2** for Vector Anomaly L1L2 Partial Sums when climatological data is supplied -11. **VCNT** for Vector Contingency Table Statistics +11. **VCNT** for Vector Continuous Statistics 12. **PCT** for Contingency Table Counts for Probabilistic forecasts diff --git a/met/docs/Users_Guide/point-stat.rst b/met/docs/Users_Guide/point-stat.rst index 162f86926c..97b93b4b40 100644 --- a/met/docs/Users_Guide/point-stat.rst +++ b/met/docs/Users_Guide/point-stat.rst @@ -121,7 +121,7 @@ The forecast value at P is chosen as the grid point inside the interpolation are HiRA framework ~~~~~~~~~~~~~~ -The Point-Stat tool has been enhanced to include the High Resolution Assessment (HiRA) verification logic (:ref:`Mittermaier, 2014 `). HiRA is analogous to neighborhood verification but for point observations. The HiRA logic interprets the forecast values surrounding each point observation as an ensemble forecast. These ensemble values are processed in two ways. First, the ensemble continuous statistics (ECNT) and the ranked probability score (RPS) line types are computed directly from the ensemble values. Second, for each categorical threshold specified, a fractional coverage value is computed as the ratio of the nearby forecast values that meet the threshold criteria. Point-Stat evaluates those fractional coverage values as if they were a probability forecast. When applying HiRA, users should enable the matched pair (MPR), probabilistic (PCT, PSTD, PJC, or PRC), continuous ensemble statistics (ECNT), or ranked probability score (RPS) line types in the **output_flag** dictionary. The number of probabilistic HiRA output lines is determined by the number of categorical forecast thresholds and HiRA neighborhood widths chosen. +The Point-Stat tool has been enhanced to include the High Resolution Assessment (HiRA) verification logic (:ref:`Mittermaier, 2014 `). HiRA is analogous to neighborhood verification but for point observations. The HiRA logic interprets the forecast values surrounding each point observation as an ensemble forecast. These ensemble values are processed in three ways. First, the ensemble continuous statistics (ECNT), the observation rank statistics (ORANK) and the ranked probability score (RPS) line types are computed directly from the ensemble values. Second, for each categorical threshold specified, a fractional coverage value is computed as the ratio of the nearby forecast values that meet the threshold criteria. Point-Stat evaluates those fractional coverage values as if they were a probability forecast. When applying HiRA, users should enable the matched pair (MPR), probabilistic (PCT, PSTD, PJC, or PRC), continuous ensemble statistics (ECNT), observation rank statistics (ORANK) or ranked probability score (RPS) line types in the **output_flag** dictionary. The number of probabilistic HiRA output lines is determined by the number of categorical forecast thresholds and HiRA neighborhood widths chosen. The HiRA framework provides a unique method for evaluating models in the neighborhood of point observations, allowing for some spatial and temporal uncertainty in the forecast and/or the observations. Additionally, the HiRA framework can be used to compare deterministic forecasts to ensemble forecasts. In MET, the neighborhood is a circle or square centered on the grid point closest to the observation location. An event is defined, then the proportion of points with events in the neighborhood is calculated. This proportion is treated as an ensemble probability, though it is likely to be uncalibrated. @@ -425,6 +425,7 @@ ________________________ pjc = BOTH; prc = BOTH; ecnt = BOTH; // Only for HiRA + orank = BOTH; // Only for HiRA rps = BOTH; // Only for HiRA eclv = BOTH; mpr = BOTH; @@ -450,9 +451,9 @@ The **output_flag** array controls the type of output that the Point-Stat tool g 9. **VL1L2** for Vector L1L2 Partial Sums -10. **VCNT** for Vector Continuous Statistics (Note that bootstrap confidence intervals are not currently calculated for this line type.) +10. **VAL1L2** for Vector Anomaly L1L2 Partial Sums when climatological data is supplied -11. **VAL1L2** for Vector Anomaly L1L2 Partial Sums when climatological data is supplied +11. **VCNT** for Vector Continuous Statistics 12. **PCT** for Contingency Table counts for Probabilistic forecasts @@ -464,13 +465,15 @@ The **output_flag** array controls the type of output that the Point-Stat tool g 16. **ECNT** for Ensemble Continuous Statistics is only computed for the HiRA methodology -17. **RPS** for Ranked Probability Score is only computed for the HiRA methodology +17. **ORANK** for Ensemble Matched Pair Information when point observations are supplied for the HiRA methodology -18. **ECLV** for Economic Cost/Loss Relative Value +18. **RPS** for Ranked Probability Score is only computed for the HiRA methodology -19. **MPR** for Matched Pair data +19. **ECLV** for Economic Cost/Loss Relative Value -Note that the first two line types are easily derived from each other. Users are free to choose which measures are most desired. The output line types are described in more detail in :numref:`point_stat-output`. +20. **MPR** for Matched Pair data + +Note that the FHO and CTC line types are easily derived from each other. Users are free to choose which measures are most desired. The output line types are described in more detail in :numref:`point_stat-output`. Note that writing out matched pair data (MPR lines) for a large number of cases is generally not recommended. The MPR lines create very large output files and are only intended for use on a small set of cases. @@ -489,9 +492,9 @@ point_stat_PREFIX_HHMMSSL_YYYYMMDD_HHMMSSV.stat where PREFIX indicates the user- The output ASCII files are named similarly: -point_stat_PREFIX_HHMMSSL_YYYYMMDD_HHMMSSV_TYPE.txt where TYPE is one of mpr, fho, ctc, cts, cnt, mctc, mcts, pct, pstd, pjc, prc, ecnt, rps, eclv, sl1l2, sal1l2, vl1l2, vcnt or val1l2 to indicate the line type it contains. +point_stat_PREFIX_HHMMSSL_YYYYMMDD_HHMMSSV_TYPE.txt where TYPE is one of mpr, fho, ctc, cts, cnt, mctc, mcts, pct, pstd, pjc, prc, ecnt, orank, rps, eclv, sl1l2, sal1l2, vl1l2, vcnt or val1l2 to indicate the line type it contains. -The first set of header columns are common to all of the output files generated by the Point-Stat tool. Tables describing the contents of the header columns and the contents of the additional columns for each line type are listed in the following tables. The ECNT line type is described in :numref:`table_ES_header_info_es_out_ECNT`. The RPS line type is described in :numref:`table_ES_header_info_es_out_RPS`. +The first set of header columns are common to all of the output files generated by the Point-Stat tool. Tables describing the contents of the header columns and the contents of the additional columns for each line type are listed in the following tables. The ECNT line type is described in :numref:`table_ES_header_info_es_out_ECNT`. The ORANK line type is described in :numref:`table_ES_header_info_es_out_ORANK`. The RPS line type is described in :numref:`table_ES_header_info_es_out_RPS`. .. _table_PS_header_info_point-stat_out: diff --git a/met/scripts/config/PointStatConfig b/met/scripts/config/PointStatConfig index 6df8e67daf..513c956ab6 100644 --- a/met/scripts/config/PointStatConfig +++ b/met/scripts/config/PointStatConfig @@ -199,6 +199,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; // Only for HiRA + orank = NONE; // Only for HiRA rps = NONE; // Only for HiRA eclv = BOTH; mpr = BOTH; diff --git a/met/src/tools/core/point_stat/point_stat.cc b/met/src/tools/core/point_stat/point_stat.cc index 9f49049fdb..36c1a3baf6 100644 --- a/met/src/tools/core/point_stat/point_stat.cc +++ b/met/src/tools/core/point_stat/point_stat.cc @@ -95,6 +95,7 @@ // 045 03/28/21 Halley Gotway Add mpr_column and mpr_thresh // filtering options. // 046 05/28/21 Halley Gotway Add MCTS HSS_EC output. +// 047 08/23/21 Seth Linden Add ORANK line type for HiRA. // //////////////////////////////////////////////////////////////////////// @@ -327,7 +328,9 @@ void setup_first_pass(const DataPlane &dp, const Grid &data_grid) { //////////////////////////////////////////////////////////////////////// void setup_txt_files() { - int i, max_col, max_prob_col, max_mctc_col, n_prob, n_cat, n_eclv; + int i, j; + int max_col, max_prob_col, max_mctc_col, max_orank_col; + int n_prob, n_cat, n_eclv, n_ens; ConcatString base_name; // Create output file names for the stat file and optional text files @@ -340,23 +343,20 @@ void setup_txt_files() { ///////////////////////////////////////////////////////////////////// // Get the maximum number of data columns - n_prob = conf_info.get_max_n_fprob_thresh(); + n_prob = max(conf_info.get_max_n_fprob_thresh(), + conf_info.get_max_n_hira_prob()); n_cat = conf_info.get_max_n_cat_thresh() + 1; n_eclv = conf_info.get_max_n_eclv_points(); + n_ens = conf_info.get_max_n_hira_ens(); - // Check for HiRA output - for(i=0; i max_stat_col ? max_prob_col : max_stat_col ); - max_col = ( max_mctc_col > max_col ? max_mctc_col : max_col ); + max_col = (max_prob_col > max_stat_col ? max_prob_col : max_stat_col); + max_col = (max_mctc_col > max_col ? max_mctc_col : max_col); + max_col = (max_orank_col > max_col ? max_orank_col : max_col); // Add the header columns max_col += n_header_columns + 1; @@ -429,6 +429,10 @@ void setup_txt_files() { max_col = get_n_eclv_columns(n_eclv) + n_header_columns + 1; break; + case(i_orank): + max_col = get_n_orank_columns(n_ens) + n_header_columns + 1; + break; + default: max_col = n_txt_columns[i] + n_header_columns + 1; break; @@ -465,6 +469,10 @@ void setup_txt_files() { write_eclv_header_row(1, n_eclv, txt_at[i], 0, 0); break; + case(i_orank): + write_orank_header_row(1, n_ens, txt_at[i], 0, 0); + break; + default: write_header_row(txt_columns[i], n_txt_columns[i], 1, txt_at[i], 0, 0); @@ -1749,6 +1757,22 @@ void do_hira_ens(int i_vx, const PairDataPoint *pd_ptr) { txt_at[i_ecnt], i_txt_row[i_ecnt]); } // end if ECNT + // Write out the ORANK line + if(conf_info.vx_opt[i_vx].output_flag[i_orank] != STATOutputType_None) { + + // Compute ensemble statistics + hira_pd.compute_pair_vals(rng_ptr); + + write_orank_row(shc, &hira_pd, + conf_info.vx_opt[i_vx].output_flag[i_orank], + stat_at, i_stat_row, + txt_at[i_orank], i_txt_row[i_orank]); + + // Reset the observation valid time + shc.set_obs_valid_beg(conf_info.vx_opt[i_vx].vx_pd.beg_ut); + shc.set_obs_valid_end(conf_info.vx_opt[i_vx].vx_pd.end_ut); + } // end if ORANK + // Write out the RPS line if(conf_info.vx_opt[i_vx].output_flag[i_rps] != STATOutputType_None) { diff --git a/met/src/tools/core/point_stat/point_stat.h b/met/src/tools/core/point_stat/point_stat.h index dc232a52de..9b2b823d2a 100644 --- a/met/src/tools/core/point_stat/point_stat.h +++ b/met/src/tools/core/point_stat/point_stat.h @@ -70,8 +70,8 @@ static const char **txt_columns[n_txt] = { sl1l2_columns, sal1l2_columns, vl1l2_columns, val1l2_columns, pct_columns, pstd_columns, pjc_columns, prc_columns, ecnt_columns, - rps_columns, eclv_columns, mpr_columns, - vcnt_columns + orank_columns, rps_columns, eclv_columns, + mpr_columns, vcnt_columns }; // Length of header columns @@ -81,8 +81,8 @@ static const int n_txt_columns[n_txt] = { n_sl1l2_columns, n_sal1l2_columns, n_vl1l2_columns, n_val1l2_columns, n_pct_columns, n_pstd_columns, n_pjc_columns, n_prc_columns, n_ecnt_columns, - n_rps_columns, n_eclv_columns, n_mpr_columns, - n_vcnt_columns + n_orank_columns, n_rps_columns, n_eclv_columns, + n_mpr_columns, n_vcnt_columns }; // Text file abbreviations @@ -92,8 +92,8 @@ static const char *txt_file_abbr[n_txt] = { "sl1l2", "sal1l2", "vl1l2", "val1l2", "pct", "pstd", "pjc", "prc", "ecnt", - "rps", "eclv", "mpr", - "vcnt" + "orank", "rps", "eclv", + "mpr", "vcnt" }; //////////////////////////////////////////////////////////////////////// diff --git a/met/src/tools/core/point_stat/point_stat_conf_info.cc b/met/src/tools/core/point_stat/point_stat_conf_info.cc index e56352d4ad..dd0d13dee1 100644 --- a/met/src/tools/core/point_stat/point_stat_conf_info.cc +++ b/met/src/tools/core/point_stat/point_stat_conf_info.cc @@ -523,6 +523,26 @@ int PointStatConfInfo::get_max_n_eclv_points() const { //////////////////////////////////////////////////////////////////////// +int PointStatConfInfo::get_max_n_hira_ens() const { + int i, n; + + for(i=0,n=0; i ECLV lines = // Message Types * Masks * Interpolations * Thresholds * @@ -1282,3 +1316,16 @@ int PointStatVxOpt::get_n_oprob_thresh() const { } //////////////////////////////////////////////////////////////////////// + +int PointStatVxOpt::get_n_hira_ens() const { + int n = (hira_info.flag ? hira_info.width.max() : 0); + return(n*n); +} + +//////////////////////////////////////////////////////////////////////// + +int PointStatVxOpt::get_n_hira_prob() const { + return(hira_info.flag ? hira_info.cov_ta.n() : 0); +} + +//////////////////////////////////////////////////////////////////////// diff --git a/met/src/tools/core/point_stat/point_stat_conf_info.h b/met/src/tools/core/point_stat/point_stat_conf_info.h index f9d0d9fe0f..5bc964069d 100644 --- a/met/src/tools/core/point_stat/point_stat_conf_info.h +++ b/met/src/tools/core/point_stat/point_stat_conf_info.h @@ -46,12 +46,13 @@ static const int i_pjc = 12; static const int i_prc = 13; static const int i_ecnt = 14; -static const int i_rps = 15; -static const int i_eclv = 16; -static const int i_mpr = 17; -static const int i_vcnt = 18; +static const int i_orank = 15; +static const int i_rps = 16; +static const int i_eclv = 17; +static const int i_mpr = 18; +static const int i_vcnt = 19; -static const int n_txt = 19; +static const int n_txt = 20; // Text file type static const STATLineType txt_file_type[n_txt] = { @@ -74,10 +75,11 @@ static const STATLineType txt_file_type[n_txt] = { stat_prc, // 13 stat_ecnt, // 14 - stat_rps, // 14 - stat_eclv, // 15 - stat_mpr, // 16 - stat_vcnt, // 17 + stat_orank, // 15 + stat_rps, // 16 + stat_eclv, // 17 + stat_mpr, // 18 + stat_vcnt, // 19 }; @@ -178,19 +180,21 @@ class PointStatVxOpt { int get_n_oprob_thresh() const; int get_n_eclv_points() const; + int get_n_hira_ens() const; + int get_n_hira_prob() const; int get_n_cdf_bin() const; int get_n_ci_alpha() const; }; //////////////////////////////////////////////////////////////////////// -inline int PointStatVxOpt::get_n_msg_typ() const { return(msg_typ.n_elements()); } -inline int PointStatVxOpt::get_n_mask() const { return(mask_name.n_elements()); } -inline int PointStatVxOpt::get_n_interp() const { return(interp_info.n_interp); } +inline int PointStatVxOpt::get_n_msg_typ() const { return(msg_typ.n()); } +inline int PointStatVxOpt::get_n_mask() const { return(mask_name.n()); } +inline int PointStatVxOpt::get_n_interp() const { return(interp_info.n_interp); } -inline int PointStatVxOpt::get_n_eclv_points() const { return(eclv_points.n_elements()); } -inline int PointStatVxOpt::get_n_cdf_bin() const { return(cdf_info.n_bin); } -inline int PointStatVxOpt::get_n_ci_alpha() const { return(ci_alpha.n_elements()); } +inline int PointStatVxOpt::get_n_eclv_points() const { return(eclv_points.n()); } +inline int PointStatVxOpt::get_n_cdf_bin() const { return(cdf_info.n_bin); } +inline int PointStatVxOpt::get_n_ci_alpha() const { return(ci_alpha.n()); } //////////////////////////////////////////////////////////////////////// @@ -266,6 +270,8 @@ class PointStatConfInfo { int get_max_n_fprob_thresh() const; int get_max_n_oprob_thresh() const; int get_max_n_eclv_points() const; + int get_max_n_hira_ens() const; + int get_max_n_hira_prob() const; // Check for any verification of vectors bool get_vflag() const; diff --git a/test/config/PointStatConfig_APCP b/test/config/PointStatConfig_APCP index 003f9f50a0..ea8b55a5b4 100644 --- a/test/config/PointStatConfig_APCP +++ b/test/config/PointStatConfig_APCP @@ -120,6 +120,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = BOTH; mpr = NONE; diff --git a/test/config/PointStatConfig_APCP_HIRA b/test/config/PointStatConfig_APCP_HIRA index eefbcfdda0..412d2d8fe7 100644 --- a/test/config/PointStatConfig_APCP_HIRA +++ b/test/config/PointStatConfig_APCP_HIRA @@ -123,6 +123,7 @@ output_flag = { pjc = STAT; prc = STAT; ecnt = STAT; + orank = STAT; rps = STAT; eclv = STAT; mpr = STAT; diff --git a/test/config/PointStatConfig_GTG_latlon b/test/config/PointStatConfig_GTG_latlon index 7e6e2c53d2..1ac24d927e 100644 --- a/test/config/PointStatConfig_GTG_latlon +++ b/test/config/PointStatConfig_GTG_latlon @@ -142,6 +142,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = BOTH; mpr = BOTH; diff --git a/test/config/PointStatConfig_GTG_lc b/test/config/PointStatConfig_GTG_lc index 4b6df47f87..5efd47662c 100644 --- a/test/config/PointStatConfig_GTG_lc +++ b/test/config/PointStatConfig_GTG_lc @@ -150,6 +150,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = BOTH; mpr = BOTH; diff --git a/test/config/PointStatConfig_INTERP_OPTS b/test/config/PointStatConfig_INTERP_OPTS index 4923dab75b..c255c5b565 100644 --- a/test/config/PointStatConfig_INTERP_OPTS +++ b/test/config/PointStatConfig_INTERP_OPTS @@ -133,6 +133,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = NONE; mpr = STAT; diff --git a/test/config/PointStatConfig_LAND_TOPO_MASK b/test/config/PointStatConfig_LAND_TOPO_MASK index 74d7e28be1..4d1ab530fc 100644 --- a/test/config/PointStatConfig_LAND_TOPO_MASK +++ b/test/config/PointStatConfig_LAND_TOPO_MASK @@ -173,6 +173,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = NONE; mpr = BOTH; diff --git a/test/config/PointStatConfig_MASK_SID b/test/config/PointStatConfig_MASK_SID index 95c30a9d16..e4174e9e9a 100644 --- a/test/config/PointStatConfig_MASK_SID +++ b/test/config/PointStatConfig_MASK_SID @@ -128,6 +128,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = BOTH; mpr = BOTH; diff --git a/test/config/PointStatConfig_PHYS b/test/config/PointStatConfig_PHYS index 4a5640ddcb..7a767a54da 100644 --- a/test/config/PointStatConfig_PHYS +++ b/test/config/PointStatConfig_PHYS @@ -129,6 +129,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = BOTH; mpr = BOTH; diff --git a/test/config/PointStatConfig_PHYS_pint b/test/config/PointStatConfig_PHYS_pint index 9102584bff..4149345038 100644 --- a/test/config/PointStatConfig_PHYS_pint +++ b/test/config/PointStatConfig_PHYS_pint @@ -124,6 +124,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = BOTH; mpr = BOTH; diff --git a/test/config/PointStatConfig_WINDS b/test/config/PointStatConfig_WINDS index 4f17fff75a..2b3ff9988e 100644 --- a/test/config/PointStatConfig_WINDS +++ b/test/config/PointStatConfig_WINDS @@ -145,6 +145,7 @@ output_flag = { prc = NONE; ecnt = NONE; rps = NONE; + orank = NONE; eclv = NONE; mpr = NONE; } diff --git a/test/config/PointStatConfig_aeronet b/test/config/PointStatConfig_aeronet index b36dd7f39c..58579bdf68 100644 --- a/test/config/PointStatConfig_aeronet +++ b/test/config/PointStatConfig_aeronet @@ -193,6 +193,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = NONE; mpr = STAT; diff --git a/test/config/PointStatConfig_airnow b/test/config/PointStatConfig_airnow index cbf3c76c11..7420b10455 100644 --- a/test/config/PointStatConfig_airnow +++ b/test/config/PointStatConfig_airnow @@ -223,6 +223,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = NONE; mpr = STAT; diff --git a/test/config/PointStatConfig_climo b/test/config/PointStatConfig_climo index cd81aa6fbf..cb6eaea84b 100644 --- a/test/config/PointStatConfig_climo +++ b/test/config/PointStatConfig_climo @@ -262,6 +262,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = NONE; mpr = BOTH; diff --git a/test/config/PointStatConfig_climo_WMO b/test/config/PointStatConfig_climo_WMO index fa7499ab84..201e794890 100644 --- a/test/config/PointStatConfig_climo_WMO +++ b/test/config/PointStatConfig_climo_WMO @@ -210,6 +210,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = BOTH; + orank = NONE; rps = BOTH; eclv = NONE; mpr = BOTH; diff --git a/test/config/PointStatConfig_climo_prob b/test/config/PointStatConfig_climo_prob index 18ac41ccdf..2447701f95 100644 --- a/test/config/PointStatConfig_climo_prob +++ b/test/config/PointStatConfig_climo_prob @@ -212,6 +212,7 @@ output_flag = { pjc = BOTH; prc = BOTH; ecnt = NONE; + orank = NONE; rps = NONE; eclv = BOTH; mpr = BOTH; diff --git a/test/config/PointStatConfig_dup b/test/config/PointStatConfig_dup index 4dfe9f5c11..9298bb58ec 100644 --- a/test/config/PointStatConfig_dup +++ b/test/config/PointStatConfig_dup @@ -146,6 +146,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = STAT; mpr = BOTH; diff --git a/test/config/PointStatConfig_mpr_thresh b/test/config/PointStatConfig_mpr_thresh index daf7a11d2f..66bde341ef 100644 --- a/test/config/PointStatConfig_mpr_thresh +++ b/test/config/PointStatConfig_mpr_thresh @@ -204,6 +204,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = NONE; mpr = NONE; diff --git a/test/config/PointStatConfig_obs_summary b/test/config/PointStatConfig_obs_summary index 908f86e44c..b8129521ec 100644 --- a/test/config/PointStatConfig_obs_summary +++ b/test/config/PointStatConfig_obs_summary @@ -135,6 +135,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = STAT; mpr = BOTH; diff --git a/test/config/PointStatConfig_obs_summary_all b/test/config/PointStatConfig_obs_summary_all index 636ed3d001..080ba56dff 100644 --- a/test/config/PointStatConfig_obs_summary_all +++ b/test/config/PointStatConfig_obs_summary_all @@ -204,6 +204,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = STAT; mpr = BOTH; diff --git a/test/config/PointStatConfig_prob b/test/config/PointStatConfig_prob index b1fe365e2b..e2ed42b77f 100644 --- a/test/config/PointStatConfig_prob +++ b/test/config/PointStatConfig_prob @@ -131,6 +131,7 @@ output_flag = { pjc = BOTH; prc = BOTH; ecnt = NONE; + orank = NONE; rps = NONE; eclv = BOTH; mpr = NONE; diff --git a/test/config/PointStatConfig_python b/test/config/PointStatConfig_python index 0c66c8c4f0..752fb7a928 100644 --- a/test/config/PointStatConfig_python +++ b/test/config/PointStatConfig_python @@ -201,6 +201,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = NONE; mpr = STAT; diff --git a/test/config/PointStatConfig_qty b/test/config/PointStatConfig_qty index 98ad597e46..6f7db1b826 100644 --- a/test/config/PointStatConfig_qty +++ b/test/config/PointStatConfig_qty @@ -153,6 +153,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = BOTH; mpr = BOTH; diff --git a/test/config/PointStatConfig_sid_inc_exc b/test/config/PointStatConfig_sid_inc_exc index 62e6e9b43f..9dc3e7fdcf 100644 --- a/test/config/PointStatConfig_sid_inc_exc +++ b/test/config/PointStatConfig_sid_inc_exc @@ -136,6 +136,7 @@ output_flag = { pjc = NONE; prc = NONE; ecnt = NONE; + orank = NONE; rps = NONE; eclv = NONE; mpr = BOTH;