Update develop-ref after #1628 (#1632)

* Getting rid of compiler warnings in PB2NC by replacing several instances of the NULL pointer with the nul character (\0) instead. * Fix typo in config_options.rst. * Feature 1408 var_name_for_grib_code (#1617) * #1408 Added get_var_id * #1408 Check variable name in the configuration to use the variable name instewad of grib code * #1408 Added point2grid_ascii2nc_surfrad_DW_PSP_by_name * Feature 1580 2d time (#1616) * #1580 Added get_grid_from_lat_lon_vars * #1580 Added get_grid_from_lat_lon_vars and support 2D time variable * #1580 Support int type variable without scale_factor and add_offset attributes * #1580 Support 2D time variable. Implemented filtering by valid_time * #1580 Bug fix: read time with dimension 0 * #1580 Support time variable with no dimension * #1580 Initial release * #1580 Added point2grid_2D_time * #1580 Check project attribute for GOES * #1580 Changed NULL to 0 to avoid co,pilation warning * #1580 Added point2grid_2D_time * #1580 Added "point2grid configuration file" section * #1580 Changed to_grid for point2grid_NCCF_UK & point2grid_2D_time Co-authored-by: Howard Soh <[email protected]> Co-authored-by: John Halley Gotway <[email protected]> * feature 1580 nccf (#1619) * #1580 Correct the precision at _apply_scale_factor * #1580 Added unit test plot_data_plane_NCCF_time * #1580 Changed argument type to double at _apply_scale_factor(double) * Bugfix 1618 develop pb2nc (#1623) Co-authored-by: Howard Soh <[email protected]> * Feature 1624 OBS_COMMAND (#1625) * Per #1627, add grid_data.regrid config option for PlotPointObs and update the tool to do the requested regridding. Still need to update the docs. * Per #1627, update docs about grid_data.regrid config option for PlotPointObs. * Per #1627, add another call to plot_point_obs to exercise the new regrid functionality. * Feature 1624 obs_command second try (#1629) * Per #1624, define OBS_COMMAND. * Per #1624, unset the test-specific environment variables after completing the run. * Per #1624, after PR #1625 merged these changes into develop, they caused 2 unexpected diffs in the NB output. These were caused by enviornment variables being unset after each test. Updating unit_netcdf.xml and unit_point2grid.xml to define more test-specific environment variables to reproduce previous NB output. * Organizing NB climatology and point2grid output files into the appopriate directories rather than having them at the top-level directory. * Update pull_request_template.md * Update the point2grid unit tests to write their temp files to the point2grid subdirectory instead of the top-level test output directory. * Update appendixC.rst Split the definition of H_RATE and POD Co-authored-by: hsoh-u <[email protected]> Co-authored-by: Howard Soh <[email protected]> Co-authored-by: John Halley Gotway <[email protected]> Co-authored-by: j-opatz <[email protected]>
dtcenter · Jan 22, 2021 · 3fa2b3a · 3fa2b3a
1 parent 5178055
commit 3fa2b3a
Show file tree

Hide file tree

Showing 14 changed files with 141 additions and 38 deletions.
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
@@ -2,17 +2,19 @@
 
 - [ ] Describe testing already performed for these changes:</br>
 
-- [ ] Recommend testing for the reviewer to perform, including the location of input datasets:</br>
+- [ ] Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:</br>
+
+- [ ] Do these changes include sufficient documentation and testing updates? **[Yes or No]**
 
 - [ ] Will this PR result in changes to the test suite? **[Yes or No]**</br>
 If **yes**, describe the new output and/or changes to the existing output:</br>
 
-- [ ] After merging, should the reviewer **DELETE** the feature branch from GitHub? **[Yes or No]**</br>
-
 ## Pull Request Checklist ##
 See the [METplus Workflow](https://dtcenter.github.io/METplus/Contributors_Guide/github_workflow.html) for details.
 - [ ] Complete the PR definition above.
 - [ ] Ensure the PR title matches the feature or bugfix branch name.
 - [ ] Define the PR metadata, as permissions allow.
 Select: **Reviewer(s)**, **Project(s)**, and **Milestone**
 - [ ] After submitting the PR, select **Linked Issues** with the original issue number.
+- [ ] After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
+- [ ] Close the linked issue and delete your feature or bugfix branch from GitHub.
diff --git a/met/data/config/PlotPointObsConfig_default b/met/data/config/PlotPointObsConfig_default
@@ -12,12 +12,21 @@ grid_data = {
 
    field = [];
 
+   regrid = {
+      to_grid    = NONE;
+      method     = NEAREST;
+      width      = 1;
+      vld_thresh = 0.5;
+      shape      = SQUARE;
+   }
+
    grid_plot_info = {
       color_table   = "MET_BASE/colortables/met_default.ctable";
       plot_min      = 0.0;
       plot_max      = 0.0;
       colorbar_flag = TRUE; 
    }
+
 }
 
 ////////////////////////////////////////////////////////////////////////////////

diff --git a/met/docs/Users_Guide/appendixC.rst b/met/docs/Users_Guide/appendixC.rst
@@ -140,19 +140,29 @@ Frequency Bias is the ratio of the total number of forecasts of an event to the
 
 A "good" value of Frequency Bias is close to 1; a value greater than 1 indicates the event was forecasted too frequently and a value less than 1 indicates the event was not forecasted frequently enough.
 
-Probability of Detection (POD)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+H_RATE
+~~~~~~
 
 Called "H_RATE" in FHO output :numref:`table_PS_format_info_FHO`
 
+H_RATE is defined as
+
+.. math:: \text{H_RATE } = \frac{n_{11}}{T}.
+
+
+H_RATE is equivalent to the H value computed by the NCEP verification system. H_RATE ranges from 0 to 1; a perfect forecast would have H_RATE = 1.
+
+Probability of Detection (POD)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 Called "PODY" in CTS output :numref:`table_PS_format_info_CTS`
 
 POD is defined as 
 
 .. math:: \text{POD } = \frac{n_{11}}{n_{11} + n_{01}} = \frac{n_{11}}{n_1}.
 
 
-It is the fraction of events that were correctly forecasted to occur. POD is equivalent to the H value computed by the NCEP verification system and is also known as the hit rate. POD ranges from 0 to 1; a perfect forecast would have POD = 1.
+It is the fraction of events that were correctly forecasted to occur. POD is also known as the hit rate. POD ranges from 0 to 1; a perfect forecast would have POD = 1.
 
 Probability of False Detection (POFD)
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

diff --git a/met/docs/Users_Guide/plotting.rst b/met/docs/Users_Guide/plotting.rst
@@ -81,6 +81,14 @@ ______________________
 
      field = [];
 
+     regrid = {
+        to_grid    = NONE;
+        method     = NEAREST;
+        width      = 1;
+        vld_thresh = 0.5;
+        shape      = SQUARE;
+     }
+
      grid_plot_info = {
         color_table   = "MET_BASE/colortables/met_default.ctable";
         plot_min      = 0.0;
@@ -91,6 +99,8 @@ ______________________
 
 The **grid_data** dictionary defines a gridded field of data to be plotted as a base image prior to plotting point locations on top of it. The data to be plotted is specified by the **field** array. If **field** is empty, no base image will be plotted. If **field** has length one, the requested data will be read from the input file specified by the **-plot_grid** command line argument.
 
+The **to_grid** entry in the **regrid** dictionary specifies if and how the requested gridded data should be regridded prior to plotting. Please see :numref:`config_options` for a description of the **regrid** dictionary options.
+
 The **grid_plot_info** dictionary inside **grid_data** specifies the options for for plotting the gridded data. The options within **grid_plot_info** are described in :numref:`config_options`.
 
 ______________________

diff --git a/met/src/tools/other/pb2nc/numpbmsg.f b/met/src/tools/other/pb2nc/numpbmsg.f
@@ -24,7 +24,29 @@ SUBROUTINE NUMPBMSG ( FID, NMSG )
 C*
         END
 
-
+
+        SUBROUTINE NUMPBMSG_NEW ( PBFILE, FID, NMSG )
+C*
+        INCLUDE    'readpb.prm'
+C*
+        CHARACTER  PBFILE*(FILEMXSTRL)
+        INTEGER    FID, FID2
+        INTEGER    NMSG
+        REAL*8     R8ARR(1, 1)
+C*
+C-----------------------------------------------------------------------
+C*
+C*      Call UFBTAB to figure out how many messages the PrepBufr file
+C*      attached to FID contains
+C*
+        OPEN  ( UNIT = FID, FILE = PBFILE, FORM = 'UNFORMATTED' )
+        FID2 = -FID
+        CALL UFBTAB  ( FID2, R8ARR, 1, 1, NMSG, ' ' )
+        CLOSE ( UNIT = FID )
+C*
+        END
+
+
         SUBROUTINE GET_TMIN ( FID, TMIN )
 C*
         INTEGER    FID, TMIN

diff --git a/met/src/tools/other/pb2nc/pb2nc.cc b/met/src/tools/other/pb2nc/pb2nc.cc
@@ -312,6 +312,7 @@ static NcFile *f_out      = (NcFile *) 0;
 
 extern "C" {
    void numpbmsg_(int *, int *);
+   void numpbmsg_new_(const char *, int *, int *);
    void openpb_(const char *, int *);
    void closepb_(int *);
    void readpb_(int *, int *, int *, double[mxr8pm],
@@ -2050,14 +2051,14 @@ void process_pbfile_metadata(int i_pb) {
            << "Invalid file ID [" << unit << "] between 1 and 99.\n\n";
    }
 
-   // Open the blocked temp PrepBufr file for reading
-   openpb_(blk_file.c_str(), &unit);
-
    // Compute the number of PrepBufr records in the current file.
-   numpbmsg_(&unit, &npbmsg);
+   numpbmsg_new_(blk_file.c_str(), &unit, &npbmsg);
    mlog << Debug(1) << method_name << " -> "
         << "the number of records: " << npbmsg << "\n";
 
+   // Open the blocked temp PrepBufr file for reading
+   openpb_(blk_file.c_str(), &unit);
+
    // Use the number of records requested by the user if there
    // are enough present.
    if(nmsg >= 0 && nmsg <= npbmsg) {

diff --git a/met/src/tools/other/plot_point_obs/plot_point_obs_conf_info.cc b/met/src/tools/other/plot_point_obs/plot_point_obs_conf_info.cc
@@ -15,6 +15,8 @@ using namespace std;
 #include "vx_config.h"
 #include "vx_data2d.h"
 #include "vx_data2d_factory.h"
+#include "vx_regrid.h"
+#include "vx_statistics.h"
 
 #include "plot_point_obs_conf_info.h"
 
@@ -487,6 +489,19 @@ void PlotPointObsConfInfo::process_config(
             exit(1);
          }
 
+         // Regrid, if requested
+         if(grid_data_info->regrid().enable) {
+            mlog << Debug(1) << "Regridding field "
+                 << grid_data_info->magic_str() << ".\n";
+            Grid to_grid(parse_vx_grid(grid_data_info->regrid(),
+                                       &grid, &grid));
+            grid_data = met_regrid(grid_data, grid, to_grid,
+                                   grid_data_info->regrid());
+
+            // Store the new grid definition
+            grid = to_grid;
+         }
+
          // Conf: grid_plot_info
          grid_plot_info = parse_conf_plot_info(
                              dict->lookup_dictionary(

diff --git a/test/config/PlotPointObsConfig b/test/config/PlotPointObsConfig
@@ -12,6 +12,14 @@ grid_data = {
 
    field = [ { name = "RH"; level = "P500"; } ];
 
+   regrid = {
+      to_grid    = ${TO_GRID};
+      method     = NEAREST;
+      width      = 1;
+      vld_thresh = 0.5;
+      shape      = SQUARE;
+   }
+
    grid_plot_info = {
       color_table   = "MET_BASE/colortables/NCL_colortables/BlueGreen14.ctable";
       plot_min      = 0.0;

diff --git a/test/perl/unit.pl b/test/perl/unit.pl
@@ -94,11 +94,13 @@ sub usage {
   elsif ($callchk) {
     $cmd = "valgrind ".$VALGRIND_OPT_CALL." ".$cmd;
   }
-  
+
   # if writing a command file, print the environment and command, then loop
   if( $cmd_only ){
     print "export $_=\'" . $test->{"env"}{$_} . "\'\n" for sort keys %{ $test->{"env"} };
-    print "$cmd\n\n";
+    print "$cmd\n";
+    print "unset $_\n" for sort keys %{ $test->{"env"} };
+    print "\n";
     next;
   }
 
@@ -137,23 +139,28 @@ sub usage {
 
   }
 
+  # unset the test environment variables
+  delete $ENV{$_} for keys %{ $test->{"env"} };
+
   # print the test result
   printf "%s - %7.3f sec\n", $ret_ok && $out_ok ? "pass" : "FAIL", sprintf("%7.3f", $t_elaps);
 
-  # build a list of environment variable exports for reporting
-  my @envs;
-  push @envs, "export $_=\'" . $test->{"env"}{$_} . "\'\n" for sort keys %{ $test->{"env"} };
+  # build a list of environment variable exports and unsets for reporting
+  my @set_envs;
+  push @set_envs, "export $_=\'" . $test->{"env"}{$_} . "\'\n" for sort keys %{ $test->{"env"} };
+  my @unset_envs;
+  push @unset_envs, "unset $_\n" for sort keys %{ $test->{"env"} };
 
   # if the log file is activated, print the test information
   if( $file_log ){
     print $fh_log "\n\n";
-    print $fh_log "$_" for (@envs, @cmd_outs);
+    print $fh_log "$_" for (@set_envs, @cmd_outs, @unset_envs);
     print $fh_log "\n\n";
   }
 
   # on failure, print the problematic test and exit, if requested
   if( !($ret_ok && $out_ok) ){
-    print "$_" for (@envs, @cmd_outs);
+    print "$_" for (@set_envs, @cmd_outs, @unset_envs);
     $noexit or exit 1;
     print "\n\n";
   }

diff --git a/test/xml/unit_climatology.xml b/test/xml/unit_climatology.xml
@@ -146,10 +146,10 @@
     <param> \
       -lookin &OUTPUT_DIR;/climatology/point_stat_WMO_CLIMO_1.5DEG_120000L_20120409_120000V.stat \
       -job aggregate_stat -line_type MPR -out_line_type CTC -fcst_lev P850 -interp_mthd NEAREST -by FCST_VAR -out_thresh '>CDP90' \
-      -out_stat &OUTPUT_DIR;/stat_analysis_WMO_1.5DEG_MPR_to_CTC_out.stat
+      -out_stat &OUTPUT_DIR;/climatology/stat_analysis_WMO_1.5DEG_MPR_to_CTC_out.stat
     </param>
     <output>
-      <stat>&OUTPUT_DIR;/stat_analysis_WMO_1.5DEG_MPR_to_CTC_out.stat</stat>
+      <stat>&OUTPUT_DIR;/climatology/stat_analysis_WMO_1.5DEG_MPR_to_CTC_out.stat</stat>
     </output>
   </test>
 
@@ -161,10 +161,10 @@
     <param> \
       -lookin &OUTPUT_DIR;/climatology/point_stat_WMO_CLIMO_1.5DEG_120000L_20120409_120000V.stat \
       -job filter -line_type MPR -column_thresh CLIMO_CDF 'lt0.1||gt0.9' \
-      -dump_row &OUTPUT_DIR;/stat_analysis_WMO_1.5DEG_FILTER_CDF_dump.stat
+      -dump_row &OUTPUT_DIR;/climatology/stat_analysis_WMO_1.5DEG_FILTER_CDF_dump.stat
     </param>
     <output>
-      <stat>&OUTPUT_DIR;/stat_analysis_WMO_1.5DEG_FILTER_CDF_dump.stat</stat>
+      <stat>&OUTPUT_DIR;/climatology/stat_analysis_WMO_1.5DEG_FILTER_CDF_dump.stat</stat>
     </output>
   </test>
 

diff --git a/test/xml/unit_netcdf.xml b/test/xml/unit_netcdf.xml
@@ -81,8 +81,9 @@
   <test name="365_days">
     <exec>&MET_BIN;/grid_stat</exec>
     <env>
-      <pair><name>OUTPUT_PREFIX</name> <value>no_leap</value></pair>
-      <pair><name>APPLY_MASK</name>    <value>FALSE</value></pair>
+      <pair><name>OUTPUT_PREFIX</name>   <value>no_leap</value></pair>
+      <pair><name>APPLY_MASK</name>      <value>FALSE</value></pair>
+      <pair><name>MET_NC_COMPRESS</name> <value>3</value></pair>
     </env>
     <param> \
       &DATA_DIR_MODEL;/easm/base_series_spei12_annual_mean.nc \

diff --git a/test/xml/unit_plot_point_obs.xml b/test/xml/unit_plot_point_obs.xml
@@ -48,6 +48,9 @@
 
   <test name="plot_point_obs_CONFIG">
     <exec>&MET_BIN;/plot_point_obs</exec>
+    <env>
+      <pair><name>TO_GRID</name> <value>NONE</value></pair>
+    </env>
     <param> \
       &OUTPUT_DIR;/pb2nc/ndas.20120409.t12z.prepbufr.tm00.nc \
       &OUTPUT_DIR;/plot_point_obs/nam_and_ndas.20120409.t12z.prepbufr_CONFIG.ps \
@@ -62,4 +65,23 @@
     </output>
   </test>
 
+  <test name="plot_point_obs_CONFIG_REGRID">
+    <exec>&MET_BIN;/plot_point_obs</exec>
+    <env>
+      <pair><name>TO_GRID</name> <value>"latlon 80 100 25 -105 0.25 0.25"</value></pair>
+    </env>
+    <param> \
+      &OUTPUT_DIR;/pb2nc/ndas.20120409.t12z.prepbufr.tm00.nc \
+      &OUTPUT_DIR;/plot_point_obs/nam_and_ndas.20120409.t12z.prepbufr_CONFIG_REGRID.ps \
+      -point_obs &OUTPUT_DIR;/ascii2nc/trmm_2012040912_3hr.nc \
+      -plot_grid &DATA_DIR_MODEL;/grib2/nam/nam_2012040900_F012.grib2 \
+      -config &CONFIG_DIR;/PlotPointObsConfig \
+      -title "NAM 2012040900 F12 vs NDAS 500mb RH and TRMM 3h > 0" \
+      -v 3
+    </param>
+    <output>
+      <ps>&OUTPUT_DIR;/plot_point_obs/nam_and_ndas.20120409.t12z.prepbufr_CONFIG_REGRID.ps</ps>
+    </output>
+  </test>
+
 </met_test>