Merge branch 'init_interp_same_grid'

New init_interp method

The main change in this tag is to introduce a new method for
init_interp. This is needed in order for subgrid areas to be the same in
initialization in runs with and without init_interp.

The choice of init_interp method is controlled by a new namelist
variable, init_interp_method, whose documentation appears below:

   Method to use for init_interp. Only applies when use_init_interp = .true.

   'general': The general-purpose method that can be used when changing
   grids, configurations, etc. This starts off with subgrid areas taken
   from the surface dataset.

   'use_finidat_areas': This starts off with subgrid areas taken from the
   input finidat file. This is needed to achieve bit-for-bit results in a
   coupled case (where areas in initialization impact initial fields sent
   to the atmosphere) (but using the 'general' method will typically have
   only a very minor impact on results in this case). For this method to
   work, the input finidat file needs to be at the same resolution as the
   current configuration. So this is a less general form of
   init_interp. However, it can be used when transitioning from a spinup
   run to a transient run, or in other cases where the only difference is
   in internal memory allocation. In order to catch possible problems, this
   uses a different algorithm for finding the input point for each output
   point, which ensures that each active output point is associated with
   exactly one input point with the same latitude, longitude and type. This
   method requires (a) the same grid for input and output, within roundoff;
   (b) any non-zero-weight point in the input must have memory allocated
   for it in this grid cell in the output (this will be satisfied if the
   point is non-zero-weight on the surface dataset or if it's a point for
   which we allocate memory even for zero-weight points); (c) any active
   point in the output (based on the surface dataset and rules for
   determining active points) must have a matching point in this grid cell
   in the input.

This tag also has some other changes:

- Fixes ESMCI#347 - endrun message behavior: needed for some of the new unit
  tests

- Fixes ESMCI#345 - Add a cmip6_evolving_icesheet usermods directory:
  unrelated, but folded in here for convenience of testing)

bld/CLMBuildNamelist.pm

@@ -1745,6 +1745,10 @@ sub process_namelist_inline_logic {
   #######################################################################
   setup_logic_hydrology_switches($nl, $physv);
 
+  #########################################
+  # namelist group: clm_initinterp_inparm #
+  #########################################
+  setup_logic_initinterp($opts, $nl_flags, $definition, $defaults, $nl, $physv);
 }
 
 #-------------------------------------------------------------------------------
@@ -3723,6 +3727,26 @@ sub setup_logic_lnd2atm {
 
 #-------------------------------------------------------------------------------
 
+sub setup_logic_initinterp {
+   #
+   # Options related to init_interp
+   #
+   my ($opts, $nl_flags, $definition, $defaults, $nl, $physv) = @_;
+
+   if ($physv->as_long() >= $physv->as_long("clm4_5")) {
+      my $var = 'init_interp_method';
+      if ( &value_is_true($nl->get_value("use_init_interp"))) {
+         add_default($opts, $nl_flags->{'inputdata_rootdir'}, $definition, $defaults, $nl, $var);
+      } else {
+         if (defined($nl->get_value($var))) {
+            $log->fatal_error("$var can only be set if use_init_interp is true");
+         }
+      }
+   }
+}
+
+#-------------------------------------------------------------------------------
+
 sub setup_logic_fates {
     #
     # Set some default options related to Ecosystem Demography

bld/namelist_files/namelist_defaults_clm4_5.xml

@@ -2415,4 +2415,10 @@ lnd/clm2/surfdata_map/surfdata_ne120np4_78pfts_CMIP6_simyr1850_c170824.nc</fsurd
 <use_fates_logging             use_fates=".true.">.false.</use_fates_logging>
 <use_fates_inventory_init      use_fates=".true.">.false.</use_fates_inventory_init>
 
+<!-- =========================================  -->
+<!-- Defaults for init_interp                   -->
+<!-- =========================================  -->
+
+<init_interp_method>general</init_interp_method>
+
 </namelist_defaults>

bld/namelist_files/namelist_definition_clm4_5.xml

@@ -2416,4 +2416,37 @@ always fill with the closest natural veg patch / column, regardless of the value
 flag. So interpolation from non-crop to crop cases can be done without setting this flag.
 </entry>
 
+<entry id="init_interp_method" type="char*64" category="datasets"
+       group="clm_initinterp_inparm"
+       valid_values="general,use_finidat_areas" >
+Method to use for init_interp. Only applies when use_init_interp = .true.
+
+'general': The general-purpose method that can be used when changing
+grids, configurations, etc. This starts off with subgrid areas taken
+from the surface dataset.
+
+'use_finidat_areas': This starts off with subgrid areas taken from the
+input finidat file. This is needed to achieve bit-for-bit results in a
+coupled case (where areas in initialization impact initial fields sent
+to the atmosphere) (but using the 'general' method will typically have
+only a very minor impact on results in this case). For this method to
+work, the input finidat file needs to be at the same resolution as the
+current configuration. So this is a less general form of
+init_interp. However, it can be used when transitioning from a spinup
+run to a transient run, or in other cases where the only difference is
+in internal memory allocation. In order to catch possible problems, this
+uses a different algorithm for finding the input point for each output
+point, which ensures that each active output point is associated with
+exactly one input point with the same latitude, longitude and type. This
+method requires (a) the same grid for input and output, within roundoff;
+(b) any non-zero-weight point in the input must have memory allocated
+for it in this grid cell in the output (this will be satisfied if the
+point is non-zero-weight on the surface dataset or if it's a point for
+which we allocate memory even for zero-weight points); (c) any active
+point in the output (based on the surface dataset and rules for
+determining active points) must have a matching point in this grid cell
+in the input.
+
+</entry>
+
 </namelist_definition>

cime_config/SystemTests/lii2finidatareas.py

@@ -0,0 +1,76 @@
+"""
+Implementation of the LII2FINIDATAREAS test.
+
+This is similar to the LII test, but tests init_interp with mode
+'use_finidat_areas'.
+
+As with the standard LII test, this must be used in a configuration for
+which we have a compatible out-of-the-box finidat file (so that the run
+with use_init_interp = .false. runs successfully). In constrast to our
+standard LII test (which uses glcMEC_spunup_1way), this one can use a
+standard CISM2%NOEVOLVE configuration: we do *not* need to set
+GLC_TWO_WAY_COUPLING=FALSE; in fact, it's a better test if we have
+GLC_TWO_WAY_COUPLING=TRUE: with this mode of operation, areas should
+match between the two runs.
+
+It may not be totally necessary to have a system test covering this
+init_interp_method: between unit tests and inline checks, we have quite
+a bit of testing of this code. It's important to have at least one LII
+test to make sure we're interpolating all the fields we're supposed to
+interpolate, but we don't necessarily need a LII test with this mode of
+operation as long as we have one with the general mode of operation.
+
+However, this test is a good check that every point has a unique type
+(because this mode of operation will fail if that's not the case); this
+is something needed even for the 'general' init_interp method to work
+correctly, but the 'general' mode won't catch problems in this regard.
+
+One other reason why this test is useful is to cover the threading
+directives in the set_single_match routine, since those aren't covered
+by unit tests. So this test mod should be used in a test with threading.
+
+To update the initial conditions (finidat) file for this test:
+
+(1) Run the test; the 'base' case should run to completion even if the
+no_interp test fails. (If the 'base' case fails, you may need to retry
+with init_interp_method='general'.)
+
+(2) Copy the finidat_interp_dest.nc file from the 'base' case to the inputdata
+space. Rename this to be similar to the out-of-the-box finidat file
+currently used by this test, but with a new creation date.
+
+(3) Update namelist defaults to point to the new finidat file. If
+updating the out-of-the-box file is not desired, then you could instead
+point to this new finidat file with a user_nl_clm file in this testmod.
+"""
+
+from CIME.XML.standard_module_setup import *
+from CIME.SystemTests.test_utils.user_nl_utils import append_to_user_nl_files
+
+# We can import lii directly because the SystemTests directory has been
+# added to sys.path.
+#
+# A cleaner and more general way to accomplish this would be: In cime:
+# Rather than adding the SystemTests directory to sys.path and then
+# importing an individual module from there: instead, allow each
+# component to have a COMPNAME_pylib directory within its cime_config,
+# and then have cime add each component's cime_config directory to
+# sys.path and then do something like:
+# import_module("COMPNAME_pylib.SystemTests.TESTNAME"). Then, for
+# example, ctsm could access its own modules via "import
+# ctsm_pylib.foo", or (in this case) "from ctsm_pylib.SystemTests.lii
+# import LII".
+from lii import LII
+
+logger = logging.getLogger(__name__)
+
+class LII2FINIDATAREAS(LII):
+
+    def __init__(self, case):
+        super(LII2FINIDATAREAS, self).__init__(case)
+
+    def _case_one_setup(self):
+        super(LII2FINIDATAREAS, self)._case_one_setup()
+        append_to_user_nl_files(caseroot = self._get_caseroot(),
+                                component = "clm",
+                                contents = "init_interp_method = 'use_finidat_areas'")

cime_config/config_tests.xml

@@ -28,6 +28,16 @@ SSP    smoke CLM spinup test (only valid for CLM compsets with CLM45)
     <HIST_N>$STOP_N</HIST_N>
   </test>
 
+  <test NAME="LII2FINIDATAREAS">
+    <DESC>CLM initial condition interpolation test using finidat_areas (requires configuration with non-blank finidat)</DESC>
+    <INFO_DBUG>1</INFO_DBUG>
+    <DOUT_S>FALSE</DOUT_S>
+    <CONTINUE_RUN>FALSE</CONTINUE_RUN>
+    <REST_OPTION>never</REST_OPTION>
+    <HIST_OPTION>$STOP_OPTION</HIST_OPTION>
+    <HIST_N>$STOP_N</HIST_N>
+  </test>
+
   <test NAME="LVG">
     <DESC>CLM test: Verify that adding virtual glacier columns doesn't change answers</DESC>
     <INFO_DBUG>1</INFO_DBUG>

cime_config/testdefs/testlist_clm.xml

@@ -1065,6 +1065,15 @@
       <option name="comment"  >Want to cover both glc_mec and crop in an LII test; this test covers both. Uses a year-2000 restart file so that the restart file has non-zero product pools, so that we exercise the gridcell-level code in init_interp. 2018-04-09: Now that SGLC tests use glc_mec, it would be more straightforward to run this test using a SGLC compset rather than a CISM compset that turns off two-way coupling; I was going to make that change now, but didn't want to risk it given the current time crunch, so let's make that change later.</option>
     </options>
   </test>
+  <test name="LII2FINIDATAREAS_D_P360x2_Ld1" grid="f09_g16_gl4" compset="I1850Clm50BgcCrop" testmods="clm/compatible_finidat_f09">
+    <machines>
+      <machine name="cheyenne" compiler="intel" category="aux_clm"/>
+    </machines>
+    <options>
+      <option name="wallclock">00:20:00</option>
+      <option name="comment"  >Exercise the init_interp_method='use_finidat_areas' option. See documentation at the top of the python script implementing this test for more details and rationale.</option>
+    </options>
+  </test>
   <test name="LVG_Ld5_D" grid="f10_f10_musgs" compset="I1850Clm50Bgc" testmods="clm/no_vector_output">
     <machines>
       <machine name="cheyenne" compiler="intel" category="aux_clm"/>
@@ -1106,13 +1115,13 @@
       <option name="wallclock">00:20:00</option>
     </options>
   </test>
-  <test name="PET_P36x2_D" grid="f10_f10_musgs" compset="I1850Clm50BgcCrop" testmods="clm/interp_f19_noncrop1850">
+  <test name="PET_P36x2_D" grid="f10_f10_musgs" compset="I1850Clm50BgcCrop" testmods="clm/default">
     <machines>
       <machine name="cheyenne" compiler="gnu" category="aux_clm"/>
     </machines>
     <options>
       <option name="wallclock">00:20:00</option>
-      <option name="comment"  >include a test of initInterp from one resolution to another; make it a PET test to exercise the OpenMP directives in initInterp</option>
+      <option name="comment"  >The main purpose of this test is to test threading of init_interp, exercising the OpenMP directives in initInterp. (Note that ERP tests don't compare threaded vs. non-threaded runs of init_interp, since init_interp won't run in the restart case.) Note that this test will use init_interp as long as we don't have out-of-the-box initial conditions at f10 resolution. We could probably get a similar level of confidence in the threading directives by deleting this test and instead changing the LII test to use threading; the main loss would be that that wouldn't test threading combined with interpolating from one resolution to another, as this one does.</option>
     </options>
   </test>
   <test name="SMS" grid="f10_f10_musgs" compset="I2000Clm50BgcCrop" testmods="clm/crop">
@@ -1231,15 +1240,6 @@
       <option name="wallclock">00:20:00</option>
     </options>
   </test>
-  <test name="SMS_D_Ld3" grid="f10_f10_musgs" compset="I1850Clm50Bgc" testmods="clm/interp_f19_crop">
-    <machines>
-      <machine name="cheyenne" compiler="gnu" category="aux_clm"/>
-    </machines>
-    <options>
-      <option name="wallclock">00:20:00</option>
-      <option name="comment"  >test interpolation of crop to non-crop (also change resolution, but that is not an essential part of this test)</option>
-    </options>
-  </test>
   <test name="SMS_D_Ld3" grid="f10_f10_musgs" compset="I1850Clm50BgcCrop" testmods="clm/default1850">
     <machines>
       <machine name="cheyenne" compiler="intel" category="aux_clm"/>
@@ -1260,15 +1260,6 @@
       <option name="wallclock">00:20:00</option>
     </options>
   </test>
-  <test name="SMS_D_Ld3" grid="f10_f10_musgs" compset="I2000Clm50BgcCrop" testmods="clm/interp_f19_noncrop">
-    <machines>
-      <machine name="cheyenne" compiler="gnu" category="aux_clm"/>
-    </machines>
-    <options>
-      <option name="wallclock">00:20:00</option>
-      <option name="comment"  >test interpolation of non-crop to crop (also change resolution, but that is not an essential part of this test)</option>
-    </options>
-  </test>
   <test name="SMS_D_Ld3" grid="f10_f10_musgs" compset="I2000Clm50BgcCruGs" testmods="clm/default">
     <machines>
       <machine name="cheyenne" compiler="intel" category="aux_clm"/>

cime_config/testdefs/testmods_dirs/clm/cmip6/README

@@ -0,0 +1,6 @@
+Note that this testmod directory includes cmip6_evolving_icesheet rather
+than simply cmip6: cmip6_evolving_icesheet includes cmip6 and adds to
+it. So, rather than having tests of the cmip6 usermods and separate
+tests of cmip6_evolving_icesheet, it's more efficient to just have tests
+of the cmip6_evolving_icesheet usermods directory, which then also cover
+the cmip6 usermods directory.

cime_config/testdefs/testmods_dirs/clm/cmip6/include_user_mods

@@ -1 +1 @@
-../../../../usermods_dirs/cmip6
+../../../../usermods_dirs/cmip6_evolving_icesheet

cime_config/testdefs/testmods_dirs/clm/compatible_finidat_f09/README

@@ -0,0 +1,23 @@
+This testmod provides an finidat file compatible with the current model
+configuration, at f09 resolution.
+
+This testmods directory is for running LII tests which compare the answers for a case where
+initial condition interpolation is on (use_init_interp=T) to a case with it's off and ensures
+they are exact. For the interpolated result to match uninterpolation, it needs to be a case that 
+essentially needs no interpolation so it's at the same resolution as the initial condition 
+file (finidat file). When surface datasets are changed, or the land-mask is changed, or an 
+important change is made to model physics (for example where new fields are added to the restart
+file) -- you'll need to update the initial conditions file in this test (finidat file in 
+the user_nl_clm file). 
+
+To update the initial conditions (finidat) file for this test:
+
+(1) Run the LII test; the 'base' case should run to completion even if the
+no_interp test fails.
+
+(2) Copy the finidat_interp_dest.nc file from the 'base' case to the inputdata
+space. Rename this to be similar to the name of the file pointed to in this
+test's user_nl_clm file, but with a new creation date.
+
+(3) Update this the user_nl_clm file in this directory to point to the new
+finidat file.

cime_config/testdefs/testmods_dirs/clm/compatible_finidat_f09/user_nl_clm

@@ -0,0 +1 @@
+finidat = '$DIN_LOC_ROOT/lnd/clm2/initdata_map/clmi.I1850Clm50BgcCrop.1366-01-01.0.9x1.25_gx1v6_simyr1850_c180424.nc'

cime_config/testdefs/testmods_dirs/clm/glcMEC_spunup_1way/README

@@ -3,7 +3,7 @@ initial condition interpolation is on (use_init_interp=T) to a case with it's of
 they are exact. For the interpolated result to match uninterpolation, it needs to be a case that 
 essentially needs no interpolation so it's at the same resolution as the initial condition 
 file (finidat file). When surface datasets are changed, or the land-mask is changed, or an 
-imporant change is made to model physics (for example where new fields are added to the restart 
+important change is made to model physics (for example where new fields are added to the restart
 file) -- you'll need to update the initial conditions file in this test (finidat file in 
 the user_nl_clm file). 
 
@@ -12,7 +12,14 @@ glacier areas are taken from the restart file rather than from
 CISM. This is partly so that changes in CISM don't affect the test
 results, but more importantly so that LII tests can pass even if CISM
 changes. (GLC two-way coupling was off in the generation of the
-initial conditions file used here, too.)
+initial conditions file used here, too.) (4-24-18: I *think* we'll be
+able to turn two-way coupling on once
+https://github.com/ESCOMP/ctsm/issues/340 is resolved, though I'm not
+positive about that. However, if
+https://github.com/ESMCI/cime/issues/2484 is done, then we'd need this
+testmods to turn off comparison of cpl initial hist files if we have
+two-way coupling on, since areas will differ in initialization between
+the two cases.)
 
 To update the initial conditions (finidat) file for this test:
 

cime_config/usermods_dirs/cmip6_evolving_icesheet/include_user_mods

@@ -0,0 +1 @@
+../cmip6

cime_config/usermods_dirs/cmip6_evolving_icesheet/user_nl_clm

@@ -0,0 +1 @@
+hist_fincl4 += 'ICE_MODEL_FRACTION'