Mapping/update runoff to ocn #1280

mnlevy1981 · 2017-03-24T22:12:32Z

Updates to the runoff_to_ocean generator tool, with three goals in mind:

There was a bug in the ocean -> ocean smoothing
With the introduction of the estuary box model (EBM) in POP, we need a tool that merges the nearest neighbor map with the smooth map so that the result is a nearest neighbor map in open ocean but a smooth map in the marginal seas
The introduction of the EBM also added new requirements to the tool itself, namely the ability to map from runoff to coastal ocean points in the nearest neighbor map and then smooth from the coastal ocean points to the global ocean in the smooth map

Test suite: N/A
Test baseline: N/A
Test namelist changes: N/A
Test status: N/A

User interface changes: the runoff map namelists are slightly different (though this code is backwards compatible with the old namelist)

Code review: @klindsay28 and I looked at how I implemented his NCL script in the code, and @kauff and I looked at the Fortran modifications.

I believe @jtruesdal is doing some test runs with maps generated from this new tool; I'm staging the pull request now while I'm thinking of it, but this probably shouldn't be accepted until we get the green light from him.

The configure script has been updated so the directions in INSTALL were a little out of date. (This will need to be updated again after issue #1265 is closed.) Also updated the clean target to also remove executable (removing need for realclean target) and the distclean target to clean up after configure.

The ocean -> ocean smooth map in the runoff to ocean map generator had a bug in it such that the ocean source point was not included in the list of destination points.

1. split namelist option file_ocn into file_ocn_global_mask and file_ocn_coastal_mask 2. created a new module mapread_mod that contains a subroutine for setting the destination of a map based on a SCRIP grid file 3. above step required a new maptype_mod for the sparse matrix type and some other parameters (map_mod and mapread_mod both need them, and map_mod uses mapread_mod so I needed to move definitions out of map_mod)

Before a user simply ran $ ./runoff_map And that read in runoff_map.nml (file name was hard-coded). Now the user runs $ ./runoff_map < [namelist file] And any file can be used. Besides the change to the fortran code, I updated the README file and removed the generic runoff_map.nml file name. Next commit will update the rest of namelists to include the coastal / global ocean grid files.

The four namelists that handle mapping between {r05,rx1} and {gx3v7,gx1v7} are up to date and point to the correct coastal mask files.

Also fixed a typo in a couple of the namelist files

Three old namelists won't work without updating the namelist to point to a coastal ocean grid. Instead of doing that, I deleted the namelist (they can be recreated if needed later)

Smooth map no longer assumes src and destination are the same (it maps from file_ocn_coastal_mask -> file_ocn_global_mask)

1) if file_ocn_coastal_mask is not specified, use file_ocn_global_mask 2) add nn_dest_is_smooth_src flag to namelist / code 3) Improve error reporting from NCL script 4) Update README to give better example of a namelist 5) When reading destination grid from netcdf file, domain_b = filename

NCL script ---------- 1) Error checking -- make sure all input files exist before reading any code -- all exits should be status_exit(1) 2) Don't use NCL_ARGS_# for environment variable, use descriptive var name 3) Add region_mask to global attributes Bash script ----------- 1) Clean up options (--map_out instead of --map-out-fname, etc) 2) Add a main() function, so that meat of script is at top of file 3) Update usage() to use cat << EOF instead of a bunch of echos 4) Add line delimiters between subroutines to make it easier to read

Things like $URL$ and $ID$ are out of date now that this code is kept in git

1. rename namelist files to match final map name; i.e. map_rx1_to_gx1v7_nnsm_e1000r300.nml can be used to generate map_rx1_to_gx1v7_nnsm_e1000r300_YYYYMMDD.nc 2. Rename some namelist variables -- file_ocn_global_mask back to file_ocn -- nn_dest_is_smooth_src to restrict_smooth_src_to_nn_dest

Want all namelists to produce files in the same format: file_nn = 'map_{roff}_to_{ocn}_nearestdtos_170324.nc' file_smooth = 'map_{roff}_to_{ocn}_sm_e{eFold}r{rMax}_170324.nc' file_new = 'map_{roff}_to_{ocn}_nnsm_e1000r300_170324.nc' And also have similar titles title = 'runoff map: {roff} -> {ocn}, nearest neighbor and smoothed' And example in README should also be consistent

I thought mapread_mod and maptype_mod were necessary to avoid circular dependencies, but smooth_mod already depends on map_mod... so map_DestGridRead is in map_mod and I don't need to move the sparse matrix type out of map_mod either.

This changes the installation instructions slightly, as you no longer need to rely on environment variables when running config. Also, "make clean" now also removes the dependency generator, and make distclean uses configure to clean up the configure-generated files.

jedwards4b · 2017-03-24T22:15:48Z

tools/mapping/gen_mapping_files/runoff_to_ocn/INSTALL

-(4) $ gmake
+(2) $ ../../../../configure --macros-format Makefile --mpilib mpi-serial
+Bash users:
+(3) $ (. .env_mach_specific.sh ; gmake)


source also works in bash

billsacks

Looks good. I'm just asking for some clarifications to the README and comments to make the usage of the new optional arguments more clear.

billsacks · 2017-03-28T19:06:01Z

tools/mapping/gen_mapping_files/runoff_to_ocn/README

+   * The optional file_ocn_coast_mask file allows mapping to a subset of ocean
+     cells, presumably the coastal cells)
+2) A smoothing map from the ocean grid to the ocean grid.
+   * The optional file_ocn_coast_mask file allows mapping from a subset of


I appreciate the detailed descriptions in the README file: thank you!

However, with these new options, I'm now confused about how I should actually create the mapping files needed by CESM. I'm especially unsure about when I should specify a file_ocn_coast_mask file.

Can we describe how to generate specific mapping files, either here or elsewhere? I'm especially interested in how to regenerate the GLC2OCN runoff mapping files (and maybe it would be best if we sat down and chatted about this to make sure we get it right), but presumably people would also like to know exactly how to generate rof2ocn_liq and rof2ocn_ice.

billsacks · 2017-03-28T19:23:08Z

tools/mapping/gen_mapping_files/runoff_to_ocn/src/smooth_mod.F90

-   type(sMatrix),intent(inout) :: map_out  ! smoothing matrix
+   character(*), intent(in)    :: ofilename                       ! name of ocn scrip grid file
+   type(sMatrix),intent(in)    :: map_in                          ! original unsmoothed, matrix
+   logical,      intent(in)    :: restrict_smooth_src_to_nn_dest  ! map_out%mask_a = pts mapped to map_in


Can you add some more comments, either here or in the PURPOSE comment below, describing why you might choose to have restrict_smooth_src_to_nn_dest false instead of true? Something along the lines of what you have in the README, perhaps.

billsacks · 2017-03-28T19:24:19Z

tools/mapping/gen_mapping_files/runoff_to_ocn/README

+     destination cells of the nearest neighbor map. Optionally, one can
+     generate a map that maps from all ocean cells, even if they are not
+     destination cells of the nearest neighbor map. This option allows the
+     smoothing map to be reused when mapping from multiple runoff grids onto


This description makes it sound like specifying this option is a convenience, and will give you the same final results (from step 3). Is that true? If so, it would help if you stated this explicitly.

mnlevy1981 · 2017-04-11T17:06:23Z

A preliminary test with mapping files generated by the updated tool seem to indicate that everything is okay, and I have updated the README file per @billsacks 's request so I think this is ready to go. Bill - I'm happy to keep tweaking the README file if you think it is still missing important information.

billsacks

This looks better now - thank you! A couple of comments on typos and one small section that could use some additional edits, then I think this is good to go.

billsacks · 2017-04-12T21:49:46Z

tools/mapping/gen_mapping_files/runoff_to_ocn/README

  eFold = smoothing eFold distance in meters (default: 1000000)
  rMax  = maximum radius of effect in meters (default: 300000)

-Settings:
+3. Settings
  title                 = ascii string to add to mapping files (default: 'unset')
  restrict_smooth_src_to_nn_dest = option to limit the srouce points for step2 to


typo: srouce -> source

billsacks · 2017-04-12T21:50:04Z

tools/mapping/gen_mapping_files/runoff_to_ocn/README

-Execute the binary ./runoff_map (again, this will read the runoff_map.nml
-namelist file - see above section for tips on creating the file)
+1) A nearest neighbor mapping from the rof grid to the ocean grid.
+   * For river runoff, it may be desireable to limit the destination grid


typo: desireable -> desirable

billsacks · 2017-04-12T21:50:56Z

tools/mapping/gen_mapping_files/runoff_to_ocn/README

+     so that runoff only enters at coastal points. The optional file_ocn_coast_mask
+     variable in the namelist allows you to specify two ocean grid files; one for
+     the full ocean and one for just the coast
+     variable can be used to point to an ocean grid 


Was this line (104) meant to be cut?

billsacks · 2017-04-12T21:51:46Z

tools/mapping/gen_mapping_files/runoff_to_ocn/README

+     variable in the namelist allows you to specify two ocean grid files; one for
+     the full ocean and one for just the coast
+     variable can be used to point to an ocean grid 
+   * The optional file_ocn_coast_mask file allows mapping to a subset of ocean


This bullet point seems to repeat some of the earlier one; can they be combined?

Review of the previous commit of this file found several typos (no major content changes)

mnlevy1981 · 2017-04-13T15:19:41Z

Thanks for the comments, @billsacks -- definitely helps to have fresh eyes read through the documentation :) Everything should be ready to go now.

Default and MPI-OpenMP PE Layouts for ne4 on Anvil: Added a default PE layout for A_WCYCL2000 ne4_Qu240 and a MPI-OpenMP 'L' layout for Anvil. [BFB]

mnlevy1981 added 18 commits March 20, 2017 16:17

Update smooth_mod to fix bug

398b2b3

The ocean -> ocean smooth map in the runoff to ocean map generator had a bug in it such that the ocean source point was not included in the list of destination points.

Updated common namelists

f8a2112

The four namelists that handle mapping between {r05,rx1} and {gx3v7,gx1v7} are up to date and point to the correct coastal mask files.

Add NCL script for merging maps

1f02e61

Also fixed a typo in a couple of the namelist files

Removed namelists that have not been updated

a067612

Three old namelists won't work without updating the namelist to point to a coastal ocean grid. Instead of doing that, I deleted the namelist (they can be recreated if needed later)

smooth_init now uses mapread_dest_grid

6876e16

Smooth map no longer assumes src and destination are the same (it maps from file_ocn_coastal_mask -> file_ocn_global_mask)

Update documentation on how to use this tool

21aee75

Remove SVN keywords from comments

3be0939

Things like $URL$ and $ID$ are out of date now that this code is kept in git

Formatting update

98164b4

Didn't need to create additional modules

c9b59c3

I thought mapread_mod and maptype_mod were necessary to avoid circular dependencies, but smooth_mod already depends on map_mod... so map_DestGridRead is in map_mod and I don't need to move the sparse matrix type out of map_mod either.

Merge branch 'master' into mapping/update_runoff_to_ocn

b35ec1d

mnlevy1981 requested a review from billsacks March 24, 2017 22:12

rljacob added the in progress label Mar 24, 2017

jedwards4b reviewed Mar 24, 2017

View reviewed changes

billsacks requested changes Mar 28, 2017

View reviewed changes

rljacob assigned billsacks Mar 31, 2017

Clean up comments based on pull request review

90fee43

billsacks requested changes Apr 12, 2017

View reviewed changes

More edits to README

6cff612

Review of the previous commit of this file found several typos (no major content changes)

billsacks approved these changes Apr 13, 2017

View reviewed changes

billsacks merged commit 5074c4a into ESMCI:master Apr 13, 2017

billsacks removed the in progress label Apr 13, 2017

mnlevy1981 deleted the mapping/update_runoff_to_ocn branch April 13, 2017 21:12

mnlevy1981 mentioned this pull request Apr 13, 2017

Update domain_b in nnsm map #1349

Merged

jgfouca pushed a commit that referenced this pull request Jun 2, 2017

Merge branch 'erichlf/anvil/A_WCYCL2000_ne4oQU240PEs' (PR #1280)

47922e2

Default and MPI-OpenMP PE Layouts for ne4 on Anvil: Added a default PE layout for A_WCYCL2000 ne4_Qu240 and a MPI-OpenMP 'L' layout for Anvil. [BFB]

billsacks mentioned this pull request Aug 23, 2017

New runoff -> ocean map #1243

Closed

jgfouca pushed a commit that referenced this pull request Feb 23, 2018

Merge branch 'erichlf/anvil/A_WCYCL2000_ne4oQU240PEs' (PR #1280)

e30e6c5

Default and MPI-OpenMP PE Layouts for ne4 on Anvil: Added a default PE layout for A_WCYCL2000 ne4_Qu240 and a MPI-OpenMP 'L' layout for Anvil. [BFB]

jgfouca pushed a commit that referenced this pull request Mar 13, 2018

Merge branch 'erichlf/anvil/A_WCYCL2000_ne4oQU240PEs' (PR #1280)

ca9d539

Default and MPI-OpenMP PE Layouts for ne4 on Anvil: Added a default PE layout for A_WCYCL2000 ne4_Qu240 and a MPI-OpenMP 'L' layout for Anvil. [BFB]

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mapping/update runoff to ocn #1280

Mapping/update runoff to ocn #1280

mnlevy1981 commented Mar 24, 2017 •

edited by rljacob

Loading

jedwards4b Mar 24, 2017

billsacks left a comment

billsacks Mar 28, 2017

billsacks Mar 28, 2017

billsacks Mar 28, 2017

mnlevy1981 commented Apr 11, 2017

billsacks left a comment

billsacks Apr 12, 2017

billsacks Apr 12, 2017

billsacks Apr 12, 2017

billsacks Apr 12, 2017

mnlevy1981 commented Apr 13, 2017

Mapping/update runoff to ocn #1280

Mapping/update runoff to ocn #1280

Conversation

mnlevy1981 commented Mar 24, 2017 • edited by rljacob Loading

Choose a reason for hiding this comment

billsacks left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mnlevy1981 commented Apr 11, 2017

billsacks left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mnlevy1981 commented Apr 13, 2017

mnlevy1981 commented Mar 24, 2017 •

edited by rljacob

Loading