-
Notifications
You must be signed in to change notification settings - Fork 318
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Modify mesh mask with new mesh_mask_modifier tool #1677
Conversation
Allow user to replace vegetation in fsurdat files with any pft/cft using the fsurdat_modifier tool option dom_plant. This option replaces now obsolete option dom_nat_pft which handled pfts only and not cfts.
As written, it's very slow because I didn't know off the top of my head how to map between the 1d mesh file and the 2d landmask without using for loops...
@billsacks @ekluzek @adrifoster and I talked at stand-up about the slowness of my code. A few ideas came up:
|
Thanks for the summary, Sam. To clarify on this:
I think both are built on top of numpy, so I'd think that general whole-array operation techniques that work in numpy would work here as well. If there were some function (in xarray or elsewhere) that was geared specifically for this kind of use (finding the overlapping points between two spatial domains) then all the better. But if it's hard to find a way to cast this in terms of existing functions or whole-array operations, then doing it in Fortran might be the best bet. |
Hey all, I was thinking about this more and it might be easier to think about this outside of xarray. I would think (?) there should be a way to do something faster if you just pull out the latitudes and longitudes into np arrays. Or perhaps, as @billsacks suggested as I was chatting with him, 1 big np array where the rows (for example) are the lats/lons from 1 file and the columns are lats/lons from the other file. Then you may be able to do some whole-array operations to get a matrix of T/F. I was messing around with an example of this but am having to get back to my regularly scheduled manuscript writing for the week... I will think on this more if I have time and report back with any other ideas! But since @slevisconsulting you know much more about the requirements for this function, perhaps these ideas aren't tractable. |
Thank you @adrifoster @billsacks for your comments. I expect to return to this around the middle of next week. (Still distracted with the move.) |
Previous version took several hours to look up matching lats/lons in nested for-loops. This version takes 2-2.5 minutes by assuming that 1d mask is in same south-to-north west-to-east order as the 2d mask.
Add outputs for annual crop sowing and harvest dates Added annual outputs of sowing and harvest dates (`SDATES` and `HDATES`, respectively). This should simplify the determination of sowing and harvest date for postprocessing. - Sowing dates are on new dimension `mxgrowseas` (maximum number of growing seasons allowed to begin in a year; currently hard-coded to 1). - Harvest dates are on new dimension `mxharvests` (maximum number of harvests allowed in a year), which is `mxgrowseas`+1. This is needed because in year Y you might harvest a field that was planted in year Y-1, then plant and harvest again. - The lengths of these dimensions are public constants of `clm_varpar`. Additionally, removed `cropplant` as discussed here: <ESCOMP#1500 (comment)>. The `mxharvests` concept enables the addition of more such outputs to further simplify crop postprocessing—for example, yield per growing season as a direct output rather than needing to cross-reference daily grain mass against the day of harvest. These changes involved some rework to the crop phenology code that changes answers for crop phenology in some circumstances. All answer changes were determined to be due to issues / oddities in the old code. See discussion in ESCOMP#1616 for details (especially see ESCOMP#1616 (comment)). To summarize briefly, some issues with the old code were: - Non-winter-cereal patches that had live crops at the beginning of the year did not get planted later that year. - There was some odd behavior for rice patches at exactly 0 deg latitude - Crop root depth had unexpected values outside the growing season; now root depth is set to 0 outside the growing season
Update comments accordingly
@ekluzek I'm initially asking for your review and would also be happy if you have additional reviewers in mind. This is a relatively simple tool and development has stopped at this point. It's probably close to ready. |
The tool already has user instructions, but I just now decided to also add detailed instructions for a specific use-case, using text that I previously included in Isla Simpson's corresponding google doc. |
Mostly I moved text from the tool's docstring to the README and made a note of that in the tool's docstring.
@slevisconsulting I have two thoughts here after starting to use it. The first is:
|
@slevisconsulting there are still some checkboxes that should be addressed. You did add some unit testing, so I'll check off that one, but the other two should be handled. |
I think I have addressed what we discussed. Just for the record: when I'm in the ncar_pylib environment, the new test's nco commands work without a |
Made test links on cheyenne and izumi, so that now both dev105 and dev106 link to dev104: |
@ekluzek I thought I'd test on izumi but ran into this problem:
Is this user error on my part or would you like me to open an issue? ADDITIONAL INFO
and got a malformatted .nc file as in #1658 but fixed the problem by unloading python: Next (out of curiosity) I tried |
@slevisconsulting go ahead and create an issue about this. I just tried on izumi on your branch and I don't see this problem. But, that means there is likely some environment issue that we need to track down. So having your problem clearly identified in an issue would help track it down. We need this to work for everyone everywhere so it's important to track down these user environment issues. |
Connect ozone from atmosphere Allow atmospheric input of ozone partial pressure (monthly, mol/mol) to CTSM and use this ozone in the OzoneMod module. Specific notes: Add ozone to atm2lndType and also add as an output history variable (ATM_O3). Read in O3 in lnd_import_export. Connect to the OzoneMod module: in CalcOzoneUptakeOnePoint we now use this input ozone, rather than a static parameter.
cheyenne testing I made links on both cheyenne and izumi from the dev109 to the dev110 baselines. |
Conflicts: doc/ChangeLog doc/ChangeSum
@slevisconsulting it looks like there is one checkbox that isn't completed. See above for this July/15th comment that has one checkbox that's open.
Can you take care of that one? It just needs a functional example of a config file that can be run to test it. |
Ok @slevisconsulting and I discussed the working config file. This was a requirement just to make sure testing was in place. There is a system test in place that creates a working config file. So I'm removing the requirement since the test sys covers it. |
@ekluzek here's notes from our discussion just now:
|
@slevisconsulting one thing I realized we should update at some point. You point to a test file in the mesh_modifier tool and it's README. /glade/work/slevis/git/mksurfdata_toolchain/tools/modify_input_files/islas_examples/modify_mesh_mask/fill_indian_ocean/fill_indianocean_slevis.nc Since it's a personal path we can't rely on that staying around. So we should either add it to python/ctsm/test/testinputs or put it in the inputdata repository somewhere. Or remove the mention of it. Personal paths to specific files on specific machines aren't useful in the long run, we should try to avoid mentioning them. We can do this after I merge this, but I wanted to add a note about it now. |
Description of changes
mesh_mask_modifier is a tool that modifies mesh_mask files. It reads the mesh_mask file and outputs a modified copy of the file.
mesh_mask_modifier will complement the fsurdat_modifier tool, which modifies fsurdat files. When a user introduces land in place of ocean grid cells in their fsurdat file, they need to modify the mesh_mask file for their fsurdat modifications to take effect in a simulation.
Specific notes
Contributors other than yourself, if any:
@ekluzek
CTSM Issues Fixed (include github issue #):
Fixes #1644
Are answers expected to change (and if so in what way)?
N/A ...new tool
Testing performed, if any:
Test run in progress. Typed:
./mesh_mask_modifier modify_slevis.cfg
in /glade/work/slevis/git/mksurfdata_toolchain/tools/modify_mesh_mask