fix stop_option=date, move neon niwo test to match user_mods

[jedwards4b]

Description of changes

Move the NEON_NIWO test to NEON/NIWO to match the names in usermods

Specific notes

Contributors other than yourself, if any:

CTSM Issues Fixed (include github issue #):
Fixes #1412

Are answers expected to change (and if so in what way)? No

Any User Interface Changes (namelist or namelist defaults changes)?

Testing performed, if any:
Hand testing of SMS_D_Vnuopc_Mmpi-serial.CLM_USRDAT.I1PtClm51Bgc.cheyenne_intel.clm-NEON-NIWO
Also ran cmeps testlist and cdeps testlist.

[jedwards4b]

Fixes #1412

[wwieder]

address this NEON_NCAR issue NEONScience/NCAR-NEON#39

[ekluzek]

I have a couple small changes requested. Let me know if you want to chat about the NEON test mods. I think the hierarchy should look like:

default
NEON -- include default
NEON/NIWO -- include NEON + user_mod/NEON/NIWO

Using the default test-mods will have output more frequently for short tests, turn on the B4BFLAG as you did, as well as fire-emissions.

[jedwards4b]

Needed to add externals and change .config_files.xml for the new cime directory structure.

[ekluzek]

@wwieder do you still want something that addresses NEONScience/NCAR-NEON#39 here? I think this does include downloading data to $RUNDIR, so partially addresses that. And I think that's sufficient for now. There might be a longer term issue that comes in later that's more comprehensive. But, is the current solution of downloading to $RUNDIR sufficient for now?

[wwieder]

@ekluzek I'm happy with how this is being handled now.

[wwieder]

The one thing I'll add its that it's potentially for users to know about this file, which lists all the sites and data available.
https://neon-ncar.s3.data.neonscience.org/listing.csv
Maybe this just goes into our documentation for how to run NEON sites.
Maybe it's time to start making a wiki or notebook for this! @negin513 and @danicalombardozzi what do you think?

[jedwards4b]

Note that the most current version of that file available at the time the case is created will be in the case run directory.

…

On Thu, Jul 8, 2021 at 2:58 PM will wieder ***@***.***> wrote: The one thing I'll add its that it's potentially for users to know about this file, which lists all the sites and data available. https://neon-ncar.s3.data.neonscience.org/listing.csv Maybe this just goes into our documentation for how to run NEON sites. Maybe it's time to start making a wiki for this! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1411 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABOXUGFRHSA4R2S55NGRPFTTWYGN7ANCNFSM47KJRBTQ> .

-- Jim Edwards CESM Software Engineer National Center for Atmospheric Research Boulder, CO

[wwieder]

oh, I didn't realize that. very good. thanks, @jedwards4b

[jedwards4b]

@ekluzek I have updated the test in testlist_clm.xml to the new testname format and tested the change.
I am currently creating tags for the externals and will update Externals.cfg when that is done.

[jedwards4b]

@negin513 just waiting for new surface datasets.

[negin513]

For testing:
The original dataset (subset from the global) for both 78 pfts and 16 pfts are in: /glade/scratch/negins/single_point
The updated dataset using NEON data is in: /glade/scratch/negins/single_point_neon_updated
There is a bug in my code for updating the surface dataset using neon and that is why the updated surface dataset list is not complete. 
I will get back to you on the complete list later today.

[wwieder]

Hi Negin, Let's prioritize getting the modified surface datasets up for Jim to point to that modify soil properties based on NEON observations. We can work on creating surface datasets for two crop sites later. Does this make sense?

…

On Thu, Jul 15, 2021 at 9:21 AM Negin Sobhani ***@***.***> wrote: For testing: The original dataset (subset from the global) for both 78 pfts and 16 pfts are in: /glade/scratch/negins/single_point The updated dataset using NEON data is in: /glade/scratch/negins/single_point_neon_updated There is a bug in my code for updating the surface dataset using neon and that is why the updated surface dataset list is not complete. I will get back to you on the complete list later today. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1411 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5IWJDMYL33ER7WDMBXDWDTX34HVANCNFSM47KJRBTQ> .

[jedwards4b]

@negin513 We use a wildcard in the name of the file: surfdata_hist_78pfts_CMIP6_simyr2000_${NEONSITE}_c210715.nc

it would sure be easier if you could create all of the files with the same date stamp. (some are 14 others are 15)

[ekluzek]

@billsacks this does use the new test syntax so it does need beyond cime6.0.0. Since, @jedwards4b has done some testing I think we should try it out and see if we run into any other problems with it. If we can get a cime6.0.3 tag that might be the best. I'll ask about that.

[billsacks]

I think we should try it out and see if we run into any other problems with it. If we can get a cime6.0.3 tag that might be the best. I'll ask about that.

I'm fine with that plan if you are. I can make a cime tag since Chris is out.

[jedwards4b]

Sounds good - thanks Bill.

…

On Thu, Jul 15, 2021 at 3:34 PM Bill Sacks ***@***.***> wrote: I think we should try it out and see if we run into any other problems with it. If we can get a cime6.0.3 tag that might be the best. I'll ask about that. I'm fine with that plan if you are. I can make a cime tag since Chris is out. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1411 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABOXUGAU5ZLV2CV27NRJZ5TTX5H53ANCNFSM47KJRBTQ> .

-- Jim Edwards CESM Software Engineer National Center for Atmospheric Research Boulder, CO

[billsacks]

By the way: it looks like cime6.0.2 isn't sufficient if you want to use the new test syntax in this PR, since it came in post-cime6.0.2 - so the upcoming cime6.0.3 will be required for this.

[ekluzek]

@negin513 note the files under $DIN_LOC_ROOT/lnd/clm2/surfdata_map/NEON/surfdata_hist_78pfts_CMIP6_simyr2000_*_c210715.nc are in NetCDF4 format and we need CESM inputdata files to be in any other NetCDF format. So can you change the script to put them in a different format? Go ahead and do that when you get a chance. Thanks.

[jedwards4b]

Good catch Erik sorry i missed it. Nccopy -k cdf5 should work.

…

On Sat, Jul 17, 2021, 11:18 Erik Kluzek ***@***.***> wrote: @negin513 <https://github.com/negin513> note the files under $DIN_LOC_ROOT/lnd/clm2/surfdata_map/NEON/surfdata_hist_78pfts_CMIP6_simyr2000_*_ c210715.nc are in NetCDF4 format and we need CESM inputdata files to be in any other NetCDF format. So can you change the script to put them in a different format? Go ahead and do that when you get a chance. Thanks. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1411 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABOXUGH4OMCLAABIVIZ65PLTYG3PPANCNFSM47KJRBTQ> .

[ekluzek]

@jedwards4b yes rimport caught if for me. I could do the nccopy easy enough, but I figure we want to modify the creation script anyway, and want to get it corrected. And we want the corrected script in this next tag anyway as well. So I'm working with @negin513 on making that happen.

[negin513]

@ekluzek
As we discussed, I update the script to write out the output in NETCDF3_64BIT instead of netcdf4.

All the data is ending with *_c210720.nc :

The original dataset (subset from the global) for 78 pfts in NETCDF3_64BIT : /glade/scratch/negins/single_point/

The updated dataset using NEON data in NETCDF3_64BIT: /glade/scratch/negins/single_point_neon_updated

The updated script for creating the data : #1375

Please let me know if there is anything else needed from my side.

[ekluzek]

OK, tests pass as expected on izumi. I have a few more tests on cheyenne that are running. The unit tests are failing, because the directory structure was changed. But, I should be able to fix that and get it working pretty easily.