Netcdf History Capability

[apcraig]

PR checklist

• Short (1 sentence) summary of your PR:
  Add netcdf history capability
• Developer(s):
  apcraig
• Suggest PR reviewers from list in the column to the right.
• Please copy the PR test results link or provide a summary of testing completed below.
  All tests pass bit-for-bit, https://github.com/CICE-Consortium/Test-Results/wiki/icepack_by_hash_forks#64afb11673798c9c1235e86403b2bcecb79a7abd
• How much do the PR code changes differ from the unmodified code?
  • bit for bit
  • different at roundoff level
  • more substantial
• Does this PR create or have dependencies on CICE or any other models?
  • Yes
  • No
• Does this PR add any new test cases?
  • Yes
  • No
• Is the documentation being updated? ("Documentation" includes information on the wiki or in the .rst files from doc/source/, which are used to create the online technical docs at https://readthedocs.org/projects/cice-consortium-cice/.)
  • Yes
  • No, does the documentation need to be updated at a later time?
    • Yes
    • No
• Please provide any additional information or relevant details below:

Add primitive netcdf history capability

• Add history_cdf namelist. This turns on netcdf history files. Netcdf history files are written for each column at each timestep for a hardcoded set of fields. There is no control of fields, frequency, or averaging. History filenames are also hardwired in the code.
• History netcdf files require the USE_NETCDF C Preprocessor flag. This is turned on by setting ICE_IOTYPE to netcdf in icepack.settings. In addition, the env and Macros files for the machine have to support builds with netcdf.
• Add icedrv_history.F90. Implements history capability.
• Add set_env.ionetcdf, set_nml.ionetcdf
• Add io_suite.ts

[apcraig]

The output fields are hardwired into the implementation. As a starting point, I included several fields from "state" and "flux". But I'm happy to remove/add any others that seem reasonable. The history file also includes trcr and trcrn in their raw multi-dimensional form. Maybe those should be split apart into subgroups of related tracers. But that would require dealing with the tracer indices.

This implementation does not provide any features like controlling fields, frequencies, columns, or time averaging. Users turn the file on or off, and the history file does NOT meet the CF conventions. I think that's a completely reasonable starting point. I see no reason to put a huge effort into this feature at this point. The netcdf output should be very useful in the proposed form for users that want it.

Again, happy to make some changes now, and we can allow it to evolve as needed. The netcdf header currently has

netcdf icepack.h.20150101 {
dimensions:
	ni = 4 ;
	ncat = 5 ;
	ntrcr = 21 ;
	time = UNLIMITED ; // (10248 currently)
variables:
	int ni(ni) ;
		ni:column_name_001 = "icefree" ;
		ni:column_name_002 = "slab" ;
		ni:column_name_003 = "full_ITD" ;
		ni:column_name_004 = "land" ;
	int ncat(ncat) ;
	int ntrcr(ntrcr) ;
	double time(time) ;
		time:long_name = "model time" ;
		time:units = "days since 2015-01-01 00:00:00" ;
		time:calendar = "NoLeap" ;
	int timestep(time) ;
	double date(time) ;
	double aice(time, ni) ;
	double vice(time, ni) ;
	double vsno(time, ni) ;
	double uvel(time, ni) ;
	double vvel(time, ni) ;
	double divu(time, ni) ;
	double shear(time, ni) ;
	double strength(time, ni) ;
	double evap(time, ni) ;
	double fsnow(time, ni) ;
	double frazil(time, ni) ;
	double fswabs(time, ni) ;
	double flw(time, ni) ;
	double flwout(time, ni) ;
	double fsens(time, ni) ;
	double fsurf(time, ni) ;
	double flat(time, ni) ;
	double frain(time, ni) ;
	double Tair(time, ni) ;
	double Qa(time, ni) ;
	double fsw(time, ni) ;
	double fcondtop(time, ni) ;
	double meltt(time, ni) ;
	double meltb(time, ni) ;
	double meltl(time, ni) ;
	double snoice(time, ni) ;
	double dsnow(time, ni) ;
	double congel(time, ni) ;
	double sst(time, ni) ;
	double sss(time, ni) ;
	double Tf(time, ni) ;
	double fhocn(time, ni) ;
	double aicen(time, ncat, ni) ;
	double vicen(time, ncat, ni) ;
	double vsnon(time, ncat, ni) ;
	double trcr(time, ntrcr, ni) ;
	double trcrn(time, ntrcr, ncat, ni) ;
}

[eclare108213]

@CICE-Consortium/devteam Would any of you be able to clone this branch and try running with the new netcdf output? You may need to update a machine file. Thx, e

[dabail10]

I'll give this a whirl.

[dabail10]

This compiles and runs fine for me on cheyenne. The netcdf data looks funny in ncview, but I'm not sure how to get around that. It looks like the history is written at every timestep? Should it follow diagfreq? It would also be nice to allow for netCDF restarts as well ... a separate issue.

[apcraig]

@dabail10, thanks for testing. The output is written at every timestep for now. I think that's fine. The files are small, so I think we can accept it for now. The initial goal is to provide something basic to the community. Lets see whether folks use it and then what features they request. Nobody has asked for restarts in netcdf format yet. Again, I think what we provide should be driven by community requests.

[rgrumbine]

netcdf restarts are something we've talked about for the UFS. We haven't made a serious request, but it's been mentioned in a 'would be nice' sense.

…

On Thu, Aug 26, 2021 at 11:58 AM Tony Craig ***@***.***> wrote: @dabail10 <https://github.com/dabail10>, thanks for testing. The output is written at every timestep for now. I think that's fine. The files are small, so I think we can accept it for now. The initial goal is to provide something basic to the community. Lets see whether folks use it and then what features they request. Nobody has asked for restarts in netcdf format yet. Again, I think what we provide should be driven by community requests. — You are receiving this because you are on a team that was mentioned. Reply to this email directly, view it on GitHub <#368 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFQ65HMIBYWKHDQZX6MMZQ3T6ZQB3ANCNFSM5AYZITAA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .

-- Tel: 301-683-3747 Coupling and Dynamics Group (he/him/his)

[apcraig]

@rgrumbine, this discussion is about Icepack standalone. The CICE model has netcdf restart capability already as well as much more control of history files. If you want to use netcdf restarts in CICE, you just need to turn it on. I don't know whether you can read in a binary restart file and write out a netcdf file. Hopefully so, but I'd need to test that to confirm. If not, that's a feature we could try to add if needed.

[eclare108213]

After fixing the machine files, this worked for me on badger. However it does not compile in my laptop conda environment. Not sure if that's a problem with my laptop or conda. @phil-blain @apcraig did one of you try it there?

icedrv_history.F90:54:10:

54 | use netcdf
| 1
Fatal Error: Cannot open module file 'netcdf.mod' for reading at (1): No such file or directory

[apcraig]

I have not tried conda, but I'll give it a try. We probably just have to add netcdf to the packages. Let me see what I can do.

[apcraig]

I just updated the conda_macos and it works for me. Hopefully, it'll work for others. You'll have to do

conda env update -f configuration/scripts/machines/environment.yml

to add the netcdf packages to the icepack environment. This is also updated in the documentation.

[eclare108213]

conda now works for me too.

[eclare108213]

@apcraig feel free to merge this when you're ready

[eclare108213]

@apcraig are you finished updating this PR?

[phil-blain]

Thanks @apcraig. I was on vacation last week so I missed this, but it might have been a good idea to also update the conda_linux port ?