Generic I/O layer for infra and netCDF #310
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## dev/gfdl #310 +/- ##
============================================
+ Coverage 37.11% 37.17% +0.06%
============================================
Files 263 265 +2
Lines 73814 74379 +565
Branches 13757 13819 +62
============================================
+ Hits 27394 27653 +259
- Misses 41357 41643 +286
- Partials 5063 5083 +20
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
Hallberg-NOAA
requested changes
Jan 26, 2023
There was a problem hiding this comment.
Overall this looks like a very nice clean-up of the MOM6 I/O interfaces. The use of Fortran classes and methods is novel in the MOM6 code, but it seems like a nice move into 21st century coding styles. Very nicely done, @marshallward!
After looking through all of the code changes, the only thing that I notice that should be fixed is that there are a number of missing doxygen descriptions for some of the functions in MOM_io_file.F90, and the doxygen description to create_MOM_file "Testing a new approach" probably needs to be updated to describe what that subroutine does. I also noted an occasional spelling error or term I was unfamiiar with in a comment (should "metata" be "metadata" on MOM_io_file.F90 line 460?). Once these very minor changes have been made, I think that this PR should be ready to go.
marshallward
force-pushed
the
oo_handle_axistype
branch
2 times, most recently
from
1d5a663 to
d7ac5c9
Compare
This patch introduces a generalized I/O interface, through a new class, `MOM_file`, for reading and writing of axis-based model fields. A common API for interacting with files is defined in the `MOM_file` class, and two implementations are provided: * `MOM_infra_file`, using the "infra" framework (i.e. FMS) * `MOM_netcdf_file`, using native netCDF framework This separation allows us to define certain generic functions by class, and platform-specific functions by type. It will also allow for removal of legacy FMS1 operations which are no longer used for the majority of I/O but are still required for isolated MOM files. This change will allow us to move incompatible files from the FMS to the netCDF implementation, while still preserving a common structure for both files. The majority of the details of `MOM_file` and its subclasses are defined in `MOM_io_file.F90`, which is exclusively accessed through `MOM_io.F90`. The netCDF implementation, designed to be used by `MOM_netcdf_file` but is designed as a standalone system, is defined in `MOM_netcdf.F90`. *Interface* `MOM_file` includes the following functions: * `open` * `close` * `flush` * `register_axis` * `register_field` * `write_attribute` * `write_field` * `file_is_open` * `get_file_info` * `get_file_fields` * `get_field_atts` * `read_field_chksum` Most are designed to resemble the existing MOM I/O operations. Note that some of these have not yet been implemented for `MOM_netcdf_file`, since they are never used in the model. `MOM_file_infra` includes the following additional operations: * `get_file_times` * `get_file_fieldtypes` See documentation for usage of these functions. The "axis"/"field" model from FMS1 has been preserved, where axes are associated with variables which contain the grid point, and fields are defined with respect to the file's axes. Operations such as field counters will exclude any variables associated with the axes. The `axistype` and `fieldtype` from FMS have been replaced with new abstractions (`MOM_axis` and `MOM_field`). Internally, these point to either the FMS types or equivalent netCDF types. *Implementation* Each file type contains an instance of its native type, as well as lists of associated axes and fields. List are implemented as linked lists of names (stored as `label`) which are used to identify and extract or write the axis/field to the internal type. The mechanics of this are largely hidden from the user and can be changed in the future, if needed, without disruption to the rest of the codebase. The current netCDF implementation very closely mirrors the FMS infra behavior, but this can be relaxed or modified as needed. netCDF I/O is currently only designed for serial I/O, with all of the data on the root PE. Further development would be needed to support any kind of parallel I/O. *Current Usage* Two functions have been transferred from infra to the netCDF I/O: * `Depth_list.nc` * `ocean.stats.nc` The following legacy functions and types from FMS have been preserved: * `create_file` * `reopen_file` * `file_type` * `open_file` * `get_file_info` * `get_file_fields` * `get_file_times` This is primarily to preserve compatibility with SIS2, but may also be useful for other code, such as model drivers. These may be phased out in the future, however.
marshallward
force-pushed
the
oo_handle_axistype
branch
from
d7ac5c9 to
1f689b7
Compare
Hallberg-NOAA
approved these changes
Jan 26, 2023
|
This PR has passed pipeline testing at https://gitlab.gfdl.noaa.gov/ogrp/MOM6/-/pipelines/18083. |
This was referenced Apr 6, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This patch introduces a generalized I/O interface, through a new class,
MOM_file, for reading and writing of axis-based model fields.A common API for interacting with files is defined in the
MOM_fileclass, and two implementations are provided:MOM_infra_file, using the "infra" framework (i.e. FMS)MOM_netcdf_file, using native netCDF frameworkThis separation allows us to define certain generic functions by class, and platform-specific functions by type. It will also allow for removal of legacy FMS1 operations which are no longer used for the majority of I/O but are still required for isolated MOM files.
This change will allow us to "remove" such files from the FMS system and reimplement them with native MOM operations, while still preserving a common structure for both files.
The majority of the details of
MOM_fileand its subclasses are defined inMOM_io_file.F90, which is exclusively accessed throughMOM_io.F90. The netCDF implementation, designed to be used byMOM_netcdf_filebut is designed as a standalone system, is defined inMOM_netcdf.F90.Interface
MOM_fileincludes the following functions:opencloseflushregister_axisregister_fieldwrite_attributewrite_fieldfile_is_openget_file_infoget_file_fieldsget_field_attsread_field_chksumMost are designed to resemble the existing MOM I/O operations.
Note that some of these have not yet been implemented for
MOM_netcdf_file, since they are never used in the model.MOM_file_infraincludes the following additional operations:get_file_timesget_file_fieldtypesSee documentation for usage of these functions.
The "axis"/"field" model from FMS1 has been preserved, where axes are associated with variables with contain the grid point, and fields are defined with respect to the file's axes. Operations such as field counters will exclude any variables associated with the axes.
The
axistypeandfieldtypefrom FMS have been replaced with new abstractions (MOM_axisandMOM_field). Internally, these point to either the FMS types or equivalent netCDF types.Implementation
Each file type contains an instance of its native type, as well as lists of associated axes and fields. List are implemented as linked lists of names (stored as
label) which are used to identify and extract or write the axis/field to the internal type. The mechanics of this are largely hidden from the user and can be changed in the future, if needed, without disruption to the rest of the codebase.The current netCDF implementation very closely mirrors the FMS infra behavior, but this can be relaxed or modified as needed.
netCDF I/O is currently only designed for serial I/O, with all of the data on the root PE. Further development would be needed to support any kind of parallel I/O.
Current Usage
Two functions have been transferred from infra to the netCDF I/O:
Depth_list.ncocean.stats.ncThe following legacy functions and types from FMS have been preserved:
create_filereopen_filefile_typeopen_fileget_file_infoget_file_fieldsget_file_timesThis is primarily to preserve compatibility with SIS2, but may also be useful for other code, such as model drivers. These may be phased out in the future, however.