Domain - Create netCDF Files #383
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
N1ckP3rsl3y
commented
Nov 13, 2023
N1ckP3rsl3y
commented
- Second part of moving to netCDF output
- If a domain netCDF has not been provided, generate one to future runs
- See Capability to create domain netCDF #362
- These files will deal with functionality in regards to netCDF inputs/outputs
- Create multiple stubs for functions that will be used in the branch `feature_nc_domain` - Silence warnings for the time being until they are filled `SW_NC_read_domain()` - Reads domain information and makes sure its consistent with the domain input file (domain.in) `SW_NC_check()` - Checks that the information is consistent with domain (domain.in) `SW_NC_create_domain_template()` - Creates a template domain for users to fill (will exit the program early for the user to fill) `SW_NC_create_template()` - Copy domain netCDF that will later be modified to contain values for a specific output variable `SW_NC_read_domain()` - Reads the given inputs from input netCDF's for specific variables `SW_NC_check_input_files()` - Make sure all given input files have consistent spacial data with the domain (domain.in)
- Filled in `SW_NC_create_domain_template()` to create basic dimensions and variables (domain, site, x, y, x_bnds, y_bnds, bnds) - New local helper functions * `create_netCDF_dim()`: Simply creates a dimension using the C-netCDF library and detects a failure if one occurs * `create_netCDF_var()`: Simply creates a variable using the C-netCDF library and detects a failure if one occurs * `fill_domain_netCDF_s()`: Creates all basic dimensions and variables that are specific to the domain type "s" * `fill_domain_netCDF_xy()`: Creates all basic dimensions and variables that are specific to the domain type "xy" - PATH_INFO gains variable to store the domain netCDF file's ID in regards to the C-netCDF library `SW_CTL_setup_domain()` gains #ifdef for SW_NETCDF and detects if "domain.nc" exists * Currently, this only cares if the file does not exist, if this is the case, the domain template will be created * Checks if an error occurred in `SW_NC_create_domain_template()` and returns * If no error occurred, the function returns after so that the user can modify the template
- This directory will be used to hold all netCDF-related input files - Add netCDF input file in `files.in` - Update file indices `SW_Files.c/h`
- Add functionality to read in `Input_nc/files_nc.in` - Update file to have domain for now - Update file description to include "key-value-value triplet" instead of "key-value pair" - New functions * `SW_NC_read()` - Read key-value-value triplets within the file - Only supports domain currently * `nc_key_to_id()` - Convert a key (e.g., "domain") into an ID in the program (i.e., index) - `SW_netCDF.c` gains local define to specify the max number of keys the file (files_nc.in) can contain - `SW_netCDF.h` gains new define - DOMAIN_NC - for new arrays in SW_DOMAIN and PATH_INFO * Two new arrays to hold - 1) netCDF input file names/paths - 2) Variable names within the netCDF file to read - `SW_CTL_setup_domain()` calls `SW_NC_read()` if SW_NETCDF is defined
- Function now goes through input files and and compares the values with `domain.in`
- Function does not need domain netCDF name - the name tested/created is `domain_template.nc` - New define in SW_netCDF.h for `domain_template.nc` - Remove quotation marks from `files_nc.in` around the new file path
- if `CPPFLAGS` defines "SWNETCDF", then compile SOILWAT2 with netCDF support * a working external netCDF library is required * makefile looks up `nc-config` flags and libraries to include (unless specified via "NC_CFLAGS" and "NC_LIBS" on the command line) * makefile also compiles "SW_netCDF.c" * code only includes functions from "SW_netCDF" if "SWNETCDF" is defined -> `CPPFLAGS=-DSWNETCDF make all` or `CPPFLAGS=-DSWNETCDF make test` * if SOILWAT2 is compiled without "SWNETCDF", then there is no dependency on netCDF
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## release/devel_v8.0.0 #383 +/- ##
========================================================
- Coverage 71.00% 70.68% -0.33%
========================================================
Files 21 21
Lines 5591 5631 +40
========================================================
+ Hits 3970 3980 +10
- Misses 1621 1651 +30 ☔ View full report in Codecov by Sentry. |
- New struct that holds information dealing with netCDF CRS (Coordinate Reference System) - Holds attributes for geographic and projected types - Holds attributes that may be present during a projected type
- Holds constant information dealing with netCDFs - Uses newest struct SW_CRS to hold CRS information for geographic and projected types - PATH_INFO no longer holds `InFilesNC` - SW_DOMAIN no longer holds `varNC` but gains instance to SW_NETCDF
- SW_DOMAIN gains new variables * crs_bbox - Name/type of the CRS * min_x/min_y - Minimum x/y coordinate value * max_x/max_y - Maximum x/y coordinate value - The user can now input the minimum/maximum x/y/lat/long coordinates for a bounding box in domain.in - The user can now input the name of their CRS - `SW_DOM_read()` now reads in these values and can hold up to 12 characters for the value in key-value pair - `domain_inkey_to_id()` now accepts the new keys in domain.in
- `create_netCDF_*()` replace domain-specific variable names with general names - `create_netCDF_*()` no longer use `snprintf()` due to the same functionality happening in `LogError()` - PATH_INFO now contains an array of input netCDF file IDs instead of a single one for domain - accommodating for more input files in the future - The name of the domain variable is now sent into `create_netCDF_var()` through varNC within SW_NETCDF
- New input file with the directory "attributes_nc.in" * Purpose: - User can specify the desired global attributes of a netCDF file - User can specify the attributes of the CRS variable which will be put in generated netCDF files - `files.in` gains `attributes_nc.in` in it's file listings - Enums/indices within `SW_Files.c/h` are adjusted for this new listing - SW_datastructs.h - - Incremented number of in files - Corrected `crs_wkt` and `primary_crs` in SW_NETCDF to be a pointer - SW_netCDF.c - - New function `nc_read_atts()` * Called from `SW_NC_read()` * Reads in `attributes_nc.in` and fails if it cannot be found * Uses a set of possible keys to search for - both geographic and projected * Fails if the function only finds a projected CRS - New local define - NUM_ATT_IN_KEYS - which is the number of possible keys in `attributes_nc.in` - Updated `nc_key_to_id()` to take in a list of possible keys and the list size to make it more general - Updated the interface of `SW_NC_read()` to accept new SW_NETCDF struct
dschlaep
reviewed
Nov 29, 2023
* adjusted maximum key length from 29 to 34 (e.g., key = "proj_latitude_of_projection_origin") * limit string that `sscanf()` reads/assigns to key to 34 characters (plus null-terminator) * if key is (projected crs) standard parallel(s), then the user may provide one or two values (that may be separated by white-space, coma, etc.) ** use NAN to indicate that user provided one value for the standard parallel
- Function can take a key, a set of possible keys, and the number of possible keys that any function within (currently) SW_Domain.c and SW_netCDF.c can use - Removed the functions `*_key_to_id()` - Replaced calls to `*_key_to_id()` with the new function
- New define - LARGE_VALUE (1024) meant to store large "values" when reading in keys - Updated domain and netCDF functions to use LARGE_VALUE for input buffers
- Horizontal coordinate variables that are written to in the domain netCDF now depend on the CRS bounding box user input - Mostly replaced the checking of if the main CRS is a geographic one, we now get the bounding box CRS - Make sure the CRS bounding box name and one CRS long name match - Simplify the interfaces of functions that create/fill the variable (values) to use SW_DOMAIN - Update the documentation note of the function `fill_domain_netCDF_vals()` - Update the function mentioned in the call to `Mem_Malloc()` within `alloc_netCDF_domain_vars()`
- Fill the function `SW_NC_read_inputs()` to read the latitude/longitude from a file (currently only domain.nc) * Created a helper function - `get_single_double_val()` - which gets a single double value from a netCDF file - New functions * `SW_NC_open_files()` - Open files (currently only domain.nc) if they exist to read from * `SW_NC_close_files()` - Close all files that were left open during the program run (currently only domain.nc) - Changed the order of the calculated/translated SUID in `SW_DOM_calc_ncSuid()` for the netCDF library to properly use
- Cast "allocArr" to void - Change name "numFreeVars" to "numAllocVars" to be more appropriate
- Use the C-netCDF provided function `nc_def_var_deflate()` with the compression level of 5 (0 - 9)
- Change this name to be consistent with the rest of the programs SW_* argument names
- Adjust whitespace in some places to try to help with readability - Correct/modify some function documentation - Attempt to make long function calls fit within 80 characters - Make it so that all trinary operators have () around the conditional
- No longer set the pointer to the template when there is only one grid cell/site - It was deemed superfluous due to the speed of copying an instance of SW_ALL
- new `set_hasKey()` marks a found key as found and warns if duplicate - new `check_requiredKeys()` throws error if a required key was not found - functions that process key-value input files gained two variables * "requiredKeys" that marks which of the "possibleKeys" is required (and which are optional) * "hasKeys" that is updated with found keys - functions that process key-value input files now call * `set_hasKey()` after each `key_to_id()`, and * `check_requiredKeys()` after input file is complete
- small updates/fixes to documentation and code comments - `SW_CTL_setup_domain()`: if SWNETCDF, then open input netCDF files only if the domain netCDF file exists - `SW_CTL_run_sw()` no longer calls `SW_MDL_get_ModelRun()` (and thus loses argument "ncInFiles") -- `SW_MDL_get_ModelRun()` is called before `SW_CTL_run_sw()` runs; those values are then copied over by `SW_ALL_deepCopy()`; this renders a second call to `SW_MDL_get_ModelRun()` by `SW_CTL_run_sw()` superfluous
dschlaep
reviewed
Jan 3, 2024
dschlaep
reviewed
Jan 3, 2024
- Move call to `SW_NC_close_files()` to `SW_DOM_close_files()` since they will always need to be closed unlike the use of `SW_OUT_close_files()`
- previously, the variable for the Y-axis was filled decreasingly; this caused problems with the presence of bounds some downstream software
…nate variables - our "domain" contains XY-axes (dimensions) - horizontal coordinate variables along those axes depend on the CRS: XY-axes are represented by lat and lon (if geographic), or by x and y (if projected) - we populate horizontal coordinate variables of the domain template netCDF that match the CRS of the provided domain bounding box; hence, we only ever calculate values for one or the other set (but not both)
This was
linked to
issues
Jun 6, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.