-
Notifications
You must be signed in to change notification settings - Fork 132
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add JRA55 dataset tool and documentation #582
Conversation
Add configuration/tools/jra55_datasets directory and scripts Move convert_restarts to configuration/tools/cice4_restart_conversio Add tools section to the developer guide Update tested compiler versions Change "compliance" to "validation" in documentation when discussing QC testing.
I will need to review the documentation once generated and probably need to do some cleanup. I leveraged much of the existing documentation (in Word), but didn't bring in all the figures and some other details. Please let me know if you feel anything is missing! I'm happy to revise a few times. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
when the compliance test or compliance test scripts (i.e., ``cice.t-test.py``) are modified. | ||
Again, this section **documents a validation process for the compliance scripts**; it does not | ||
describe to how run the compliance test itself. | ||
The code validation (QC) test validates non bit-for-bit model changes. The directory |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Define what QC is, if it's not defined above, e.g. "The code validation (Quality Control, QC) test..."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch, I have updated the documentation to mix validation and QC better. I was looking at the documentation and got confused about the word compliance. "code compliance" sounded like coding syntax, format, interfaces, and similar. That's why I switched to "code validation".
|
||
Two scripts have been created to automatically validate the code compliance script. | ||
Two scripts have been created to automatically validate the code validation script. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Instead of "validate the code validation script", maybe "validate the QC script"? This is probably why we used 'compliance' before. (also in other files)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See above.
|
||
The "Data access" tabs will provide a list of product categories. | ||
The JRA55 data of interest are located in 2 separate products. Winds, air | ||
temperature, and specific humidity on the model grid are included in "JRA-55 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
not on the model grid... on the JRA55 grid?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I cleaned that up.
The updated documentation appears here in readthedocs, https://cice-consortium-cice--582.org.readthedocs.build/en/582/developer_guide/dg_tools.html. I do have one question for @dabail10, @rallard77, @daveh150 with regard to requirements. The word document mentioned argparse and datetime as required python packages. Is that still the case. When I look at my packages on cheyenne, I don't actually see those listed. Are any other packages needed that are not listed in the document? This is what we have now, I'd like to make sure we have the list correct.
|
Hi Tony,
I think argparse and datetime are part of the standard Python Library (historically they weren’t). They probably do not need to be explicitly listed.
Thanks,
David
From: Tony Craig ***@***.***>
Sent: Tuesday, March 23, 2021 11:24 AM
To: CICE-Consortium/CICE ***@***.***>
Cc: David Hebert, Code 7322 ***@***.***>; Mention ***@***.***>
Subject: Re: [CICE-Consortium/CICE] Add JRA55 dataset tool and documentation (#582)
The updated documentation appears here in readthedocs, https://cice-consortium-cice--582.org.readthedocs.build/en/582/developer_guide/dg_tools.html.
I do have one question for @dabail10 <https://github.com/dabail10> , @rallard77 <https://github.com/rallard77> , @daveh150 <https://github.com/daveh150> with regard to requirements. The word document mentioned argparse and datetime as required python packages. Is that still the case. When I look at my packages on cheyenne, I don't actually see those listed. Are any other packages needed that are not listed in the document? This is what we have now, I'd like to make sure we have the list correct.
* python3 (python3.7.9)
* numpy (1.18.5)
* netCDF4 (1.5.5)
* ESMPy (8.0.0)
* xesmf (0.3.0)
* argparse
* datetime
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#582 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE52VPHAEBG3RAIDDGYF63DTFC6DZANCNFSM4ZUFVQBQ> . <https://github.com/notifications/beacon/AE52VPGZBTREFIFDJ7MHQZDTFC6DZA5CNFSM4ZUFVQB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOF757S7Y.gif>
|
Tony,
If I understand correctly, in ice_forcing.F90, the code first reads all the data into fsnow. Then the code calls subroutine prepare_forcing. In that subroutine it checks for Tair and changes from fsnow to frain if Tair is above freezing.
Let me know if that is not correct. Thanks!
Dave
From: Tony Craig ***@***.***>
Sent: Tuesday, March 23, 2021 11:22 AM
To: CICE-Consortium/CICE ***@***.***>
Cc: David Hebert, Code 7322 ***@***.***>; Mention ***@***.***>
Subject: Re: [CICE-Consortium/CICE] Add JRA55 dataset tool and documentation (#582)
@apcraig commented on this pull request.
_____
In doc/source/developer_guide/dg_tools.rst <#582 (comment)> :
+Precipitation and downward radiation fluxes are found in "JRA-55 3-Hourly
+Model Resolution 2-Dimensional Average Diagnostic Fields". (Note the
+difference between instantaneous and averaged data products. There are several
+JRA55 datasets available, you will likely have to scroll down the page to find
+these datasets.) Data are also available on a coarser 1.25° grid, but the tools
+are best used with the native TL319 JRA55 grid.
+
+The fields needed for CICE are
+
+- specific humidity (3-hourly instantaneous), Qa
+- temperature (3-hourly instantaneous), Tair
+- u-component of wind (3-hourly instantaneous), uatm
+- v-component of wind(3-hourly instantaneous), vatm
+- downward longwave radiation flux (3 hourly average), flw
+- downward solar radiation flux (3 hourly average), fsw
+- total precipitation (3 hourly average), fsnow
I had a look. The total precip is treated as snow in the model regardless of air temperature or season. There is no disaggregation into rain and snow based on temperature or anything else. I added a note in the documentation about this. I will not judge whether this is reasonable or not.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#582 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE52VPEEATO7FO2FAA6P74LTFC52BANCNFSM4ZUFVQBQ> . <https://github.com/notifications/beacon/AE52VPH2MCY6D7DLYCPLO2LTFC52BA5CNFSM4ZUFVQB2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOETRCRAY.gif>
|
Also, this version of the code does not use ESMPy. It only uses xESMF. I'll have more of a look at the documentation. |
Dave,
From the xESMF documentation: xESMF uses ESMPy as a backend. So I think ESMPy is also required. When xESMF is installed it probably checks for that dependency.
xESMF: Universal Regridder for Geospatial Data — xESMF 0.3.0 documentation <https://xesmf.readthedocs.io/en/latest/>
Thanks,
David
From: David A. Bailey ***@***.***>
Sent: Wednesday, March 24, 2021 10:21 AM
To: CICE-Consortium/CICE ***@***.***>
Cc: David Hebert, Code 7322 ***@***.***>; Mention ***@***.***>
Subject: Re: [CICE-Consortium/CICE] Add JRA55 dataset tool and documentation (#582)
Also, this version of the code does not use ESMPy. It only uses xESMF. I'll have more of a look at the documentation.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#582 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE52VPG24REI2PRX5QAWT73TFH7MJANCNFSM4ZUFVQBQ> . <https://github.com/notifications/beacon/AE52VPB7MIIWK4QTBFCWLVTTFH7MJA5CNFSM4ZUFVQB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOGAEUV5A.gif>
|
Thanks @daveh150, you are correct that prepare_forcing disaggregates the precip to rain or snow depending on Tair, so that is good. I will update the documentation. |
I have updated the documentation to clarify rain/snow and remove argparse and datetime from explicitly listed packages. Any other feedback? |
- All files should be placed in a common directory. This includes the raw JRA55 input files, the CICE grid file, and **interp_jra55_ncdf_bilinear.py**. The output files will be written to the same directory. | ||
- The script **configuration/tools/jra55_datasets/make_forcing.csh** was used on the NCAR cheyenne machine in March, 2021 to generate CICE forcing data. It assumes the raw JRA55 is downloaded, but then sets up the python environment, links all the data in a common directory, runs **interp_jra55_ncdf_bilinear.py** and then aggregates the quarterly data using NCO. | ||
- The new forcing files can then be defined in the **ice_in** namelist file using the input variables, ``atm_data_type``, ``atm_data_format``, ``atm_data_dir``, ``fyear_init``, and ``ycycle``. See :ref:`forcing` for more information. | ||
- The total precipitation field is mm/day in JRA55. This field is initially read in as snow, but prepare_forcing in **ice_forcing.F90** splits that into rain or snow forcing depending on the air temperature. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Technically, it's probably mm/day of liquid water (rain), which is read into a variable called 'fsnow' before being adjusted with snow density in the forcing module. But that's a minor quibble.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like both snow and rain are in units of kg/m2-s in the forcing. I assume that water flux is converted to a snow depth output the forcing module?
If I recall, I think Dave Bailey changed the Python code to scale the precip into units of mm/sec. Again, minor point.
I think all else looks good. Thank you for updating the docs!
David
From: Elizabeth Hunke ***@***.***>
Sent: Thursday, March 25, 2021 11:24 AM
To: CICE-Consortium/CICE ***@***.***>
Cc: David Hebert, Code 7322 ***@***.***>; Mention ***@***.***>
Subject: Re: [CICE-Consortium/CICE] Add JRA55 dataset tool and documentation (#582)
@eclare108213 approved this pull request.
_____
In doc/source/developer_guide/dg_tools.rst <#582 (comment)> :
+ ./interp_jra55_ncdf_bilinear.py 1996040100_1996063021 grid_gx3.nc JRA55_gx3_03hr_forcing_1996-q2.nc
+ ./interp_jra55_ncdf_bilinear.py 1996070100_1996093021 grid_gx3.nc JRA55_gx3_03hr_forcing_1996-q3.nc
+ ./interp_jra55_ncdf_bilinear.py 1996100100_1996123121 grid_gx3.nc JRA55_gx3_03hr_forcing_1996-q4.nc
+
+In this case, the 4 quarters of 1996 JRA55 data is going to be interpolated to the gx3 grid.
+NCO can be used to aggregate these files into a single file ::
+
+ ncrcat JRA55_gx3_03hr_forcing_1996-??.nc JRA55_${grid}_03hr_forcing_1996.nc
+
+NOTES
+
+- The scripts are designed to read a CICE grid file in netCDF format. This is the "grid_gx3.nc" file above. The NetCDF grid names are hardcoded in **interp_jra55_ncdf_bilinear.py**. If you are using a different grid file with different variable names, this subroutine needs to be updated.
+- All files should be placed in a common directory. This includes the raw JRA55 input files, the CICE grid file, and **interp_jra55_ncdf_bilinear.py**. The output files will be written to the same directory.
+- The script **configuration/tools/jra55_datasets/make_forcing.csh** was used on the NCAR cheyenne machine in March, 2021 to generate CICE forcing data. It assumes the raw JRA55 is downloaded, but then sets up the python environment, links all the data in a common directory, runs **interp_jra55_ncdf_bilinear.py** and then aggregates the quarterly data using NCO.
+- The new forcing files can then be defined in the **ice_in** namelist file using the input variables, ``atm_data_type``, ``atm_data_format``, ``atm_data_dir``, ``fyear_init``, and ``ycycle``. See :ref:`forcing` for more information.
+- The total precipitation field is mm/day in JRA55. This field is initially read in as snow, but prepare_forcing in **ice_forcing.F90** splits that into rain or snow forcing depending on the air temperature.
Technically, it's probably mm/day of liquid water (rain), which is read into a variable called 'fsnow' before being adjusted with snow density in the forcing module. But that's a minor quibble.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <#582 (review)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AE52VPDUABRBFJ4BMA24NSLTFNPQZANCNFSM4ZUFVQBQ> . <https://github.com/notifications/beacon/AE52VPG5U7QKPQLMRQGCFUDTFNPQZA5CNFSM4ZUFVQB2YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOEUEJ7GY.gif>
|
PR checklist
Short (1 sentence) summary of your PR:
Add JRA55 dataset tool
Developer(s):
apcraig, dabail10, rallard77, daveh150
Suggest PR reviewers from list in the column to the right.
Please copy the PR test results link or provide a summary of testing completed below.
Only documentation/tools changes, no model testing. Did test the JRA55 tool and verified output is bit-for-bit with @dabail10 output.
How much do the PR code changes differ from the unmodified code?
Does this PR create or have dependencies on Icepack or any other models?
Does this PR add any new test cases?
Is the documentation being updated? ("Documentation" includes information on the wiki or in the .rst files from doc/source/, which are used to create the online technical docs at https://readthedocs.org/projects/cice-consortium-cice/. A test build of the technical docs will be performed as part of the PR testing.)
Please provide any additional information or relevant details below:
Add configuration/tools/jra55_datasets directory and scripts
Move convert_restarts to configuration/tools/cice4_restart_conversio
Add tools section to the developer guide
Unrelated to tools