Globsim 4.0

[nicholas512]

Recent changes to the ERA5 download API mean that certain globsim libraries have to be updated. As a consequence, moving forward globsim will be designed to use python>=3.10 and we will use the upgraded ESMF 8.6 library for regridding. The next major release of globsim (4.0) will likely not be backwards compatible with python 3.6, or with ESMF 7.x. In addition, due to changes in the GRIB to NetCDF conversion done by CDS going forward, ERA5 files will be different than before and will have to be re-downloaded.

New Features

• 🦺 Safe(r) memory usage globsim checks to see if memory usage is too high (>90%) and exits if it is. This reduces the likelihood of corrupted files, making it possible to recover them.
• ▶️ Resume Interpolation. You can pick up an interpolation where you left off with the --resume flag. Useful if you ran out of memory partway through. Experimental.
• 🗾JRA3Q - this version has support for JRA3Q downloaded from NCAR. you can select -d JRA3Q for the 1.25 degree lat/lon grid or -d JRA3QG for the 0.5 degree gaussian grid (experimental)

Updating your globsim environment

Changes to libraries and supported python version means the globsim environment will have to be updated.

1. Create a new conda environment (e.g. conda create -n globsim4 python=3.10)
2. Install globsim (using pip install -e .)
3. Install required packages for globsim (trial and error using pip if not all the required depdencies are installed.)
4. Set up environment variables to point esmpy to the correct ESMF files:

   cd $CONDA_PREFIX
   mkdir -p ./etc/conda/activate.d
   mkdir -p ./etc/conda/deactivate.d
   echo "export ESMFMKFILE=/opt/esmf-8.6.1/lib/libO/Linux.gfortran.64.openmpi.default/esmf.mk" > ./etc/conda/activate.d/env_vars.sh
   echo "unset ESMFMKFILE"  > ./etc/conda/deactivate.d/env_vars.sh
   

5. Install new version of ESMF (now called esmpy)

cd /opt/esmf-8.6.1/src/addon/esmpy 
pip install .

Closed issues

closes #147
closes #135
closes #136
closes #115
closes #139 (for now... a3876af)
closes #148