Error when trying to calculate pressure load anomaly

[DaniJonesOcean]

I've received an error when trying to run this script:

# add one directory above to path
import sys
sys.path.append("..")

# import other modules
import mitgcm_python
import MITgcmutils
from mitgcm_python.ics_obcs import *

# pressure load anomaly
calc_load_anomaly('../../grid/',
                  '../initial_conditions/pload_gyre',
                  ini_temp_file='../initial_conditions/THETA_BSOSE.ini',
                  ini_salt_file='../initial_conditions/SALT_BSOSE.ini',
                  constant_t=-1.9,
                  constant_s=34.4,
                  rhoConst=1035)

The error indicates that an equation of state module isn't available:

Reading ../initial_conditions/THETA_BSOSE.ini
Reading ../initial_conditions/SALT_BSOSE.ini
RMS error = 14579222.6099
Traceback (most recent call last):
  File "a08_pressure_load.py", line 17, in <module>
    rhoConst=1035)
  File "../mitgcm_python/ics_obcs.py", line 412, in calc_load_anomaly
    drho_c = density(eosType, salt, temp, press, rhoConst=rhoConst, Tref=Tref, Sref=Sref, tAlpha=tAlpha, sBeta=sBeta) - rhoConst
  File "../mitgcm_python/diagnostics.py", line 197, in density
    from MITgcmutils.mdjwf import densmdjwf
ImportError: No module named mdjwf

Indeed, I can't find this script. What's going on here?

[DaniJonesOcean]

It seemed to come from this line:

https://github.com/knaughten/mitgcm_python/blob/master/diagnostics.py#L197

Shouldn't the EOS type be JMD95?

The function calc_load_anomaly was using the default value, which was incorrect. Let's add an argument to specify the eosType to match what is used in MITgcm. It's supposed to be JMD95!

[DaniJonesOcean]

Hopefully that bug was fixed by PR #30. Unfortunately, I can't test it on ARCHER2, as I'm having (sigh) environment issues, as always. Here's the new command and error:

dannes1@ln01:~/work/MITgcm_sowise_dev/MITgcm/experiments/so-wise-gyre/setup/scripts> python3 a08_pressure_load.py 
Traceback (most recent call last):
  File "a08_pressure_load.py", line 8, in <module>
    from mitgcm_python.ics_obcs import *
  File "../mitgcm_python/ics_obcs.py", line 10, in <module>
    from scipy.interpolate import interp1d
  File "/opt/cray/pe/python/3.8.5.0/lib/python3.8/site-packages/scipy/interpolate/__init__.py", line 167, in <module>
    from .interpolate import *
  File "/opt/cray/pe/python/3.8.5.0/lib/python3.8/site-packages/scipy/interpolate/interpolate.py", line 15, in <module>
    import scipy.special as spec
  File "/opt/cray/pe/python/3.8.5.0/lib/python3.8/site-packages/scipy/special/__init__.py", line 637, in <module>
    from . import _basic
  File "/opt/cray/pe/python/3.8.5.0/lib/python3.8/site-packages/scipy/special/_basic.py", line 19, in <module>
    from . import orthogonal
  File "/opt/cray/pe/python/3.8.5.0/lib/python3.8/site-packages/scipy/special/orthogonal.py", line 81, in <module>
    from numpy import (exp, inf, pi, sqrt, floor, sin, cos, around, int,
ImportError: cannot import name 'int' from 'numpy' (/home/n01/n01/dannes1/.local/lib/python3.8/site-packages/numpy/__init__.py)

I'm using the following python environment on ARCHER2:

Currently Loaded Modules Matching: python
  1) cray-python/3.8.5.0

I'd like to move towards a container approach to this. That has always saved me in the past.

[DaniJonesOcean]

Trying this on BAS HPC, using python3, we get the following error:

Traceback (most recent call last):
  File "a08_pressure_load.py", line 8, in <module>
    from mitgcm_python.ics_obcs import *
  File "../mitgcm_python/ics_obcs.py", line 8, in <module>
    from .interpolation import extend_into_mask, discard_and_fill, neighbours_z, interp_slice_helper, interp_grid
  File "../mitgcm_python/interpolation.py", line 327
    import .file_io as fio
           ^
SyntaxError: invalid syntax

I really want a stable environment...

[DaniJonesOcean]

@DanJonesOcean: share the current SO-WISE setup with @sg2400 via /n01/shared

@sg2400: look into environment issues

[DaniJonesOcean]

One note for @sg2400: the Python script I'm trying to run is located here:

MITgcm/experiments/so-wise-gyre/setup/scripts/a08_pressure_load.py

And I was trying to run it simply from the command line:

python3 a08_pressure_load.py

Note that you have to have a couple environment variables set first, as detailed in @knaughten's repo:

https://github.com/knaughten/mitgcm_python

[DaniJonesOcean]

Update for @sg2400. Just FYI, I tried this using a Docker container. Here's what I ran:

docker run --rm -it -p 8888:8888 -v $PWD:/work -w /work pangeo/pangeo-notebook:2021.11.09 jupyter-lab --no-browser --ip=0.0.0.0

I then opened up a terminal within the JupyterLab client and set some environment variables:

jovyan@d4d3e70f18ee:/work/MITgcm$ pwd
/work/MITgcm
jovyan@d4d3e70f18ee:/work/MITgcm$ export ROOTDIR=/work/MITgcm
jovyan@d4d3e70f18ee:/work/MITgcm$ echo $ROOTDIR
/work/MITgcm
jovyan@d4d3e70f18ee:/work/MITgcm$ export PYTHONPATH=$PYTHONPATH:$ROOTDIR/utils/python/MITgcmutils
jovyan@d4d3e70f18ee:/work/MITgcm$ echo $PYTHONPATH
:/work/MITgcm/utils/python/MITgcmutils

Next, I navigated to /work/MITgcm/experiments/so-wise-gyre/setup/scripts and ran the following:

jovyan@d4d3e70f18ee:/work/MITgcm/experiments/so-wise-gyre/setup/scripts$ python3 a08_pressure_load.py 
Reading ../initial_conditions/THETA_BSOSE.ini
Reading ../initial_conditions/SALT_BSOSE.ini
RMS error = 14579222.60994877
Killed

So at least it ran for a while, although it was killed for some reason. The terminal output was:

TermSocket.open: 1
not able to serialize: expected string or bytes-like object
TermSocket.open: Opened 1

But it was good to see that I can at least get the script to run within a Docker container. Progress?

[DaniJonesOcean]

The RMS error looks pretty high. I'm guessing that the pressure solver didn't manage to converge. But maybe it ran into memory or formatting issues before it could even try a second iteration.

[DaniJonesOcean]

Trying on ARCHER2. First, use the container:

singularity shell -B /mnt/lustre/a2fs-work1/work/n01/n01/dannes1/MITgcm_sowise_dev/ pangeo-notebook:2021.11.09.sif

Next, set some environment variables (maybe we could eventually put these in the Docker file?)

export ROOTDIR=/mnt/lustre/a2fs-work1/work/n01/n01/dannes1/MITgcm_sowise_dev/MITgcm/
export PYTHONPATH=$PYTHONPATH:$ROOTDIR/utils/python/MITgcmutils

Navigate to the directory and try to run the script:

cd /mnt/lustre/a2fs-work1/work/n01/n01/dannes1/MITgcm_sowise_dev/
cd MITgcm/experiments/so-wise-gyre/setup/scripts/
Singularity> python3 a08_pressure_load.py 

So we can at least test it on a compute node...it's still very slow is what I've noticed so far...

[DaniJonesOcean]

Looks like it's working, albeit very slowly:

Singularity> python3 a08_pressure_load.py 
Reading ../initial_conditions/THETA_BSOSE.ini
Reading ../initial_conditions/SALT_BSOSE.ini
RMS error = 14579222.60994877
RMS error = 59314.58320809659
RMS error = 231.40620525431606
RMS error = 1.460721968705925

[DaniJonesOcean]

Ok! This seems to work, sort of. Here's what the file looks like now. I'm not sure if the pressure load is right, given the strange stripes along one side of the Ronne-Filchner. That's a little suspicious.

[DaniJonesOcean]

The odd horizontal stripes in the pressure anomaly field remind me of those from this error:

#34

Could this be a clue as to what is going on? Do the issues have the same root somewhere?