SRW modulefiles to use updated HPC-stack libraries and miniconda3 modules, on NOAA RDHPCs, in EPIC-managed common spaces

[natalie-perlin]

Description

Modules of the srw-weather-app in ./modulefiles/ directory need to be updated to use the recent builds, in common EPIC-managed space, and to share locations with the UFS-Weather-Model. Miniconda3 modules with identical installations of the base conda and regional_workflow conda environments have been installed and tested on all the platforms. The environment regional_workflow contains all the packages for running the SRW, plotting figures using basic python examples, and met verification.

Additional updated hpc-stack libraries with the netcdf-c/4.9.0 have been build on Hera with several compiler/mpi options, for future testing. Other systems (Cheyenne, Gaea, Jet, Orion) are pending updates with netcdf-c/4.9.0, after successful testing on Hera is confirmed.

Locations of updated hpc-stack modules on the NOAA RDHPCs:

Cheyenne intel/2022.1:
module load intel/2022.1
module load mpit/2.25
module use /glade/work/epicufsrt/contrib/hpc-stack/intel2022.1/modulefiles/stack
module load hpc/1.2.0
module load hpc-intel/2022.1
module load hpc-mpt/2.25

Cheyenne gnu/11.2:
module gnu/11.2.0
module mpt/2.25
module use /glade/work/epicufsrt/contrib/hpc-stack/gnu11.2.0/modulefiles/stack
module load hpc/1.2.0
module load hpc-gnu/11.2.0
module load hpc-mpt/2.25

Hera intel/2022.1.2 :
module load intel/2022.1.2
module load impi/2022.1.2
use /scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/intel-2022.1.2/modulefiles/stack
module load hpc/1.2.0
module load hpc-intel/2022.1.2
module load hpc-impi/2022.1.2

Hera gnu/9.2 + mpich/3.3.2 :
module load gnu/9.2
module use /scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/gnu-9.2/modulefiles/stack
module load hpc/1.2.0
module load hpc-gnu/9.2
module load mpich/3.3.2
module load hpc-mpich/3.3.2

Hera gnu/10.2 + mpich/3.3.2 :
module use /scratch1/NCEPDEV/nems/role.epic/gnu/modulefiles
module load gnu/10.2.0
module use /scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/gnu-10.2/modulefiles/stack
module load hpc/1.2.0
module load hpc-gnu/10.2
module load mpich/3.3.2
module load hpc-mpich/3.3.2

Hera gnu/10.2 + openmpi/4.1.2 :
module use /scratch1/NCEPDEV/nems/role.epic/gnu/modulefiles
module load gnu/10.2.0
module use /scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/gnu-10.2_openmpi/modulefiles/stack
module load hpc/1.2.0
module load hpc-gnu/10.2
module load openmpi/4.1.2
module load hpc-openmpi/4.1.2

Hera gnu/9.2 + mpich/3.3.2 + netcdf 4.9:
module load gnu/9.2
module use /scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/gnu-9.2_ncdf49/modulefiles/stack
module load hpc/1.2.0
module load hpc-gnu/9.2
module load mpich/3.3.2
module load hpc-mpich/3.3.2

Hera gnu/10.2 + mpich/3.3.2 + netcdf/4.9:
module use /scratch1/NCEPDEV/nems/role.epic/gnu/modulefiles
module load gnu/10.2.0
module use /scratch1/NCEPDEV/nems/role.epic/hpc-stack/libs/gnu-10.2_ncdf49/modulefiles/stack
module load hpc/1.2.0
module load hpc-gnu/10.2
module load mpich/3.3.2
module load hpc-mpich/3.3.2

Gaea intel/2021.3.0:
Lmod initialization on Gaea could to be done by sourcing the following script:
source /lustre/f2/dev/role.epic/contrib/Lmod_init.sh

module use /lustre/f2/dev/role.epic/contrib/hpc-stack/intel-2021.3.0/modulefiles/stack
module load hpc/1.2.0
module load intel/2021.3.0
module load hpc-intel/2021.3.0
module load hpc-cray-mpich/7.7.11

Orion intel/2022.1.2:
module load intel/2022.1.2
module load impi/2022.1.2
module use /work/noaa/epic-ps/role-epic-ps/hpc-stack/libs/intel-2022.1.2/modulefiles/stack
module load hpc/1.2.0
module load hpc-intel/2022.1.2
module load hpc-impi/2022.1.2

Jet intel/2022.1.2:
module load intel/2022.1.2
module load impi/2022.1.2
module use /mnt/lfs4/HFIP/hfv3gfs/role.epic/hpc-stack/libs/intel-2022.1.2/modulefiles/stack
module load hpc/1.2.0
module load hpc-intel/2022.1.2
module load hpc-impi/2022.1.2

Miniconda3/4.12.0 locations on the RDHPCs, with the regional_workflow environment created containing all the packages for running, plotting, and met verification.

Cheyenne:
module use /glade/work/epicufsrt/contrib/miniconda3/modulefiles
module load miniconda3/4.12.0
Hera:
module use /scratch1/NCEPDEV/nems/role.epic/miniconda3/modulefiles
module load miniconda3/4.12.0
Gaea:
module use /lustre/f2/dev/role.epic/contrib/modulefiles
module load miniconda3/4.12.0
Orion:
module use /work/noaa/epic-ps/role-epic-ps/miniconda3/modulefiles
module load miniconda3/4.12.0
Jet:
module use /mnt/lfs4/HFIP/hfv3gfs/role.epic/miniconda3/modulefiles
module load miniconda3/4.12.0

Solution

SRW modulefiles to be updated:
./modulefiles/build_[.<compiler}>].lua
./modulefiles/wflow_.lua - load miniconda3/4.12.0 and rocoto when needed
./modulefiles/tasks/machine/.local.lua - use regional_workflow environment installed under miniconda3/4.12.0

Requirements**

A shared location for software libraries for both ufs-srweather-app and ufs-weather-model to be used

Acceptance Criteria (Definition of Done)

Dependencies (optional)

ufs-srweather-app/PR-419

Similar/parallel Issue has been created in ufs-weather-model/issue-1465:

Alternative Solutions (optional)

[MichaelLueken]

The Cheyenne, Jet, and Orion modulefiles were updated to point to EPIC-managed locations at d036849. The Hera modulefiles were updated to point to EPIC-managed locations at d03e646. The Gaea modulefile was updated to point to EPIC-managed location at 550f400. Closing this issue now.