-
Notifications
You must be signed in to change notification settings - Fork 18
Building and Testing MAPL as a standalone
MAPL can be built by itself as long as you have the dependencies. If you are inside the NASA HPC ecosystem on NAS or NCCS computer clusters it is really simple since the SI Team maintains the dependencies needed.
If you have an account as NAS or NCCS it is really, really simple to build MAPL (we promise) since we maintain all the dependencies. There is no need to read beyond this section. You will need "mepo" in your path. On NCCS:
module use -a /discover/swdev/gmao_SIteam/modulefiles-SLES15
module load GEOSenv
or at NAS:
module use -a /nobackup/gmao_SIteam/modulefiles
module load GEOSenv
Once you have mepo in your path, it
module load git cmake other/mepo
git clone --filter=blob:none [email protected]:GEOS-ESM/MAPL.git
Note that this command, up to and including the mepo clone must be done on a login node since the compute nodes can't see the internet By default the clone will be to a directory named MAPL.
If you want the checkout in a different directory, in this example I'll call it MAPL_v2.50.2 you can do this
git clone --filter=blob:none [email protected]:GEOS-ESM/MAPL.git MAPL_v2.50.2
Either way cd to the cloned directory, either "MAPL" or some other name if you did that.
cd MAPL
Now, checkout the version of MAPL you want. I'll use v2.50.2 as an example, but you should checkout the latest release (highest number on that page)
git checkout v2.50.2
mepo clone
From here you would probably want a compute node to do the build. If you do more than -j2 on a login node NCCS/NAS will get mad and it will take longer. Next source g5_modules which loads our compiler, MPI stack, etc. If you are on bash:
source ESMA_env/g5_modules.sh
if using tcsh:
source ESMA_env/g5_modules
Now you can build MAPL. Assuming bash (because of $(pwd)):
cmake -B build -S . --install-prefix=$(pwd)/install -DCMAKE_BUILD_TYPE=Release
cmake --build build --target install -j N
where N is at most 2 if on a login node, or ~10 on a compute node.
Once you do this, you will have an install directory with libraries and executables created by MAPL. The instructions above will build the "Release" build which enables optimization flags and turns off some error checking like going past array bounds. We highly recommend for any development work to use the "Debug" built which turns off optimization flags and turns on error checking such as making sure arrays do not go out of bounds and uninitialized variables will be set to signaling NANs. To turn on the "Debug" built you simple replace -DCMAKE_BUILD_TYPE=Release with -DCMAKE_BUILD_TYPE=Debug in the cmake command when you build. Note you can even have multiple installations. Just build twice and point to a different installation directory in each!
As a side note, if you can do the above you can build the full GEOSgcm or GEOSadas in the same manner, you just clone a different repo to start. We would recommend that you do that rather than use the parallel_build scripts in those repos. As you can see, building by hand is not hard and you will actually understand what is going on!
While we support builds of MAPL on NASA HPC systems due to our maintenance of Baselibs on them, outside of NASA there are two paths. The easy path is Spack, the harder path is Building Baselibs yourself.
MAPL is now in Spack, so all you need to do is:
spack install mapl
and you will get MAPL. As always, you can decorate this with compiler and MPI stacks.
If you are not inside a NASA HPC system, it is more exciting, as you will have to build the dependencies yourself. If you are on a Linux system with a C and Fortran compiler with MPI, you could try build our GMAO baselibs. Assuming you have the compiler/mpi stack, in theory baselibs should build easily and that is detailed below. On macOS, you can also build baselibs but it may be a little more exciting.
MAPL has the following dependencies (really the easiest way to get the dependencies is to build our Baselibs):
- MPI with Fortran interfaces
- ESMF
- NetCDF
- HDF5
- NetCDF
- NetCDF-Fortran
- NetCDF
- HDF5
- NetCDF
- ESMA_cmake
- ecbuild
- GFE (Goddard Fortran Ecosystem, omnibus repo for a collection of managed version of Fortran libraries written mostly by Tom Clune)
Testing
- GFE
Note 1: We are assuming that MPI is available in the desired computing environment and offer no instructions for building and installing MPI.
Baselibs can be obtained here:
https://github.com/GEOS-ESM/ESMA-Baselibs
Before building make sure you have these environment variables set for you chosen compilers:
Compilers:
$ export FC=...
$ export CC=...
$ export CXX=...
Then, be certain that the MPI wrappers (mpif90/mpifort/..., mpicc, mpicxx) are in your path.
In theory if the above is true you can build baselibs like so with GNUmake. For the sake of this example I will assume you want to build Baselibs v8.7.0 and note that I am assuming it is Intel MPI you are using if you have a different flavour of MPI you will need specify that.
mkdir ESMA-Baselibs-8.7.0
cd ESMA-Baselibs-8.7.0
wget https://github.com/GEOS-ESM/ESMA-Baselibs/releases/download/v8.7.0/ESMA-Baselibs-v8.7.0.COMPLETE.tar.xz
cd ESMA-Baselibs-v8.7.0
make download
make -j10 install ESMF_COMM=intelmpi BUILD=ESSENTIALS CONFIG_SETUP=ifort_2021.13.0-intelmpi_2021.13.0
Once Baselibs is built, you would need to point the g5_modules in your MAPL clone to the correct baselibs and make sure any modules you need are loaded on your system. From there you would build as detailed in the beginning of the document.
If you want to clone MAPL via Git, you'll also need to get mepo:
https://github.com/GEOS-ESM/mepo/
which is our tool to handle our subrepositories. mepo is a Python3 script that needs PyYAML, and all you need to do is add the repo to your path. The easiest way to get mepo is via Spack, Brew or Pip:
pip install mepo
spack install mepo
brew install gmao-si-team/packages/mepo
Once you have mepo:
mepo clone -b v2.50.2 https://github.com/GEOS-ESM/MAPL.git
cd MAPL
mepo clone
Note that git clone is the only way to get non-tags of MAPL.
The other easier way to get MAPL is via a "complete" tarfile. You can find these in the Assets for a version, such as here:
https://github.com/GEOS-ESM/MAPL/releases/tag/v2.50.2
where you will see:
MAPL-v2.50.2.COMPLETE.tar.xz
This tarfile has had mepo run on it and is "complete".
- Create a build directory
$ mkdir <build-dir> $ cd <build-dir> - Configure cmake
$ cmake -DCMAKE_PREFIX_PATH=<install-dir> -DESMF= -DNETCDF= - Build and run unit tests
$ make -j $ make -j tests $ ctest -V
Note: The make tests step must be done after the regular make step. This is because some dependencies are not correctly captured in the pFUnit macro used to define the tests.