Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace bash env files with modules #238

Merged
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
7fa8d3e
Pass machine name to build scripts.
danielabdi-noaa Apr 9, 2022
dbc9d2f
Use modules environment instead of shell scripts.
danielabdi-noaa Apr 9, 2022
f4e8232
Leave conda activation to the user.
danielabdi-noaa Apr 20, 2022
3235ef4
Remove set_machine script.
danielabdi-noaa Apr 20, 2022
6e352ec
Rename env to modulefiles
danielabdi-noaa Apr 20, 2022
9e5d2ad
Minor fix.
danielabdi-noaa Apr 25, 2022
9a13ae2
Merge branch 'develop' into feature/modules
danielabdi-noaa Apr 27, 2022
27e5174
Minor fix
danielabdi-noaa Apr 27, 2022
5a72fdc
Take out *module purge* from modufiles and put it in devbuild.sh
danielabdi-noaa Apr 27, 2022
5e3a525
Activate conda directly in signularity modulefile.
danielabdi-noaa Apr 27, 2022
e4a8c8c
Minor fixes.
danielabdi-noaa Apr 28, 2022
290e269
Merge branch 'develop' into feature/modules
danielabdi-noaa Apr 28, 2022
2189a2f
Add Gaea modulefiles.
danielabdi-noaa Apr 28, 2022
c53859a
Restore odin env files.
danielabdi-noaa Apr 29, 2022
c7c2970
Bug fixes in singularity modulefiles.
danielabdi-noaa Apr 29, 2022
6e305ac
Merge branch 'develop' into feature/modules
danielabdi-noaa Apr 29, 2022
569d32a
Move activation of Lmod to devbuild.sh
danielabdi-noaa Apr 29, 2022
f1583e8
Don't do 'module purge' on cray systems
danielabdi-noaa Apr 29, 2022
8be4258
Put Lmod initialization code in separate script.
danielabdi-noaa Apr 29, 2022
7bb0487
Go back to using modulefile for odin.
danielabdi-noaa Apr 29, 2022
ed44471
Optionally pass machine name to lmod-setup.sh
danielabdi-noaa Apr 29, 2022
b1bf8f9
Modify odin wflow modulefile.
danielabdi-noaa Apr 30, 2022
ac4d3d0
Allow unknown platforms in devbuild.sh
danielabdi-noaa Apr 30, 2022
25c0ea9
Update documentation.
danielabdi-noaa Apr 30, 2022
376b47a
Move cmake init out of lmod-setup.sh on odin
danielabdi-noaa Apr 30, 2022
bb06ae8
Also update markup language build documentation.
danielabdi-noaa Apr 30, 2022
754cb6f
Lmod setup script for both bash and tcsh login shells.
danielabdi-noaa May 1, 2022
7eb75fa
Some fixes for tcsh login shell.
danielabdi-noaa May 1, 2022
20b19db
Add singularity platform to lmod-setup
danielabdi-noaa May 1, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 23 additions & 13 deletions devbuild.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# usage instructions
usage () {
cat << EOF_USAGE
Usage: $0 [OPTIONS]...
Usage: $0 --platform=PLATFORM [OPTIONS]...

OPTIONS
-h, --help
Expand Down Expand Up @@ -93,8 +93,6 @@ BUILD_JOBS=4
CLEAN=false
CONTINUE=false
VERBOSE=false
# detect PLATFORM (MACHINE)
source ${SRC_DIR}/env/detect_machine.sh

# process required arguments
if [[ ("$1" == "--help") || ("$1" == "-h") ]]; then
Expand Down Expand Up @@ -138,6 +136,17 @@ while :; do
shift
done

# check if PLATFORM is set
if [ -z $PLATFORM ] ; then
printf "\nERROR: Please set PLATFORM.\n\n"
usage
exit 0
fi

# set PLATFORM (MACHINE)
MACHINE="${PLATFORM}"
printf "PLATFORM(MACHINE)=${PLATFORM}\n" >&2

set -eu

# automatically determine compiler
Expand All @@ -159,18 +168,18 @@ if [ "${VERBOSE}" = true ] ; then
settings
fi

# set ENV_FILE for this platform/compiler combination
ENV_FILE="${SRC_DIR}/env/build_${PLATFORM}_${COMPILER}.env"
if [ ! -f "${ENV_FILE}" ]; then
printf "ERROR: environment file does not exist for platform/compiler\n" >&2
printf " ENV_FILE=${ENV_FILE}\n" >&2
# set MODULE_FILE for this platform/compiler combination
MODULE_FILE="build_${PLATFORM}_${COMPILER}"
if [ ! -f "${SRC_DIR}/modulefiles/${MODULE_FILE}" ]; then
printf "ERROR: module file does not exist for platform/compiler\n" >&2
printf " MODULE_FILE=${MODULE_FILE}\n" >&2
printf " PLATFORM=${PLATFORM}\n" >&2
printf " COMPILER=${COMPILER}\n\n" >&2
usage >&2
exit 64
fi

printf "ENV_FILE=${ENV_FILE}\n" >&2
printf "MODULE_FILE=${MODULE_FILE}\n" >&2

# if build directory already exists then exit
if [ "${CLEAN}" = true ]; then
Expand Down Expand Up @@ -228,10 +237,11 @@ if [ "${VERBOSE}" = true ]; then
MAKE_SETTINGS="${MAKE_SETTINGS} VERBOSE=1"
fi

# source the environment file for this platform/compiler combination, then build the code
printf "... Source ENV_FILE and create BUILD directory ...\n"
module use ${SRC_DIR}/env
. ${ENV_FILE}
# source the module file for this platform/compiler combination, then build the code
printf "... Load MODULE_FILE and create BUILD directory ...\n"
module use ${SRC_DIR}/modulefiles
chan-hoo marked this conversation as resolved.
Show resolved Hide resolved
module purge
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that a purge here was causing some issues on Cheyenne in the past. @mkavulich Can you confirm that is still problematic.

module load ${MODULE_FILE}
module list
mkdir -p ${BUILD_DIR}
cd ${BUILD_DIR}
Expand Down
22 changes: 0 additions & 22 deletions env/build_jet_intel.env

This file was deleted.

58 changes: 0 additions & 58 deletions env/build_odin_intel.env

This file was deleted.

22 changes: 0 additions & 22 deletions env/build_orion_intel.env

This file was deleted.

40 changes: 0 additions & 40 deletions env/build_singularity_gnu.env

This file was deleted.

24 changes: 0 additions & 24 deletions env/build_wcoss_dell_p3_intel.env

This file was deleted.

107 changes: 0 additions & 107 deletions env/detect_machine.sh

This file was deleted.

6 changes: 0 additions & 6 deletions env/wflow_cheyenne.env

This file was deleted.

7 changes: 0 additions & 7 deletions env/wflow_hera.env

This file was deleted.

7 changes: 0 additions & 7 deletions env/wflow_jet.env

This file was deleted.

18 changes: 0 additions & 18 deletions env/wflow_macos.env

This file was deleted.

Loading