-
Notifications
You must be signed in to change notification settings - Fork 14
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
getting_started/spack.md: incorporate feedback
* Thanks to Claire Carouge, Chris Bull and Davide Marchegiani
- Loading branch information
Showing
1 changed file
with
155 additions
and
10 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,15 +1,14 @@ | ||
| # Set up Spack for building ACCESS models | ||
|
|
||
| !!! danger | ||
| This page is tailored to experienced users and collaborators developing ACCESS models. | ||
| This page is tailored to experienced users and collaborators developing ACCESS models. This step is *not* required if you *only* want to run a model. | ||
|
|
||
| [Spack](https://spack.io/about/) is a build-from-source package manager, specifically designed to simplify the installation of scientific software on supercomputers. | ||
|
|
||
| To use _Spack_, please familiarise yourself with the [Basic Usage instructions](https://spack.readthedocs.io/en/latest/basic_usage.html) and [Environments](https://spack.readthedocs.io/en/latest/environments.html). | ||
|
|
||
| We also recommend that you refer to the [Spack 101 Tutorial](https://spack-tutorial.readthedocs.io/en/latest/). | ||
|
|
||
|
|
||
| ## Prerequisites | ||
| These instructions are tailored specifically for _Gadi_. To use _Spack_ on _Gadi_, you must have an NCI account. For instructions on how to set up an account, refer to [Set Up your NCI Account](/getting_started/set_up_nci_account). | ||
|
|
||
|
|
@@ -20,11 +19,11 @@ These instructions are tailored specifically for _Gadi_. To use _Spack_ on _Gadi | |
|
|
||
| ### Create a directory for Spack | ||
|
|
||
| Create a directory on the filesystem where _Spack_ will be installed (e.g. `/g/data/$PROJECT/$USER/spack/0.22`). | ||
| Create a directory on the filesystem where _Spack_ will be installed (e.g. `/g/data/$PROJECT/$USER/spack/0.22`). Use the `/g/data` filesystem if you wish to run the binaries on the compute nodes. | ||
|
|
||
| ``` | ||
| mkdir -p spack/0.22 | ||
| cd spack/0.22 | ||
| mkdir -p /g/data/$PROJECT/$USER/spack/0.22 | ||
| cd /g/data/$PROJECT/$USER/spack/0.22 | ||
| ``` | ||
|
|
||
| ### Clone the relevant git repositories | ||
|
|
@@ -47,21 +46,167 @@ ln -s -r -v spack-config/v0.22/gadi/* spack/etc/spack/ | |
| !!! success | ||
| Your _Spack_ setup is complete! | ||
|
|
||
| ## Test Spack (OPTIONAL) | ||
| ## Enable Spack | ||
|
|
||
| To test _Spack_ we will create an `ACCESS-TEST` environment and build the relevant packages. Then, we will uninstall all the packages and remove the environment. | ||
| !!! warning | ||
| This step needs to be carried out for any new login or new shell environment. | ||
|
|
||
| ``` | ||
| cd /g/data/$PROJECT/$USER/spack/0.22 | ||
| module purge | ||
| cd spack/0.22 | ||
| . spack-config/spack-enable.bash | ||
| ``` | ||
|
|
||
| !!! warning | ||
| There is a space between the `.` and the path to the file, as we are sourcing the file. | ||
|
|
||
| ## Test Spack (OPTIONAL) | ||
|
|
||
| To test _Spack_ we will create an `ACCESS-TEST` environment and build the relevant packages. It takes approximately 30 minutes to compile. Then, we will uninstall all the packages and remove the environment. | ||
|
|
||
|
|
||
| ### Create the Spack managed environment | ||
|
|
||
| ``` | ||
| git clone https://github.com/ACCESS-NRI/ACCESS-TEST.git | ||
| spack env create test ACCESS-TEST/spack.yaml | ||
| ``` | ||
|
|
||
| <terminal-window> | ||
| <terminal-line data="input">spack env create test ACCESS-TEST/spack.yaml</terminal-line> | ||
| <terminal-line><span class="indigo bold">\==></span> Created environment <span class="cyan">test</span> in: <span class="cyan">/g/data/$PROJECT/$USER/spack/0.22/environments/test</span></terminal-line> | ||
| <terminal-line><span class="indigo bold">\==></span> Activate with: <span class="cyan">spack env activate test</span></terminal-line> | ||
| </terminal-window> | ||
|
|
||
| ### Compile packages | ||
|
|
||
| ``` | ||
| spack env activate -p test | ||
| spack find | ||
| ``` | ||
|
|
||
| <terminal-window> | ||
| <terminal-line data="input">spack env activate -p test</terminal-line> | ||
| <terminal-line data="input">[test]$ spack find</terminal-line> | ||
| <terminal-line>\==> In environment test</terminal-line> | ||
| <terminal-line>\==> 1 root specs</terminal-line> | ||
| <terminal-line> - [email protected]=2024.09.20</terminal-line> | ||
| <terminal-line></terminal-line> | ||
| <terminal-line>\==> 0 installed packages</terminal-line> | ||
| </terminal-window> | ||
|
|
||
| ``` | ||
| spack concretize -f --fresh | ||
| spack install --verbose | ||
| spack install | ||
| ``` | ||
|
|
||
| <terminal-window> | ||
| <terminal-line data="input">[test]$ spack concretize -f --fresh</terminal-line> | ||
| <terminal-line>\==> Concretized [email protected]=2024.09.20</terminal-line> | ||
| <terminal-line> - hmy75yl [email protected]=2024.09.20%[email protected]~deterministic build_system=bundle arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line>[e] 5elnsoi ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - kok5n7h ^[email protected]=2023.11.09%[email protected]~deterministic~optimisation_report build_system=makefile arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - doeoclg ^[email protected]%[email protected]~guile build_system=generic arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - ntfunrm ^[email protected]%[email protected]~doc+pic+shared build_system=autotools arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - vob7om3 ^[email protected]%[email protected]+blosc~byterange~dap~fsync~hdf4~jna+mpi~nczarr_zip+optimize~parallel-netcdf+pic+shared+szip+zstd build_system=autotools patches=0161eb8 arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - 357vng5 ^[email protected]%[email protected]~debug~pic+shared build_system=generic arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - y7n7vkn ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - 55ipnye ^[email protected]%[email protected] build_system=autotools libs=shared,static arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - zrjfo56 ^[email protected]%[email protected]+avx2~ipo build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line>[e] rldyvqn ^[email protected]%[email protected]~doc+ncurses+ownlibs build_system=generic build_type=Release arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - 7a5olrr ^[email protected]%[email protected]+pic build_system=makefile libs=shared,static arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - wk4pvru ^[email protected]%[email protected]~ipo+pic+shared build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - aumsrgz ^[email protected]%[email protected]~cxx~fortran+hl~ipo~java~map+mpi+shared~subfiling~szip~threadsafe+tools api=default build_system=cmake build_type=Release generator=make patches=82088c8 arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - vrupasu ^[email protected]%[email protected] build_system=autotools arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - x4hw5jq ^[email protected]%[email protected]~ipo+shared build_system=cmake build_type=Release generator=make arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - wa2x7rh ^[email protected]%[email protected]+compat+new_strategies+opt+pic+shared build_system=autotools arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line> - kzc7pcv ^[email protected]%[email protected]~programs build_system=makefile libs=shared,static arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line>[e] uvea7q2 ^[email protected]%[email protected] cppflags='-diag-disable=10441' ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~internal-libevent~internal-pmix~java~legacylaunchers~lustre~memchecker~openshmem~orterunprefix~romio+rsh~singularity~static+vt+wrapper-rpath build_system=autotools fabrics=none romio-filesystem=none schedulers=none arch=linux-rocky8-x86_64</terminal-line> | ||
| <terminal-line data="input">[test]$ spack install</terminal-line> | ||
| <terminal-line>[+] /usr (external glibc-2.28-5elnsoiqgcg5k5zmmwsp33bmnmaa3g5p)</terminal-line> | ||
| <terminal-line>[+] /apps/cmake/3.24.2 (external cmake-3.24.2-rldyvqnwmmg4cepzstrlmu75vx2wf6cj)</terminal-line> | ||
| <terminal-line>\==> [email protected] : has external module in ['openmpi/4.1.5']</terminal-line> | ||
| <terminal-line>[+] /apps/openmpi/4.1.5 (external openmpi-4.1.5-uvea7q2exkjwz2v7azix3x4zfcddnc23)</terminal-line> | ||
| <terminal-line>\==> Installing gmake-4.4.1-doeoclgsn63cvh7zfx3trjmyk7rlcqn4 [4/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/gmake-4.4.1-doeoclgsn63cvh7zfx3trjmyk7rlcqn4</terminal-line> | ||
| <terminal-line>\==> Installing libaec-1.0.6-x4hw5jqq3zvnrgjicgweicomeaelulqq [5/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/libaec-1.0.6-x4hw5jqq3zvnrgjicgweicomeaelulqq</terminal-line> | ||
| <terminal-line>\==> Installing zlib-ng-2.1.6-wa2x7rho3km6qpiki56dpjlpsce4c5n6 [6/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/zlib-ng-2.1.6-wa2x7rho3km6qpiki56dpjlpsce4c5n6</terminal-line> | ||
| <terminal-line>\==> Installing snappy-1.1.10-wk4pvrufyvy7v3hxn5nwa3i3fncp3azm [7/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/snappy-1.1.10-wk4pvrufyvy7v3hxn5nwa3i3fncp3azm</terminal-line> | ||
| <terminal-line>\==> Installing pkgconf-2.2.0-vrupasu7smpgcbarzpdwap45fcvjbjoa [8/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/pkgconf-2.2.0-vrupasu7smpgcbarzpdwap45fcvjbjoa</terminal-line> | ||
| <terminal-line>\==> Installing zstd-1.5.6-kzc7pcve7csxlonb2uaxzgyyuqfx6cz4 [9/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/zstd-1.5.6-kzc7pcve7csxlonb2uaxzgyyuqfx6cz4</terminal-line> | ||
| <terminal-line>\==> Installing lz4-1.9.4-7a5olrrnewy7kmlh5x4bstziuheiqkz4 [10/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/lz4-1.9.4-7a5olrrnewy7kmlh5x4bstziuheiqkz4</terminal-line> | ||
| <terminal-line>\==> Installing libiconv-1.17-55ipnyeeqcpbfgaqfanu36viaqqni3sx [11/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/libiconv-1.17-55ipnyeeqcpbfgaqfanu36viaqqni3sx</terminal-line> | ||
| <terminal-line>\==> Installing hdf5-1.14.3-aumsrgzvbh6grtyyegzuufilnqa7ftm7 [12/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/hdf5-1.14.3-aumsrgzvbh6grtyyegzuufilnqa7ftm7</terminal-line> | ||
| <terminal-line>\==> Installing c-blosc-1.21.5-zrjfo567d2n6ctwayae77z3b54mf23yc [13/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/c-blosc-1.21.5-zrjfo567d2n6ctwayae77z3b54mf23yc</terminal-line> | ||
| <terminal-line>\==> Installing diffutils-3.10-y7n7vkngczu47neysm3retisvlsmw53l [14/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/diffutils-3.10-y7n7vkngczu47neysm3retisvlsmw53l</terminal-line> | ||
| <terminal-line>\==> Installing bzip2-1.0.8-357vng5dpd7w7s7lletycxccjbl45ngt [15/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/bzip2-1.0.8-357vng5dpd7w7s7lletycxccjbl45ngt</terminal-line> | ||
| <terminal-line>\==> Installing netcdf-c-4.9.2-vob7om32jopqwss5jilrdtqqogjvcmzb [16/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/netcdf-c-4.9.2-vob7om32jopqwss5jilrdtqqogjvcmzb</terminal-line> | ||
| <terminal-line>\==> Installing netcdf-fortran-4.6.1-ntfunrmysxanqqu7sqfmf66zdkd2xemy [17/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/netcdf-fortran-4.6.1-ntfunrmysxanqqu7sqfmf66zdkd2xemy</terminal-line> | ||
| <terminal-line>\==> Installing oasis3-mct-git.2023.11.09=2023.11.09-kok5n7hvm374eicnidcedxhgxmmytc2p [18/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/oasis3-mct-git.2023.11.09_2023.11.09-kok5n7hvm374eicnidcedxhgxmmytc2p</terminal-line> | ||
| <terminal-line>\==> Installing access-test-git.2024.09.20=2024.09.20-hmy75yl26hexivgsw7zhlvbnjgst3gwc [19/19]</terminal-line> | ||
| <terminal-line>...</terminal-line> | ||
| <terminal-line>[+] /g/data/$PROJECT/$USER/spack/0.22/release/linux-rocky8-x86_64/intel-2021.10.0/access-test-git.2024.09.20_2024.09.20-hmy75yl26hexivgsw7zhlvbnjgst3gwc</terminal-line> | ||
| <terminal-line>\==> Updating view at /g/data/$PROJECT/$USER/spack/0.22/environments/test/.spack-env/view</terminal-line> | ||
| </terminal-window> | ||
|
|
||
| !!! info | ||
| The full output has been truncated for brevity. | ||
|
|
||
| ``` | ||
| spack find | ||
| ``` | ||
|
|
||
| <terminal-window> | ||
| <terminal-line data="input">[test]$ spack find</terminal-line> | ||
| <terminal-line>\==> In environment test</terminal-line> | ||
| <terminal-line>\==> 1 root specs</terminal-line> | ||
| <terminal-line>[+] [email protected]=2024.09.20</terminal-line> | ||
| <terminal-line></terminal-line> | ||
| <terminal-line>\==> Installed packages</terminal-line> | ||
| <terminal-line>-- linux-rocky8-x86_64 / [email protected] ------------------------</terminal-line> | ||
| <terminal-line>[email protected]=2024.09.20 [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]=2023.11.09</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected] [email protected]</terminal-line> | ||
| <terminal-line>[email protected]</terminal-line> | ||
| <terminal-line>\==> 19 installed packages</terminal-line> | ||
| </terminal-window> | ||
|
|
||
| ``` | ||
| spack uninstall --remove --all | ||
| spack env deactivate | ||
| spack env rm test | ||
|
|
@@ -73,7 +218,7 @@ rm -rf ACCESS-TEST | |
| Keep your Spack instance up-to-date by doing the following: | ||
|
|
||
| ``` | ||
| cd spack/0.22 | ||
| cd /g/data/$PROJECT/$USER/spack/0.22 | ||
| git -C spack fetch --all -Pp | ||
| git -C spack reset --hard origin/releases/v0.22 | ||
| git -C spack-config pull | ||
|
|
||