diff --git a/ReadMe.md b/ReadMe.md index fa156f2bd8..82c84c33db 100644 --- a/ReadMe.md +++ b/ReadMe.md @@ -28,11 +28,11 @@ ADIOS 2 focuses on: In addition, ADIOS 2 APIs are based on: -* **MPI** ADIOS 2 is MPI-based, it can be used in non-MPI serial code. +* **MPI** ADIOS 2 is MPI-based, it can be used in non-MPI serial code. Non-MPI 100% serial build is optional. * **Data Groups** ADIOS 2 favors a deferred/prefetch/grouped variables transport mode by default to maximize data-per-request ratios. Sync mode, one variable at a time, is treated as the special case. -* **Data Steps** ADIOS 2 follow the actual production/consumption of data using an “steps” abstraction removing the need to manage extra indexing information. +* **Data Steps** ADIOS 2 follows the actual production/consumption of data using an I/O “steps” abstraction removing the need to manage extra indexing information. * **Data Engines** ADIOS 2 Engine abstraction allows for reusing the APIs for different transport modes removing the need for drastic code changes. @@ -42,13 +42,22 @@ Please find [The ADIOS 2 User Guide at readthedocs](https://adios2.readthedocs.i ## Getting ADIOS2 -* From source: [Install ADIOS 2](https://adios2.readthedocs.io/en/latest/setting_up/setting_up.html#) requires CMake v3.6 or above +* From source: [Install ADIOS 2 documentation](https://adios2.readthedocs.io/en/latest/setting_up/setting_up.html#) requires CMake v3.6 or above. For a cmake configuration example see [scripts/runconf/runconf.sh](https://github.com/ornladios/ADIOS2/blob/master/scripts/runconf/runconf.sh) -* Conda packages: [https://anaconda.org/williamfgc](https://anaconda.org/williamfgc) +* Conda packages: + * [https://anaconda.org/williamfgc](https://anaconda.org/williamfgc) + * [https://anaconda.org/conda-forge/adios2](https://anaconda.org/conda-forge/adios2) - Once ADIOS 2 is installed refer to: +* Spack: [adios2 package](https://spack.readthedocs.io/en/latest/package_list.html#adios2) + + +* Docker images: + * Ubuntu 18.04: under [scripts/docker/images/ubuntu18.04/Dockerfile](https://github.com/ornladios/ADIOS2/tree/master/scripts/docker/images/ubuntu18.04/Dockerfile) + + +Once ADIOS 2 is installed refer to: * [Linking ADIOS 2](https://adios2.readthedocs.io/en/latest/setting_up/setting_up.html#linking-adios-2) diff --git a/docs/user_guide/source/conf.py b/docs/user_guide/source/conf.py index 566d3b9052..98d93dbfea 100644 --- a/docs/user_guide/source/conf.py +++ b/docs/user_guide/source/conf.py @@ -106,7 +106,7 @@ 'setting_up/source/ctest.rst', 'setting_up/source/cpp_c.rst', 'setting_up/source.rst', - 'setting_up/conda.rst', + 'setting_up/package.rst', 'setting_up/linking.rst', 'api_full/fortran.rst', 'api_full/python.rst', diff --git a/docs/user_guide/source/setting_up/conda.rst b/docs/user_guide/source/setting_up/conda.rst deleted file mode 100644 index cbc5930f01..0000000000 --- a/docs/user_guide/source/setting_up/conda.rst +++ /dev/null @@ -1,9 +0,0 @@ -################## -Install from Conda -################## - -Currently ADIOS 2 can be obtained from anaconda cloud: - -https://anaconda.org/williamfgc/repo - -We strongly recommend using conda-forge dependencies. diff --git a/docs/user_guide/source/setting_up/package.rst b/docs/user_guide/source/setting_up/package.rst new file mode 100644 index 0000000000..32f6295085 --- /dev/null +++ b/docs/user_guide/source/setting_up/package.rst @@ -0,0 +1,28 @@ +########## +As Package +########## + +***** +Conda +***** + +Currently ADIOS 2 can be obtained from anaconda cloud: + +* x86-64 and MacOS: `williamfgc adios2-openmpi adios2-mpich adios2-nompi `_ +* Multiple archs: `conda-forge adios2 `_ + + +***** +Spack +***** + +ADIOS 2 is fully supported in the latest Spack `adios2 package `_ + + +****** +Docker +****** + +Docker images including building and installation of ADIOS 2 in a container: +* Ubuntu 18.04: under `scripts/docker/images/ubuntu18.04/Dockerfile `_ + diff --git a/docs/user_guide/source/setting_up/setting_up.rst b/docs/user_guide/source/setting_up/setting_up.rst index 0de88dd387..980f04c7c9 100644 --- a/docs/user_guide/source/setting_up/setting_up.rst +++ b/docs/user_guide/source/setting_up/setting_up.rst @@ -1,4 +1,4 @@ .. include:: source.rst -.. include:: conda.rst +.. include:: package.rst .. include:: linking.rst diff --git a/docs/user_guide/source/setting_up/source.rst b/docs/user_guide/source/setting_up/source.rst index 48f589b33b..69d323ed69 100644 --- a/docs/user_guide/source/setting_up/source.rst +++ b/docs/user_guide/source/setting_up/source.rst @@ -3,7 +3,7 @@ Install from Source ################### ADIOS2 uses `CMake `_ version 3.6 or above, for building, -testing and installing the library, examples and utilities. +testing and installing the library and utilities. .. include:: source/cmake.rst .. include:: source/hpc_systems.rst diff --git a/scripts/docker/images/ubuntu18.04/Dockerfile b/scripts/docker/images/ubuntu18.04/Dockerfile new file mode 100644 index 0000000000..ea95608458 --- /dev/null +++ b/scripts/docker/images/ubuntu18.04/Dockerfile @@ -0,0 +1,83 @@ +# Dockerfile for ADIOS2 on Ubuntu 18.04 +##################################################################################################### +# +# This is an example Dockerfile image to build and install adios2 master from source in Ubuntu 18.04 +# including dependencies from apt-get and (optionally) the gray-scott tutorial example at the end of +# this file +# Modify at your convenience +# Usage ($ is your terminal prompt): +# 1) build docker image +# $ cd ADIOS2/scripts/docker/images/ubuntu18.04 ; docker build . +# or: +# $ docker build -f /path/to/ADIOS2/scripts/docker/images/ubuntu18.04/Dockerfile . +# add a tag: +# $ docker build -f /path/to/ADIOS2/scripts/docker/images/ubuntu18.04/Dockerfile -t adios2/master +# 2) run the built image in a container +# $ docker run -i -t ubuntu:18.04 /bin/bash +# or with tag: +# $ docker run -i -t adios2/master /bin/bash +##################################################################################################### + +FROM ubuntu:18.04 +MAINTAINER William F Godoy godoywf@ornl.gov +RUN apt-get update -y &&\ + apt-get install build-essential sudo gfortran \ + openmpi-bin libopenmpi-dev libzmq3-dev \ + python3-dev python3-mpi4py python3-numpy python3-pip \ + libblosc-dev libbz2-dev libpng-dev \ + libhdf5-openmpi-dev \ + cmake \ + git vim \ + -y + +RUN cd /tmp &&\ + mkdir -p Software + +RUN cd /tmp/Software &&\ + git clone https://github.com/LLNL/zfp.git --single-branch --branch 0.5.5 --depth 1 &&\ + mkdir -p zfp-build &&\ + cd zfp-build &&\ + cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/opt/zfp -DBUILD_EXAMPLES=OFF ../zfp &&\ + make -j &&\ + make install + +RUN cd /tmp/Software &&\ + git clone https://github.com/disheng222/SZ.git --single-branch --branch v2.1.5.0 --depth 1 &&\ + cd SZ &&\ + ./configure --prefix=/opt/sz &&\ + make -j &&\ + make install + +RUN cd /tmp/Software &&\ + git clone https://github.com/ornladios/ADIOS2.git --single-branch --branch master --depth 1 &&\ + mkdir -p ADIOS2-build &&\ + cd ADIOS2-build &&\ + cmake -DCMAKE_INSTALL_PREFIX=/opt/adios2 \ + -DZFP_ROOT=/opt/zfp \ + -DSZ_ROOT=/opt/sz \ + -DADIOS2_BUILD_TESTING=OFF \ + -DADIOS2_BUILD_EXAMPLES=OFF \ + ../ADIOS2 &&\ + make -j &&\ + make install + +RUN cd /tmp &&\ + rm -fr Software + +ENV PYTHONPATH=/usr/local/lib/python3.6/site-packages/:/opt/adios2/lib/python3.6/site-packages +ENV PATH=/opt/adios2/bin +ENV LD_LIBRARY_PATH=/usr/local/lib:/opt/zfp/lib:/opt/sz/lib:/opt/adios2/lib + +# optionally add a new user and start using adios2 with an example +# RUN useradd -m wfg && echo "wfg:wfg" | chpasswd && adduser wfg sudo +# USER wfg +# CMD /bin/bash + +# RUN cd ~ &&\ +# git clone https://github.com/pnorbert/adiosvm.git --single-branch --branch master --depth 1 &&\ +# cd adiosvm/Tutorial/gray-scott &&\ +# mkdir -p build &&\ +# cd build &&\ +# cmake -DCMAKE_BUILD_TYPE=Release .. &&\ +# make &&\ +# cd .. diff --git a/scripts/runconf/runconf.sh b/scripts/runconf/runconf.sh index 6d3b39b3f6..82c3ea21ec 100644 --- a/scripts/runconf/runconf.sh +++ b/scripts/runconf/runconf.sh @@ -1,5 +1,11 @@ #!/bin/bash +# Usage: +# ./runconf.sh ON lean (MPI On, minimal adios2) +# ./runconf.sh ON (MPI On, adios2 builds with all dependencies, might need -D*_DIR or -D*_ROOT) +# ./runconf.sh OFF lean (MPI Off, minimal adios2) +# ./runconf.sh OFF (MPI Off, adios2 builds with all dependencies, might need -D*_DIR or -D*_ROOT) + USE_MPI=${1:-ON} echo USE_MPI=$USE_MPI @@ -11,14 +17,15 @@ if [ x"$2" == x"lean" ]; then VALUE=OFF fi -ADIOS2_SOURCE=${PWD}/.. +# truly out of source build +ADIOS2_SOURCE=${PWD}/../ADIOS2 echo "ADIOS2_SOURCE: " ${ADIOS2_SOURCE} # -DCMAKE_BUILD_TYPE options: Debug / Release / RelWithDebInfo / MinSizeRel # Change -DCMAKE_INSTALL_PREFIX to your preferred location # Default mode is AUTO # If mode option is ON, it will fail if cmake can't find the dependency location -# Pass *_ROOT for dependency locations e.g. -DZFP_ROOT=/opt/zfp +# Pass *_ROOT or *_DIR for dependency locations e.g. -DZFP_ROOT=/opt/zfp cmake -DCMAKE_INSTALL_PREFIX=${PWD}/install \ -DADIOS2_USE_MPI=${USE_MPI} \ @@ -27,12 +34,20 @@ cmake -DCMAKE_INSTALL_PREFIX=${PWD}/install \ -DADIOS2_USE_Fortran=${VALUE} \ -DADIOS2_USE_Python=${VALUE} \ -DADIOS2_USE_SST=${VALUE} \ + -DADIOS2_USE_SSC=${VALUE} \ + -DADIOS2_USE_DataMan=${VALUE} \ + -DADIOS2_USE_DataSpaces=${VALUE} \ + -DADIOS2_USE_Table=${VALUE} \ -DADIOS2_USE_BZip2=${VALUE} \ + -DADIOS2_USE_PNG=${VALUE} \ + -DADIOS2_USE_Blosc=${VALUE} \ + -DADIOS2_USE_SZ=${VALUE} \ -DADIOS2_USE_ZFP=${VALUE} \ -DBUILD_SHARED_LIBS=ON \ -DCMAKE_POSITION_INDEPENDENT_CODE=ON \ -DCMAKE_BUILD_TYPE=Release \ -DADIOS2_BUILD_TESTING=${VALUE} \ -DADIOS2_BUILD_EXAMPLES=${VALUE} \ + -DADIOS2_USE_Endian_Reverse=OFF \ ${ADIOS2_SOURCE} \ No newline at end of file