Merge pull request #1854 from williamfgc/runconf_update

Minor docs, cmake runconf example update and Dockerfile

ReadMe.md

@@ -28,11 +28,11 @@ ADIOS 2 focuses on:
 
 In addition, ADIOS 2 APIs are based on:
 
-* **MPI** ADIOS 2 is MPI-based, it can be used in non-MPI serial code.
+* **MPI** ADIOS 2 is MPI-based, it can be used in non-MPI serial code. Non-MPI 100% serial build is optional.
 
 * **Data Groups** ADIOS 2 favors a deferred/prefetch/grouped variables transport mode by default to maximize data-per-request ratios. Sync mode, one variable at a time, is treated as the special case.
 
-* **Data Steps** ADIOS 2 follow the actual production/consumption of data using an “steps” abstraction removing the need to manage extra indexing information.
+* **Data Steps** ADIOS 2 follows the actual production/consumption of data using an I/O “steps” abstraction removing the need to manage extra indexing information.
 
 * **Data Engines** ADIOS 2 Engine abstraction allows for reusing the APIs for different transport modes removing the need for drastic code changes.
 
@@ -42,13 +42,22 @@ Please find [The ADIOS 2 User Guide at readthedocs](https://adios2.readthedocs.i
 
 ## Getting ADIOS2
 
-* From source: [Install ADIOS 2](https://adios2.readthedocs.io/en/latest/setting_up/setting_up.html#) requires CMake v3.6 or above
+* From source: [Install ADIOS 2 documentation](https://adios2.readthedocs.io/en/latest/setting_up/setting_up.html#) requires CMake v3.6 or above. For a cmake configuration example see [scripts/runconf/runconf.sh](https://github.com/ornladios/ADIOS2/blob/master/scripts/runconf/runconf.sh)
 
 
-* Conda packages: [https://anaconda.org/williamfgc](https://anaconda.org/williamfgc)
+* Conda packages: 
+    * [https://anaconda.org/williamfgc](https://anaconda.org/williamfgc)
+    * [https://anaconda.org/conda-forge/adios2](https://anaconda.org/conda-forge/adios2)
 
 
-  Once ADIOS 2 is installed refer to: 
+* Spack: [adios2 package](https://spack.readthedocs.io/en/latest/package_list.html#adios2)
+
+
+* Docker images: 
+    * Ubuntu 18.04: under [scripts/docker/images/ubuntu18.04/Dockerfile](https://github.com/ornladios/ADIOS2/tree/master/scripts/docker/images/ubuntu18.04/Dockerfile)
+
+
+Once ADIOS 2 is installed refer to: 
 
 * [Linking ADIOS 2](https://adios2.readthedocs.io/en/latest/setting_up/setting_up.html#linking-adios-2)
 

docs/user_guide/source/conf.py

@@ -106,7 +106,7 @@
     'setting_up/source/ctest.rst',
     'setting_up/source/cpp_c.rst',
     'setting_up/source.rst',
-    'setting_up/conda.rst',
+    'setting_up/package.rst',
     'setting_up/linking.rst',
     'api_full/fortran.rst',
     'api_full/python.rst',

docs/user_guide/source/setting_up/package.rst

@@ -0,0 +1,28 @@
+##########
+As Package
+##########
+
+*****
+Conda
+*****
+
+Currently ADIOS 2 can be obtained from anaconda cloud:
+
+* x86-64 and MacOS: `williamfgc adios2-openmpi adios2-mpich adios2-nompi <https://anaconda.org/williamfgc>`_  
+* Multiple archs: `conda-forge adios2 <https://anaconda.org/conda-forge/adios2>`_
+
+
+*****
+Spack
+*****
+
+ADIOS 2 is fully supported in the latest Spack `adios2 package <https://spack.readthedocs.io/en/latest/package_list.html#adios2>`_
+
+
+******
+Docker
+******
+
+Docker images including building and installation of ADIOS 2 in a container: 
+* Ubuntu 18.04: under `scripts/docker/images/ubuntu18.04/Dockerfile <https://github.com/ornladios/ADIOS2/tree/master/scripts/docker/images/ubuntu18.04/Dockerfile>`_
+

docs/user_guide/source/setting_up/setting_up.rst

@@ -1,4 +1,4 @@
 
 .. include:: source.rst
-.. include:: conda.rst
+.. include:: package.rst
 .. include:: linking.rst

docs/user_guide/source/setting_up/source.rst

@@ -3,7 +3,7 @@ Install from Source
 ###################
 
 ADIOS2 uses `CMake <https://cmake.org/>`_ version 3.6 or above, for building,
-testing and installing the library, examples and utilities.
+testing and installing the library and utilities.
 
 .. include:: source/cmake.rst
 .. include:: source/hpc_systems.rst

scripts/docker/images/ubuntu18.04/Dockerfile

@@ -0,0 +1,83 @@
+# Dockerfile for ADIOS2 on Ubuntu 18.04 
+#####################################################################################################
+#
+# This is an example Dockerfile image to build and install adios2 master from source in Ubuntu 18.04
+# including dependencies from apt-get and (optionally) the gray-scott tutorial example at the end of 
+# this file
+# Modify at your convenience
+# Usage ($ is your terminal prompt): 
+# 1) build docker image
+# 	$ cd ADIOS2/scripts/docker/images/ubuntu18.04 ; docker build .
+#   or:
+# 	$ docker build -f /path/to/ADIOS2/scripts/docker/images/ubuntu18.04/Dockerfile .
+#   add a tag:
+# 	$ docker build -f /path/to/ADIOS2/scripts/docker/images/ubuntu18.04/Dockerfile -t adios2/master
+# 2) run the built image in a container
+#   $ docker run -i -t ubuntu:18.04 /bin/bash
+#   or with tag:
+#   $ docker run -i -t adios2/master /bin/bash 
+#####################################################################################################  
+
+FROM ubuntu:18.04
+MAINTAINER William F Godoy [email protected]
+RUN apt-get update -y &&\
+    apt-get install build-essential sudo gfortran \
+                    openmpi-bin libopenmpi-dev libzmq3-dev \
+                    python3-dev python3-mpi4py python3-numpy python3-pip \
+                    libblosc-dev libbz2-dev libpng-dev \
+                    libhdf5-openmpi-dev \
+                    cmake \
+                    git vim \
+                    -y
+
+RUN cd /tmp &&\
+    mkdir -p Software
+
+RUN cd /tmp/Software &&\
+    git clone https://github.com/LLNL/zfp.git --single-branch --branch 0.5.5 --depth 1 &&\
+    mkdir -p zfp-build &&\
+    cd zfp-build &&\
+    cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/opt/zfp -DBUILD_EXAMPLES=OFF ../zfp &&\
+    make -j &&\
+    make install
+
+RUN cd /tmp/Software &&\
+    git clone https://github.com/disheng222/SZ.git --single-branch --branch v2.1.5.0 --depth 1 &&\
+    cd SZ &&\
+    ./configure --prefix=/opt/sz &&\
+    make -j &&\
+    make install
+
+RUN cd /tmp/Software &&\
+    git clone https://github.com/ornladios/ADIOS2.git --single-branch --branch master --depth 1 &&\
+    mkdir -p ADIOS2-build &&\
+    cd ADIOS2-build &&\
+    cmake -DCMAKE_INSTALL_PREFIX=/opt/adios2 \
+          -DZFP_ROOT=/opt/zfp \
+          -DSZ_ROOT=/opt/sz \
+          -DADIOS2_BUILD_TESTING=OFF \
+          -DADIOS2_BUILD_EXAMPLES=OFF \
+          ../ADIOS2 &&\
+    make -j &&\
+    make install
+
+RUN cd /tmp &&\
+    rm -fr Software
+
+ENV PYTHONPATH=/usr/local/lib/python3.6/site-packages/:/opt/adios2/lib/python3.6/site-packages
+ENV PATH=/opt/adios2/bin
+ENV LD_LIBRARY_PATH=/usr/local/lib:/opt/zfp/lib:/opt/sz/lib:/opt/adios2/lib
+
+# optionally add a new user and start using adios2 with an example
+# RUN useradd -m wfg && echo "wfg:wfg" | chpasswd && adduser wfg sudo
+# USER wfg
+# CMD /bin/bash
+
+# RUN cd ~ &&\
+#    git clone https://github.com/pnorbert/adiosvm.git --single-branch --branch master --depth 1 &&\
+#    cd adiosvm/Tutorial/gray-scott &&\
+#    mkdir -p build &&\
+#    cd build &&\
+#    cmake -DCMAKE_BUILD_TYPE=Release .. &&\
+#    make &&\
+#    cd ..

scripts/runconf/runconf.sh

@@ -1,5 +1,11 @@
 #!/bin/bash
 
+# Usage: 
+#   ./runconf.sh ON lean   (MPI On, minimal adios2)
+#   ./runconf.sh ON        (MPI On, adios2 builds with all dependencies, might need -D*_DIR or -D*_ROOT)
+#   ./runconf.sh OFF lean  (MPI Off, minimal adios2)  
+#   ./runconf.sh OFF       (MPI Off, adios2 builds with all dependencies, might need -D*_DIR or -D*_ROOT)
+
 USE_MPI=${1:-ON}
 echo USE_MPI=$USE_MPI
 
@@ -11,14 +17,15 @@ if [ x"$2" == x"lean" ]; then
     VALUE=OFF
 fi
 
-ADIOS2_SOURCE=${PWD}/..
+# truly out of source build
+ADIOS2_SOURCE=${PWD}/../ADIOS2
 echo "ADIOS2_SOURCE: " ${ADIOS2_SOURCE}
 
 # -DCMAKE_BUILD_TYPE options: Debug / Release / RelWithDebInfo / MinSizeRel
 # Change -DCMAKE_INSTALL_PREFIX to your preferred location
 # Default mode is AUTO
 # If mode option is ON, it will fail if cmake can't find the dependency location
-# Pass *_ROOT for dependency locations e.g. -DZFP_ROOT=/opt/zfp
+# Pass *_ROOT or *_DIR for dependency locations e.g. -DZFP_ROOT=/opt/zfp
 
 cmake -DCMAKE_INSTALL_PREFIX=${PWD}/install \
           -DADIOS2_USE_MPI=${USE_MPI} \
@@ -27,12 +34,20 @@ cmake -DCMAKE_INSTALL_PREFIX=${PWD}/install \
           -DADIOS2_USE_Fortran=${VALUE} \
           -DADIOS2_USE_Python=${VALUE} \
           -DADIOS2_USE_SST=${VALUE} \
+          -DADIOS2_USE_SSC=${VALUE} \
+          -DADIOS2_USE_DataMan=${VALUE} \
+          -DADIOS2_USE_DataSpaces=${VALUE} \
+          -DADIOS2_USE_Table=${VALUE} \
           -DADIOS2_USE_BZip2=${VALUE} \
+          -DADIOS2_USE_PNG=${VALUE} \
+          -DADIOS2_USE_Blosc=${VALUE} \
+          -DADIOS2_USE_SZ=${VALUE} \
           -DADIOS2_USE_ZFP=${VALUE} \
           -DBUILD_SHARED_LIBS=ON \
           -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
           -DCMAKE_BUILD_TYPE=Release \
           -DADIOS2_BUILD_TESTING=${VALUE} \
           -DADIOS2_BUILD_EXAMPLES=${VALUE} \
+          -DADIOS2_USE_Endian_Reverse=OFF \
           ${ADIOS2_SOURCE}