From 49e80a17ed4dd68ee6a333a93eb8736d9cda382c Mon Sep 17 00:00:00 2001 From: Vicente Adolfo Bolea Sanchez Date: Wed, 24 May 2023 20:34:20 -0400 Subject: [PATCH] docs: update whatsnew --- .../source/introduction/whatsnew.rst | 46 +++++++++++-------- 1 file changed, 27 insertions(+), 19 deletions(-) diff --git a/docs/user_guide/source/introduction/whatsnew.rst b/docs/user_guide/source/introduction/whatsnew.rst index af8aae7c08..39cefb7509 100644 --- a/docs/user_guide/source/introduction/whatsnew.rst +++ b/docs/user_guide/source/introduction/whatsnew.rst @@ -1,26 +1,34 @@ -################## -What's new in 2.8? -################## +================== +What's new in 2.9? +================== -Important changes to the API +Summary +======= - * **adios2::Mode::ReadRandomAccess** mode is introduced for reading files with access to all steps. - BeginStep/EndStep calls are *NOT allowed*. SetStepSelection() can be used to access specific steps in the file. - * **adios2::Mode::Read** mode now requires using BeginStep/EndStep loop to access steps serially one by one. Variable inquiry - fails outside BeginStep/EndStep sections. You need to modify your Open() statement to use the random-access mode if your - code wants to access all steps in any order in an existing file. - * **adios2::ADIOS::EnterComputationBlock()**, **adios2::ADIOS::ExitComputationBlock()** are hints to ADIOS that a process is in a computing (i.e. non-communicating) phase. BP5 asynchronous I/O operations can schedule writing during such phases to avoid interfering with the application's own communication. - * GPU-aware I/O supports passing device-memory data pointers to the ADIOS2 `Put()/Get()` functions, and ADIOS2 will automatically download/upload data from/to the device during I/O. Alternatively, an extra member function of the Variable class, **SetMemorySpace(const adios2::MemorySpace mem)** can explicitly tell ADIOS2 whether the pointer points to device memory or host memory. +This is a major release with new features and lots of bug fixes. -New features +General +------- - * **BP5** data format and engine. This new engine optimizes for many variables and many steps at large scale. - It is also more memory efficient than previous engines, see :ref:`BP5`. - * **Plugin** architecture to support external *engines* and *operators* outside the ADIOS2 installation, see :ref:`Plugins` - * **GPU-Aware I/O** for reading/writing data to/from device memory, using CUDA (NVidia GPUs only), see :ref:`GPU-aware I/O` +- GPU-Aware I/O enabled by using Kokkos. Device pointers can be passed to Put()/Get() calls directly. Kokkos 3.7.x required for this release. Works with CUDA, HIP and Kokkos applications. https://adios2.readthedocs.io/en/latest/advanced/gpu_aware.html#gpu-aware-i-o +- GPU-compression. MGARD and ZFP operators can compress data on GPU if they are built for GPU. MGARD operator can be fed with host/device pointers and will move data automaticaly. ZFP operator requires matching data and compressor location. +- Joined Array concept (besides Global Array and Local Array), which lets writers dump Local Arrays (no offsets no global shape) that are put together into a Global Array by the reader. One dimension of the arrays is selected for this join operation, while other dimensions must be the same for all writers. https://adios2.readthedocs.io/en/latest/components/components.html?highlight=Joined#shapes -Other changes +File I/O +-------- - * SST scales better for large N-to-1 staging, by managing the limits of outstanding remote direct memory access requests. - Of course one still introduces a literal bottleneck with such a pattern into an in situ workflow. +- Default File engine is now BP5. If for some reason this causes problems, manually specify using "BP4" for your application. +- BP5 engine supports multithreaded reading to accelerate read performance for low-core counts. +- BP5 Two level metadata aggregation and reduction reduced memory impact of collecting metadata and therefore is more scalable in terms of numbers of variables and writers than BP4. +- Uses Blosc-2 instead of Blosc for lossless compression. The new compression operator is backward compatible with old files compressed with blosc. The name of the operator remains "blosc". +Staging +------- + +- UCX dataplane added for SST staging engine to support networks under the UCX consortium +- MPI dataplane added for SST staging engine. It relies on MPI intercommunicators to connect multiple independent MPI applications for staging purposes. Applications must enable multithreaded MPI for this dataplane. + +Experimental features +--------------------- + +- Preliminary support for data structs. A struct can have single variables of basic types, and 1D fixed size arrays of basic types. Supported by BP5, SST and SSC engines.