Skip to content

Commit

Permalink
Merge pull request #3634 from vicentebolea/change-whatsnew
Browse files Browse the repository at this point in the history
docs: update whatsnew
  • Loading branch information
vicentebolea authored May 25, 2023
2 parents 21f65d2 + 49e80a1 commit dbaaeba
Showing 1 changed file with 27 additions and 19 deletions.
46 changes: 27 additions & 19 deletions docs/user_guide/source/introduction/whatsnew.rst
Original file line number Diff line number Diff line change
@@ -1,26 +1,34 @@
##################
What's new in 2.8?
##################
==================
What's new in 2.9?
==================

Important changes to the API
Summary
=======

* **adios2::Mode::ReadRandomAccess** mode is introduced for reading files with access to all steps.
BeginStep/EndStep calls are *NOT allowed*. SetStepSelection() can be used to access specific steps in the file.
* **adios2::Mode::Read** mode now requires using BeginStep/EndStep loop to access steps serially one by one. Variable inquiry
fails outside BeginStep/EndStep sections. You need to modify your Open() statement to use the random-access mode if your
code wants to access all steps in any order in an existing file.
* **adios2::ADIOS::EnterComputationBlock()**, **adios2::ADIOS::ExitComputationBlock()** are hints to ADIOS that a process is in a computing (i.e. non-communicating) phase. BP5 asynchronous I/O operations can schedule writing during such phases to avoid interfering with the application's own communication.
* GPU-aware I/O supports passing device-memory data pointers to the ADIOS2 `Put()/Get()` functions, and ADIOS2 will automatically download/upload data from/to the device during I/O. Alternatively, an extra member function of the Variable class, **SetMemorySpace(const adios2::MemorySpace mem)** can explicitly tell ADIOS2 whether the pointer points to device memory or host memory.
This is a major release with new features and lots of bug fixes.

New features
General
-------

* **BP5** data format and engine. This new engine optimizes for many variables and many steps at large scale.
It is also more memory efficient than previous engines, see :ref:`BP5`.
* **Plugin** architecture to support external *engines* and *operators* outside the ADIOS2 installation, see :ref:`Plugins`
* **GPU-Aware I/O** for reading/writing data to/from device memory, using CUDA (NVidia GPUs only), see :ref:`GPU-aware I/O`
- GPU-Aware I/O enabled by using Kokkos. Device pointers can be passed to Put()/Get() calls directly. Kokkos 3.7.x required for this release. Works with CUDA, HIP and Kokkos applications. https://adios2.readthedocs.io/en/latest/advanced/gpu_aware.html#gpu-aware-i-o
- GPU-compression. MGARD and ZFP operators can compress data on GPU if they are built for GPU. MGARD operator can be fed with host/device pointers and will move data automaticaly. ZFP operator requires matching data and compressor location.
- Joined Array concept (besides Global Array and Local Array), which lets writers dump Local Arrays (no offsets no global shape) that are put together into a Global Array by the reader. One dimension of the arrays is selected for this join operation, while other dimensions must be the same for all writers. https://adios2.readthedocs.io/en/latest/components/components.html?highlight=Joined#shapes

Other changes
File I/O
--------

* SST scales better for large N-to-1 staging, by managing the limits of outstanding remote direct memory access requests.
Of course one still introduces a literal bottleneck with such a pattern into an in situ workflow.
- Default File engine is now BP5. If for some reason this causes problems, manually specify using "BP4" for your application.
- BP5 engine supports multithreaded reading to accelerate read performance for low-core counts.
- BP5 Two level metadata aggregation and reduction reduced memory impact of collecting metadata and therefore is more scalable in terms of numbers of variables and writers than BP4.
- Uses Blosc-2 instead of Blosc for lossless compression. The new compression operator is backward compatible with old files compressed with blosc. The name of the operator remains "blosc".

Staging
-------

- UCX dataplane added for SST staging engine to support networks under the UCX consortium
- MPI dataplane added for SST staging engine. It relies on MPI intercommunicators to connect multiple independent MPI applications for staging purposes. Applications must enable multithreaded MPI for this dataplane.

Experimental features
---------------------

- Preliminary support for data structs. A struct can have single variables of basic types, and 1D fixed size arrays of basic types. Supported by BP5, SST and SSC engines.

0 comments on commit dbaaeba

Please sign in to comment.