Skip to content

Commit

Permalink
Merge pull request #3082 from pnorbert/bufferchunksize-default-128mb
Browse files Browse the repository at this point in the history
Set BP5's BufferChunkSize to 128MB by default.
  • Loading branch information
pnorbert authored Mar 2, 2022
2 parents 9511a41 + 4a41c8f commit 127f280
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 4 deletions.
4 changes: 2 additions & 2 deletions docs/user_guide/source/engines/bp5.rst
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ This engine allows the user to fine tune the buffering operations through the fo

#. **BufferVType**: *chunk* or *malloc*, default is chunking. Chunking maintains the buffer as a list of memory blocks, either ADIOS-owned for sync-ed Puts and small Puts, and user-owned pointers of deferred Puts. Malloc maintains a single memory block and extends it (reallocates) whenever more data is buffered. Chunking incurs extra cost in I/O by having to write data in chunks (multiple write system calls), which can be helped by increasing *BufferChunkSize* and *MinDeferredSize*. Malloc incurs extra cost by reallocating memory whenever more data is buffered (by Put()), which can be helped by increasing *InitialBufferSize*.

#. **BufferChunkSize**: (for *chunk* buffer type) The size of each memory buffer chunk, default is 16MB but it is worth increasing up to 2GB if possible for maximum write performance.
#. **BufferChunkSize**: (for *chunk* buffer type) The size of each memory buffer chunk, default is 128MB but it is worth increasing up to 2GB if possible for maximum write performance.

#. **MinDeferredSize**: (for *chunk* buffer type) Small user variables are always buffered, default is 4MB.

Expand Down Expand Up @@ -134,7 +134,7 @@ This engine allows the user to fine tune the buffering operations through the fo
StripeSize integer+units **4KB**
MaxShmSize integer+units **4294762496**
BufferVType string **chunk**, malloc
BufferChunkSize integer+units **16MB**, worth increasing up to min(2GB, datasize/process/step)
BufferChunkSize integer+units **128MB**, worth increasing up to min(2GB, datasize/process/step)
MinDeferredSize integer+units **4MB**
InitialBufferSize float+units >= 16Kb **16Kb**, 10Mb, 0.5Gb
GrowthFactor float > 1 **1.05**, 1.01, 1.5, 2
Expand Down
4 changes: 2 additions & 2 deletions source/adios2/common/ADIOSTypes.h
Original file line number Diff line number Diff line change
Expand Up @@ -248,8 +248,8 @@ constexpr uint64_t DefaultMaxBufferSize = MaxSizeT - 1;
constexpr float DefaultBufferGrowthFactor = 1.05f;

/** default Buffer Chunk Size
* 16Mb */
constexpr uint64_t DefaultBufferChunkSize = 16 * 1024 * 1024;
* 128Mb */
constexpr uint64_t DefaultBufferChunkSize = 128 * 1024 * 1024;

/** default minimum size not copying deferred writes
* 4Mb */
Expand Down

0 comments on commit 127f280

Please sign in to comment.