[Feature Request] Data Sieving support in Parallel HDF5 #3402
jhendersonHDF
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
A user made a feature request for supporting the data sieving feature in parallel HDF5 (specifically, the MPI I/O driver). Similar to data sieving in the serial case, this would tell HDF5 that they are doing small dataset writes and those writes should not hit disk right away when doing parallel I/O because it would result in bad performance. The library would hold off on those writes until a certain threshold or when the user decides to flush the raw data.
It is not clear whether this feature would be beneficial as it's something that MPI implementations might already be doing.
Beta Was this translation helpful? Give feedback.
All reactions