[WIP] ADIOS2: Fix Parallel Resize #1126
Conversation
This was the exact same behavior in ADIOS2 before resizing was implemented there, so I guess it was forgotten somewhere. |
Names are: - `ADIOS1` - `MPI_ADIOS1` - `ADIOS2` - `MPI_ADIOS2`
ax3l
force-pushed
the
fix-resizeADIOS2
branch
from
81b34e2 to
bdb114e
Compare
| @@ -390,7 +390,7 @@ extendDataset( std::string const & ext ) | |||
| std::iota( data2.begin(), data2.end(), 25 ); | |||
| { | |||
| Series write( filename, Access::CREATE, MPI_COMM_WORLD ); | |||
There was a problem hiding this comment.
Should we move the write below the return for ADIOS1?
Otherwise we create that empty ghost series.
There was a problem hiding this comment.
It's a bit unfortunate, but we have to do it this way unfortunately, see write.backend() in the if condition. However, I can add a commit in #1043 that uses {"backend": "adios2"} to explicitly select backends up front.
|
Marked as WIP as I am waiting for further tests to generally pass in ECP-WarpX/WarpX#1898 before trying the next reproducer. |
We found in WarpX (ECP-WarpX/WarpX#1898) that MPI-parallel resizes (at least in file-based encoding) do not yet work with the ADIOS2 backend (parallel HDF5 is fine).
The current behavior is that following resizes are not performed or do not show up in read as a changed extent/shape at least. (Tests do not crash in write because ADIOS let's data be written independent of extent, which is meta-data.)