Using adios2 with multiple mpi communicators #3336
-
|
If one has several mpi communicators and defines an adios object for each of these comms, then, is reading a .bp file using engines created using these objects a valid and safe operation ? If say |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Yes it is valid and safe. You can also have one ADIOS object that has a communicator that is a superset of the other communicators, and use the individual communicators explicitly in the io.Open() calls. Open() inherits the communicator from the ADIOS object when it is not provided as an argument. A .bp file is actually a directory, which can have several data.* files. So if many process read data from a dataset and decompose the read requests similar to how the data was written, different readers will read data from different files. But in general, every reader process is working on its own (in current BP engines) and hence many process may read data from the same data file. What situation do you have that needs to be avoided? |
Beta Was this translation helpful? Give feedback.
-
|
Thank you for the answer. Just as I wrote in the question, I am trying to read a dataset (my bad that I called it a file) that was written with a single mpi communicator (not world), using several similar mpi communicators. I was just trying to make sure I am not doing something inherently wrong before actually implemnting it. |
Beta Was this translation helpful? Give feedback.
Yes it is valid and safe. You can also have one ADIOS object that has a communicator that is a superset of the other communicators, and use the individual communicators explicitly in the io.Open() calls. Open() inherits the communicator from the ADIOS object when it is not provided as an argument.
A .bp file is actually a directory, which can have several data.* files. So if many process read data from a dataset and decompose the read requests similar to how the data was written, different readers will read data from different files. But in general, every reader process is working on its own (in current BP engines) and hence many process may read data from the same data file.
What situati…