-
Notifications
You must be signed in to change notification settings - Fork 128
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Truncate option for Engine Open mode Append #2775
Comments
Just to add, more precisely, we would like to specify "truncate all steps after ADIOS step |
@franzpoeschel BP5 in the master branch now has a parameter AppendAfterSteps. It enables overwriting some steps as you requested. If you have 10 steps in the file,
Default value is MAX_INT, so it always appends after the last step. -1 would be another good default value. We won't add this feature to BP4 because it would require implementing the entire metadata parsing process of the read side in the writing engine. |
Great, thanks for this! |
This is finally going to work properly in 2.8.1 |
Why is this feature important?
Scenario:
Consequence: After restarting from a checkpoint:
=> The output file will contain the same steps twice
Solution: Allow to truncate a number of last steps when opening a file for appending
What is the potential impact of this feature in the community?
ADIOS steps can be used more sensibly for output in simulations that use checkpointing
Is your feature request related to a problem? Please describe.
see above
Describe the solution you'd like and potential required effort
see above
effort depends on the file format
Describe alternatives you've considered and potential required effort
Alternative: Just accept that the output file will have some time steps twice. Required effort: Additional complexity in reading data produced by ADIOS
Additional context
Discussed offline with @pnorbert and @ax3l
openPMD/openPMD-api#1007
The text was updated successfully, but these errors were encountered: