HDDS-1786 : Datanodes takeSnapshot should delete previously created s… #1163
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
July 25, 2019 14:58
…napshots. (Added negative unit test)
|
@mukul1987 / @lokeshj1703 / @bharatviswa504 / @bshashikant Please review when you have a chance. |
|
/label ozone |
|
cc @swagle |
swagle
reviewed
Jul 25, 2019
...a/org/apache/hadoop/ozone/container/common/transport/server/ratis/ContainerStateMachine.java
Outdated
Show resolved
Hide resolved
...tion-test/src/test/java/org/apache/hadoop/ozone/client/rpc/TestContainerStateMachineInt.java
Outdated
Show resolved
Hide resolved
|
Thanks @avijayanhwx for working on this. In the patch , it seems at one point of time we will only have one snapshot in raft log directory. The default value can be 1. This will help in cases where the latest snapshot file gets corrupted/inaccessible but if we have older snapshots(in case if we have not purged the raft logs) we can always roll back and start up from there. This will be more useful in debugging issues as well. |
|
💔 -1 overall
This message was automatically generated. |
|
I will work on adding a configurable policy in Ratis so that Ozone Manager can also configure it as needed. After that is done, I will update this PR to use that policy. cc @mukul1987 |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
Thanks @avijayanhwx for working on this. The changes look good. I would prefer to have a test in Ozone as well to verify the snapshot retention behaviour of Ratis so that, in case there are changes made in Ratis related to this, we should be able to catch this here in ozone. A simple unit test where can change snapshot threshold to 1 entry and verify we have n snapshot files after n transactions in the raft log directory. |
Added unit test. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
Thanks @avijayanhwx for working on the patch. The patch looks good to me. Can u check the compilation issue? |
@bshashikant I have fixed an edge case in the unit test. The rest of the failures are unrelated. |
|
💔 -1 overall
This message was automatically generated. |
|
Thanks @avijayanhwx for the contribution. I have committed this. |
amahussein
pushed a commit
to amahussein/hadoop
that referenced
this pull request
Oct 29, 2019
RogPodge
pushed a commit
to RogPodge/hadoop
that referenced
this pull request
Mar 25, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
…napshots.
Right now, after after taking a new snapshot, the previous snapshot file is left in the raft log directory. When a new snapshot is taken, the previous snapshot should be deleted.
Fixed the issue in the takeSnapshot method. Manually tested on docker and added unit test.