HDFS-16427. Add debug log for BlockManager#chooseExcessRedundancyStriped #3888
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
💔 -1 overall
This message was automatically generated. |
|
@tomscut Sorry for being late. I agree with adding more EC logs, but I feel that the log information is too detailed, and some variables duplicate with other variables. Could you please make it more concise and descriptive? |
Thanks @tasanuma for your review. These states are related to each other, and if the upper and lower states don't match, there is something wrong with the process in between. Maybe If simplify this log, I would remove |
|
At that time, we found that the |
|
💔 -1 overall
This message was automatically generated. |
|
Hi @tasanuma , could you please take a look again. Thank you very much. |
|
@tomscut IMHO, adding unit tests and assertions would be better than logging all variables to prevent inconsistent conditions between variables. For example, I suggest adding the following Preconditions to avoid inconsistencies between Preconditions.checkArgument(candidates.containsAll(replicasToDelete));I still agree with adding more EC debug logs for not only developers but also non-developers. I prefer more descriptive logging rather than using variable names directly. And I think LOG.debug("Choose redundant EC replicas to delete from blk_{} which is located in {}", sblk.getBlockId(), storage2index);
LOG.debug("Storages with candidate blocks to be deleted: {}", candidates);
LOG.debug("Storages with blocks to be deleted: {}", replicasToDelete); |
Thanks @tasanuma for your advice, it makes sense to me. I updated it. Please take a look. Thank you. |
tasanuma
reviewed
Jan 26, 2022
...ct/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
Outdated
Show resolved
Hide resolved
Co-authored-by: tasanuma <[email protected]> Co-authored-by: Takanobu Asanuma <[email protected]>
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
tasanuma
approved these changes
Jan 27, 2022
|
Merged it. Thanks for your contribution, @tomscut! |
bogthe
pushed a commit
to bogthe/hadoop
that referenced
this pull request
Feb 2, 2022
…ped (apache#3888) (cherry picked from commit 6136d63)
HarshitGupta11
pushed a commit
to HarshitGupta11/hadoop
that referenced
this pull request
Nov 28, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
JIRA: HDFS-16427.
To solve this issue HDFS-16420 , we added some debug logs, which were also necessary. If there are other problems, we set the log level to DEBUG, which is convenient to analyze it.