Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

HADOOP-16769 LocalDirAllocator to provide diagnostics when file creat… #1892

Open
wants to merge 2 commits into
base: trunk
Choose a base branch
from
Open

HADOOP-16769 LocalDirAllocator to provide diagnostics when file creat… #1892

wants to merge 2 commits into from
+40 −4

Conversation

ramesh0201
Copy link

No description provided.

steveloughran
steveloughran requested changes Apr 21, 2020
View reviewed changes
Copy link
Contributor

@steveloughran steveloughran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good, the main thing it needs is that caught exception to be propagated to be the inner cause of the new exception raised. That way complicated failures don't lose their stack traces in the logs we get to see

hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java Show resolved Hide resolved
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java Outdated
if (errorText != null) {
newErrorText = newErrorText + " due to " + errorText;
}
throw new DiskErrorException(newErrorText);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if any exception had been caught and stored in L484, use it in the constructor/setCause here. Stack traces are too important to lose.

...p-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalDirAllocator.java Outdated
String dir0 = buildBufferDir(ROOT, 0);
String dir1 = buildBufferDir(ROOT, 1);
conf.set(CONTEXT, dir0 + "," + dir1);
LambdaTestUtils.intercept(DiskErrorException.class, "as the max capacity" +
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor nit, move the "as the max" capacity string onto a line on its own' merge it back to a single string,

@ramesh0201
Copy link
Author

Thank you for the comments. For some reason I am not getting any email notifications about these git PR review comments. I just manually verified and found I missed to address these. I have addressed the comments and updated the PR. Can you please help me confirm the changes are expected?

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 34s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 @author 0m 1s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 23m 32s trunk passed
+1 💚 compile 25m 3s trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚 compile 19m 42s trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚 checkstyle 0m 47s trunk passed
+1 💚 mvnsite 1m 26s trunk passed
+1 💚 shadedclient 16m 8s branch has no errors when building and testing our client artifacts.
-1 ❌ javadoc 0m 36s hadoop-common in trunk failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.
+1 💚 javadoc 0m 56s trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+0 🆗 spotbugs 2m 29s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 2m 27s trunk passed
_ Patch Compile Tests _
+1 💚 mvninstall 0m 54s the patch passed
+1 💚 compile 19m 51s the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1
+1 💚 javac 19m 51s the patch passed
+1 💚 compile 18m 30s the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚 javac 18m 30s the patch passed
-0 ⚠️ checkstyle 0m 48s hadoop-common-project/hadoop-common: The patch generated 1 new + 39 unchanged - 0 fixed = 40 total (was 39)
+1 💚 mvnsite 1m 22s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedclient 14m 22s patch has no errors when building and testing our client artifacts.
-1 ❌ javadoc 0m 43s hadoop-common in the patch failed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.
+1 💚 javadoc 1m 6s the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚 findbugs 2m 46s the patch passed
_ Other Tests _
+1 💚 unit 10m 31s hadoop-common in the patch passed.
+1 💚 asflicense 0m 53s The patch does not generate ASF License warnings.
165m 10s
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1892/3/artifact/out/Dockerfile
GITHUB PR #1892
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux 072e291fbc9e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality personality/hadoop.sh
git revision trunk / e756fe3
Default Java Private Build-1.8.0_252-8u252-b09-1~18.04-b09
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09
javadoc https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1892/3/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt
checkstyle https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1892/3/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
javadoc https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1892/3/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1.txt
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1892/3/testReport/
Max. process+thread count 3046 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-1892/3/console
versions git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@apache apache deleted a comment from hadoop-yetus Oct 12, 2020
@apache apache deleted a comment from hadoop-yetus Oct 12, 2020
@steveloughran
Copy link
Contributor

Sorry, I'd completely missed this too.

@ramesh0201 -can you do a rebase and force push? I'd like to see what the checkstyle warnings were.

Other than the checkstyle issues, the patch LGTM & is almost ready to go in

@ashutoshcipher
Copy link
Contributor

Thanks @ramesh0201 for the PR. Wanted to check if you are still working on it or I can take it forward.

@ramesh0201
Copy link
Author

@ashutoshcipher Please feel free to take it forward

@ashutoshcipher ashutoshcipher mentioned this pull request Sep 1, 2022
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@steveloughran steveloughran steveloughran requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ramesh0201 @hadoop-yetus @steveloughran @ashutoshcipher