Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDFS-16043. Add markedDeleteBlockScrubberThread to delete blocks asyn… #3882

Merged
merged 1 commit into from
Jan 15, 2022

Conversation

zhuxiangyi
Copy link
Contributor

cherry-pick to branch-3.3

@zhuxiangyi
Copy link
Contributor Author

@Hexiaoqiao I created a new PR here.

@zhuxiangyi
Copy link
Contributor Author

Looks like there's a problem, I'll fix it this week.

@Hexiaoqiao Hexiaoqiao requested a review from sunchao January 14, 2022 03:18
@Hexiaoqiao
Copy link
Contributor

@zhuxiangyi It seems building failed now. Would you mind to fix it first?

@zhuxiangyi
Copy link
Contributor Author

@Hexiaoqiao Resubmitted waiting for Jenkins build.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 36s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 15 new or modified test files.
_ branch-3.3 Compile Tests _
+1 💚 mvninstall 31m 6s branch-3.3 passed
+1 💚 compile 1m 16s branch-3.3 passed
+1 💚 checkstyle 0m 51s branch-3.3 passed
+1 💚 mvnsite 1m 21s branch-3.3 passed
+1 💚 javadoc 1m 28s branch-3.3 passed
+1 💚 spotbugs 3m 12s branch-3.3 passed
+1 💚 shadedclient 25m 34s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 13s the patch passed
+1 💚 compile 1m 5s the patch passed
+1 💚 javac 1m 5s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 43s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 451 unchanged - 2 fixed = 453 total (was 453)
+1 💚 mvnsite 1m 10s the patch passed
+1 💚 javadoc 1m 23s the patch passed
+1 💚 spotbugs 3m 7s the patch passed
+1 💚 shadedclient 25m 6s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 190m 1s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 asflicense 0m 47s The patch does not generate ASF License warnings.
287m 37s
Reason Tests
Failed junit tests hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized
hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3882/2/artifact/out/Dockerfile
GITHUB PR #3882
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux 04339ec2a237 4.15.0-161-generic #169-Ubuntu SMP Fri Oct 15 13:41:54 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.3 / f811afd
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3882/2/testReport/
Max. process+thread count 3568 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3882/2/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@zhuxiangyi
Copy link
Contributor Author

@Hexiaoqiao The build failed, doesn't seem to be related to this patch, can you trigger Jinkens to rebuild?

@Hexiaoqiao
Copy link
Contributor

@Hexiaoqiao The build failed, doesn't seem to be related to this patch, can you trigger Jinkens to rebuild?

Trigger again. Let's wait what it will say.

@zhuxiangyi
Copy link
Contributor Author

`[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 19.417 s <<< FAILURE! - in org.apache.hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized
[ERROR] testWithKerberizedCluster(org.apache.hadoop.hdfs.TestDFSInotifyEventInputStreamKerberized) Time elapsed: 19.326 s <<< ERROR!
java.io.IOException: DestHost:destPort localhost:10476 , LocalHost:localPort 04339ec2a237/172.17.0.2:0. Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

[ERROR] Tests run: 4, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 205.883 s <<< FAILURE! - in org.apache.hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
[ERROR] testBalancerWithObserverWithFailedNode(org.apache.hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes) Time elapsed: 180.186 s <<< ERROR!
org.junit.runners.model.TestTimedOutException: test timed out after 180000 milliseconds`

@Hexiaoqiao The build failed again. The reason for the failure was the same as last time. The above two tests passed in my local build.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 35s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 15 new or modified test files.
_ branch-3.3 Compile Tests _
+1 💚 mvninstall 31m 3s branch-3.3 passed
+1 💚 compile 1m 13s branch-3.3 passed
+1 💚 checkstyle 0m 53s branch-3.3 passed
+1 💚 mvnsite 1m 22s branch-3.3 passed
+1 💚 javadoc 1m 33s branch-3.3 passed
+1 💚 spotbugs 3m 4s branch-3.3 passed
+1 💚 shadedclient 25m 18s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 10s the patch passed
+1 💚 compile 1m 5s the patch passed
+1 💚 javac 1m 5s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 0m 43s /results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 451 unchanged - 2 fixed = 453 total (was 453)
+1 💚 mvnsite 1m 12s the patch passed
+1 💚 javadoc 1m 24s the patch passed
+1 💚 spotbugs 3m 2s the patch passed
+1 💚 shadedclient 24m 48s patch has no errors when building and testing our client artifacts.
_ Other Tests _
-1 ❌ unit 190m 30s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 asflicense 0m 45s The patch does not generate ASF License warnings.
287m 23s
Reason Tests
Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
hadoop.hdfs.server.datanode.TestBPOfferService
hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3882/3/artifact/out/Dockerfile
GITHUB PR #3882
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname Linux 20e0dafc5572 4.15.0-161-generic #169-Ubuntu SMP Fri Oct 15 13:41:54 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.3 / f811afd
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3882/3/testReport/
Max. process+thread count 3752 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3882/3/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@apache apache deleted a comment from hadoop-yetus Jan 14, 2022
@Hexiaoqiao
Copy link
Contributor

Thanks @zhuxiangyi for your works. Just checking the failed unit tests, It seems not related to this changes.
Overall LGTM, +1 from my side.
@sunchao would you mind to give another review? Will commit if no more objection, Thanks.

@Hexiaoqiao Hexiaoqiao merged commit b5e7f59 into apache:branch-3.3 Jan 15, 2022
@Hexiaoqiao
Copy link
Contributor

Committed to branch-3.3. Thanks @zhuxiangyi for your contributions!

@sunchao
Copy link
Member

sunchao commented Jan 15, 2022

Thanks, looks OK to me too. I'll cherry-pick this to branch-3.3.2 now.

sunchao pushed a commit that referenced this pull request Jan 15, 2022
…chronously (#3882).  Contributed by Xiangyi Zhu.

Signed-off-by: He Xiaoqiao <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants