HDFS-17354: Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up #6498

ThinkerLei · 2024-01-25T04:26:22Z

In HDFS-16876 , we use SingleThreadScheduledExecutor with initialDelay is zero to clear expired namespace. because initialDelay is zero , we will invoke method in stateStore with an exception before stateStore startUp . So I think we should start clear expired namespace at RouterRpcServer RUNNING phase.

hadoop-yetus · 2024-01-25T06:52:51Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	42m 23s		trunk passed
+1 💚	compile	0m 39s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 36s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 29s		trunk passed
+1 💚	mvnsite	0m 42s		trunk passed
+1 💚	javadoc	0m 42s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 33s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	1m 21s		trunk passed
+1 💚	shadedclient	32m 51s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 30s		the patch passed
+1 💚	compile	0m 32s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 32s		the patch passed
+1 💚	compile	0m 28s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	0m 28s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 18s		the patch passed
+1 💚	mvnsite	0m 33s		the patch passed
+1 💚	javadoc	0m 27s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 23s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	1m 20s		the patch passed
+1 💚	shadedclient	32m 56s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	22m 2s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		145m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6498/1/artifact/out/Dockerfile
GITHUB PR	#6498
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 233a59bef7f1 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `ba48a82`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6498/1/testReport/
Max. process+thread count	2738 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6498/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

ThinkerLei · 2024-01-25T11:16:56Z

@simbadzina @zhangshuyan0 anyone please help to review, thanks a lot!

zhangshuyan0

The description of this PR is too vague to understand. Can you make it clearer? By the way, a ut may be needed here.

ThinkerLei · 2024-01-26T12:32:28Z

The description of this PR is too vague to understand. Can you make it clearer? By the way, a ut may be needed here.

Renew description and thanks a lot for your reply @zhangshuyan0

simbadzina · 2024-01-27T00:04:16Z

Changes generally looks okay to me. Is this just an optimization to avoid clearing a map which is empty, or there can be an error if we clear before the router is in the RUNNING state.

Can you please add a test case.

…ring router start up

hadoop-yetus · 2024-02-07T11:12:57Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	12m 39s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	43m 6s		trunk passed
+1 💚	compile	0m 39s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 37s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	0m 30s		trunk passed
+1 💚	mvnsite	0m 42s		trunk passed
+1 💚	javadoc	0m 42s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 31s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	1m 20s		trunk passed
+1 💚	shadedclient	32m 59s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 30s		the patch passed
+1 💚	compile	0m 32s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 32s		the patch passed
+1 💚	compile	0m 29s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	0m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 18s		the patch passed
+1 💚	mvnsite	0m 31s		the patch passed
+1 💚	javadoc	0m 28s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 24s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	1m 20s		the patch passed
+1 💚	shadedclient	32m 48s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	9m 43s		hadoop-hdfs-rbf in the patch passed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		145m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6498/2/artifact/out/Dockerfile
GITHUB PR	#6498
JIRA Issue	HDFS-17354
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 3c4257fdeb6b 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `24fc113`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6498/2/testReport/
Max. process+thread count	2572 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6498/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

ThinkerLei · 2024-02-07T11:29:26Z

@zhangshuyan0 @simbadzina Add a test case, please help to review again.

zhangshuyan0

+1. LGTM.

simbadzina

LGTM. Thanks for the contribution @ThinkerLei

…ring router start up (#6498) (cherry picked from commit 12498b3)

github-actions bot added HDFS trunk RBF labels Jan 25, 2024

zhangshuyan0 reviewed Jan 26, 2024

View reviewed changes

HDFS-17354:Delay invoke clearStaleNamespacesInRouterStateIdContext du…

24fc113

…ring router start up

ThinkerLei force-pushed the HDFS-17354 branch from ba48a82 to 24fc113 Compare February 7, 2024 08:46

zhangshuyan0 approved these changes Feb 8, 2024

View reviewed changes

simbadzina approved these changes Feb 14, 2024

View reviewed changes

simbadzina merged commit 12498b3 into apache:trunk Feb 14, 2024
4 checks passed

tasanuma pushed a commit that referenced this pull request Mar 26, 2024

HDFS-17354:Delay invoke clearStaleNamespacesInRouterStateIdContext du…

317d2db

…ring router start up (#6498) (cherry picked from commit 12498b3)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-17354: Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up #6498

HDFS-17354: Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up #6498

ThinkerLei commented Jan 25, 2024 •

edited

Loading

hadoop-yetus commented Jan 25, 2024

ThinkerLei commented Jan 25, 2024

zhangshuyan0 left a comment

ThinkerLei commented Jan 26, 2024

simbadzina commented Jan 27, 2024

hadoop-yetus commented Feb 7, 2024

ThinkerLei commented Feb 7, 2024

zhangshuyan0 left a comment

simbadzina left a comment

HDFS-17354: Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up #6498

HDFS-17354: Delay invoke clearStaleNamespacesInRouterStateIdContext during router start up #6498

Conversation

ThinkerLei commented Jan 25, 2024 • edited Loading

hadoop-yetus commented Jan 25, 2024

ThinkerLei commented Jan 25, 2024

zhangshuyan0 left a comment

Choose a reason for hiding this comment

ThinkerLei commented Jan 26, 2024

simbadzina commented Jan 27, 2024

hadoop-yetus commented Feb 7, 2024

ThinkerLei commented Feb 7, 2024

zhangshuyan0 left a comment

Choose a reason for hiding this comment

simbadzina left a comment

Choose a reason for hiding this comment

ThinkerLei commented Jan 25, 2024 •

edited

Loading