HADOOP-18851: Performance improvement for DelegationTokenSecretManager. #6001

vikaskr22 · 2023-08-30T10:27:12Z

Description of PR

AbstractDelegationTokenSecretManager's method all synchronized and are blocking each other, even multiple readers threads are blocking each other. This PR is an efforts towards optimising the synchronization contexts.
For detailed description, please go through the discussion on https://issues.apache.org/jira/browse/HADOOP-18851

How was this patch tested?

Build is working fine. And this is more about a logical change like where to acquire or release a lock. It requires careful review. There is not any functional logic change.

For code changes:

[ Y] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
[ N/A] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
[ N/A] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ N/A] If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

vikaskr22 · 2023-08-30T10:28:20Z

@jojochuang , as per our discussion on JIRA, this is the PR. Please review and provide your inputs. Thanks.

hadoop-yetus · 2023-08-30T14:18:51Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 58s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	44m 15s		trunk passed
+1 💚	compile	18m 45s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	compile	17m 0s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	checkstyle	1m 17s		trunk passed
+1 💚	mvnsite	1m 37s		trunk passed
+1 💚	javadoc	1m 11s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 47s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 35s		trunk passed
-1 ❌	shadedclient	38m 18s		branch has errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 54s		the patch passed
+1 💚	compile	17m 27s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javac	17m 27s		the patch passed
+1 💚	compile	16m 37s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	javac	16m 37s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 10s		the patch passed
+1 💚	mvnsite	1m 34s		the patch passed
+1 💚	javadoc	1m 4s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 46s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 40s		the patch passed
-1 ❌	shadedclient	38m 35s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 44s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 57s		The patch does not generate ASF License warnings.
		230m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/1/artifact/out/Dockerfile
GITHUB PR	#6001
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 0fcdd0e8cfdb 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `913d690`
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/1/testReport/
Max. process+thread count	2487 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Hexiaoqiao

@vikaskr22 Thanks for your works. leave some comments inline. FYI.

...rc/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java

Hexiaoqiao · 2023-09-03T15:57:54Z

...rc/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java

+
+      return ++currentSeqNum;
+
+    }finally{


code style:

extra blank.

leave a blank space before or after brace.

incorporated. Thanks.

Hexiaoqiao · 2023-09-03T15:58:12Z

...rc/main/java/org/apache/hadoop/security/token/delegation/ZKDelegationTokenSecretManager.java

    }

-    return ++currentSeqNum;
+


remove extra blank here.

Incorporated. Thanks.

hadoop-yetus · 2023-09-05T18:10:05Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 1s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	50m 3s		trunk passed
+1 💚	compile	18m 22s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	compile	17m 15s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	checkstyle	1m 14s		trunk passed
+1 💚	mvnsite	1m 35s		trunk passed
+1 💚	javadoc	1m 11s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 46s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 34s		trunk passed
+1 💚	shadedclient	40m 12s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 55s		the patch passed
+1 💚	compile	17m 27s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javac	17m 27s		the patch passed
+1 💚	compile	16m 43s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	javac	16m 43s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 10s		the patch passed
+1 💚	mvnsite	1m 36s		the patch passed
+1 💚	javadoc	1m 5s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 45s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 38s		the patch passed
+1 💚	shadedclient	40m 47s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 37s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 56s		The patch does not generate ASF License warnings.
		240m 17s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/2/artifact/out/Dockerfile
GITHUB PR	#6001
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux c2fdfdb580eb 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `827b881`
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/2/testReport/
Max. process+thread count	1245 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2023-09-08T15:41:42Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 57s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	43m 45s		trunk passed
+1 💚	compile	17m 23s		trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	compile	16m 25s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	checkstyle	1m 22s		trunk passed
+1 💚	mvnsite	1m 47s		trunk passed
+1 💚	javadoc	1m 21s		trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 0s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 35s		trunk passed
+1 💚	shadedclient	36m 18s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 59s		the patch passed
+1 💚	compile	16m 30s		the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javac	16m 30s		the patch passed
+1 💚	compile	16m 34s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	javac	16m 34s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 19s	/results-checkstyle-hadoop-common-project_hadoop-common.txt	hadoop-common-project/hadoop-common: The patch generated 3 new + 27 unchanged - 0 fixed = 30 total (was 27)
+1 💚	mvnsite	1m 45s		the patch passed
+1 💚	javadoc	1m 19s		the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 58s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
-1 ❌	spotbugs	2m 49s	/new-spotbugs-hadoop-common-project_hadoop-common.html	hadoop-common-project/hadoop-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	35m 56s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	19m 24s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
+1 💚	asflicense	1m 12s		The patch does not generate ASF License warnings.
		225m 50s

Reason	Tests
SpotBugs	module:hadoop-common-project/hadoop-common
	org.apache.hadoop.security.token.delegation.ZKDelegationTokenSecretManager.incrementDelegationTokenSeqNum() does not release lock on all paths At ZKDelegationTokenSecretManager.java:on all paths At ZKDelegationTokenSecretManager.java:[line 527]
Failed junit tests	hadoop.security.token.delegation.TestZKDelegationTokenSecretManager

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/5/artifact/out/Dockerfile
GITHUB PR	#6001
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 31c7383541ef 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `da69485`
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/5/testReport/
Max. process+thread count	3149 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2023-09-08T15:50:32Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 59s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	47m 29s		trunk passed
+1 💚	compile	18m 37s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	compile	16m 57s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	checkstyle	1m 15s		trunk passed
+1 💚	mvnsite	1m 37s		trunk passed
+1 💚	javadoc	1m 10s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 45s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 35s		trunk passed
+1 💚	shadedclient	39m 34s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 54s		the patch passed
+1 💚	compile	17m 20s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javac	17m 20s		the patch passed
+1 💚	compile	20m 27s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	javac	20m 27s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 25s		the patch passed
+1 💚	mvnsite	1m 44s		the patch passed
+1 💚	javadoc	1m 20s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	1m 4s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	3m 9s		the patch passed
+1 💚	shadedclient	40m 57s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 52s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 55s		The patch does not generate ASF License warnings.
		242m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/4/artifact/out/Dockerfile
GITHUB PR	#6001
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 4f676426ecde 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `827b881`
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/4/testReport/
Max. process+thread count	3144 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

...n/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java

hadoop-yetus · 2023-09-12T15:05:59Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 3s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	57m 30s		trunk passed
+1 💚	compile	18m 40s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	compile	17m 3s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	checkstyle	1m 14s		trunk passed
+1 💚	mvnsite	1m 36s		trunk passed
+1 💚	javadoc	1m 9s		trunk passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 47s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 34s		trunk passed
+1 💚	shadedclient	40m 2s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 55s		the patch passed
+1 💚	compile	17m 39s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javac	17m 39s		the patch passed
+1 💚	compile	16m 45s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	javac	16m 45s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 11s	/results-checkstyle-hadoop-common-project_hadoop-common.txt	hadoop-common-project/hadoop-common: The patch generated 3 new + 27 unchanged - 0 fixed = 30 total (was 27)
+1 💚	mvnsite	1m 34s		the patch passed
+1 💚	javadoc	1m 3s		the patch passed with JDK Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 45s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 39s		the patch passed
+1 💚	shadedclient	41m 50s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 20s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 56s		The patch does not generate ASF License warnings.
		249m 12s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/6/artifact/out/Dockerfile
GITHUB PR	#6001
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux b357d2e349cb 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `ab7a3a6`
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20+8-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/6/testReport/
Max. process+thread count	3144 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/6/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

vikaskr22 · 2023-09-13T06:52:00Z

Hi @Hexiaoqiao , @jojochuang , I have incorporated all the review comments. Now CI pipeline is failing due to lack of test cases.
As this changes are logical and not changing any functional part, so I have not added any new test case. But I patched my KMS cluster with the built dev jar and executed our benchmarking test suite and it was working fine.

Please let me know if this sufficient , if not, please suggest further. Thanks.

Hexiaoqiao · 2023-09-13T13:32:09Z

It is OK for me. Please fix the checkstyle as Yetus reported before approve and checkin. Thanks.

hadoop-yetus · 2023-09-13T17:44:25Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 58s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	44m 28s		trunk passed
+1 💚	compile	17m 7s		trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	compile	15m 59s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	checkstyle	1m 23s		trunk passed
+1 💚	mvnsite	1m 47s		trunk passed
+1 💚	javadoc	1m 22s		trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 59s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 50s		trunk passed
+1 💚	shadedclient	37m 42s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 0s		the patch passed
+1 💚	compile	17m 21s		the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javac	17m 21s		the patch passed
+1 💚	compile	16m 16s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	javac	16m 16s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 13s		the patch passed
+1 💚	mvnsite	1m 41s		the patch passed
+1 💚	javadoc	1m 13s		the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 53s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	2m 44s		the patch passed
+1 💚	shadedclient	38m 39s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 38s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 5s		The patch does not generate ASF License warnings.
		230m 30s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/7/artifact/out/Dockerfile
GITHUB PR	#6001
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux a2e40121feb1 4.15.0-212-generic #223-Ubuntu SMP Tue May 23 13:09:22 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `d5eded8`
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/7/testReport/
Max. process+thread count	1278 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6001/7/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

vikaskr22 · 2023-09-14T13:21:34Z

@Hexiaoqiao , @jojochuang , are we good to merge this ? I can see one approval. If yes, then can anyone of you please do the needful. Thanks.

Hexiaoqiao

LGTM. +1.

Hexiaoqiao · 2023-09-15T04:33:14Z

Committed to trunk. Thanks @vikaskr22 for your works and @jojochuang for your review!

…r. (apache#6001). Contributed by Vikas Kumar. Signed-off-by: Wei-Chiu Chuang <[email protected]> Signed-off-by: He Xiaoqiao <[email protected]>

…etManager. (#6001). Contributed by Vikas Kumar." This reverts commit e283375.

…etManager. (apache#6001). Contributed by Vikas Kumar." This reverts commit e283375.

HADOOP-18851: Perfm improvement for ZKDT management

913d690

github-actions bot added trunk Common labels Aug 30, 2023

Hexiaoqiao reviewed Sep 3, 2023

View reviewed changes

Hexiaoqiao changed the title ~~HADOOP-18851: Perfm improvement for ZKDT management~~ HADOOP-18851: Performance improvement for DelegationTokenSecretManager. Sep 3, 2023

Merge branch 'apache:trunk' into HADOOP-18851

827b881

vikaskr22 force-pushed the HADOOP-18851 branch from c49127c to 827b881 Compare September 8, 2023 11:46

HADOOP-18851: Perfm improvement for ZKDT management

da69485

jojochuang reviewed Sep 11, 2023

View reviewed changes

...n/java/org/apache/hadoop/security/token/delegation/AbstractDelegationTokenSecretManager.java Show resolved Hide resolved

HADOOP-18851: Perfm improvement for ZKDT management

ab7a3a6

HADOOP-18851: Perfm improvement for ZKDT management-CodeStyle changes

d5eded8

jojochuang approved these changes Sep 13, 2023

View reviewed changes

Hexiaoqiao approved these changes Sep 15, 2023

View reviewed changes

Hexiaoqiao merged commit e283375 into apache:trunk Sep 15, 2023

ChenSammi added a commit that referenced this pull request May 7, 2024

Revert "HADOOP-18851: Performance improvement for DelegationTokenSecr…

43e8ca4

…etManager. (#6001). Contributed by Vikas Kumar." This reverts commit e283375.

Hexiaoqiao mentioned this pull request May 9, 2024

HADOOP-18851. Performance improvement for DelegationTokenSecretManager #6803

Merged

2 tasks

K0K0V0K pushed a commit to K0K0V0K/hadoop that referenced this pull request May 17, 2024

Revert "HADOOP-18851: Performance improvement for DelegationTokenSecr…

77f5a5e

…etManager. (apache#6001). Contributed by Vikas Kumar." This reverts commit e283375.

K0K0V0K pushed a commit to K0K0V0K/hadoop that referenced this pull request May 17, 2024

Revert "HADOOP-18851: Performance improvement for DelegationTokenSecr…

4da8a0a

…etManager. (apache#6001). Contributed by Vikas Kumar." This reverts commit e283375.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-18851: Performance improvement for DelegationTokenSecretManager. #6001

HADOOP-18851: Performance improvement for DelegationTokenSecretManager. #6001

vikaskr22 commented Aug 30, 2023

vikaskr22 commented Aug 30, 2023

hadoop-yetus commented Aug 30, 2023

Hexiaoqiao left a comment

Hexiaoqiao Sep 3, 2023

vikaskr22 Sep 8, 2023

Hexiaoqiao Sep 3, 2023

vikaskr22 Sep 8, 2023

hadoop-yetus commented Sep 5, 2023

hadoop-yetus commented Sep 8, 2023

hadoop-yetus commented Sep 8, 2023

hadoop-yetus commented Sep 12, 2023

vikaskr22 commented Sep 13, 2023

Hexiaoqiao commented Sep 13, 2023

hadoop-yetus commented Sep 13, 2023

vikaskr22 commented Sep 14, 2023

Hexiaoqiao left a comment

Hexiaoqiao commented Sep 15, 2023

HADOOP-18851: Performance improvement for DelegationTokenSecretManager. #6001

HADOOP-18851: Performance improvement for DelegationTokenSecretManager. #6001

Conversation

vikaskr22 commented Aug 30, 2023

Description of PR

How was this patch tested?

For code changes:

vikaskr22 commented Aug 30, 2023

hadoop-yetus commented Aug 30, 2023

Hexiaoqiao left a comment

Choose a reason for hiding this comment

Hexiaoqiao Sep 3, 2023

Choose a reason for hiding this comment

vikaskr22 Sep 8, 2023

Choose a reason for hiding this comment

Hexiaoqiao Sep 3, 2023

Choose a reason for hiding this comment

vikaskr22 Sep 8, 2023

Choose a reason for hiding this comment

hadoop-yetus commented Sep 5, 2023

hadoop-yetus commented Sep 8, 2023

hadoop-yetus commented Sep 8, 2023

hadoop-yetus commented Sep 12, 2023

vikaskr22 commented Sep 13, 2023

Hexiaoqiao commented Sep 13, 2023

hadoop-yetus commented Sep 13, 2023

vikaskr22 commented Sep 14, 2023

Hexiaoqiao left a comment

Choose a reason for hiding this comment

Hexiaoqiao commented Sep 15, 2023