HDFS-16429. Add DataSetLockManager to manage fine-grain locks for FsDataSetImpl #3900

MingXiangLi · 2022-01-18T07:47:23Z

this pr is sub task of the #3889

hadoop-yetus · 2022-01-18T15:22:09Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 39s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 59s		Maven dependency ordering for branch
+1 💚	mvninstall	22m 24s		trunk passed
+1 💚	compile	22m 22s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	19m 26s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	3m 36s		trunk passed
+1 💚	mvnsite	3m 19s		trunk passed
+1 💚	javadoc	2m 28s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	3m 37s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	5m 49s		trunk passed
+1 💚	shadedclient	23m 3s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 12s		the patch passed
+1 💚	compile	21m 35s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
-1 ❌	javac	21m 35s	/results-compile-javac-root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt	root-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 generated 1 new + 1811 unchanged - 0 fixed = 1812 total (was 1811)
+1 💚	compile	19m 11s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 ❌	javac	19m 11s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt	root-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 generated 1 new + 1687 unchanged - 0 fixed = 1688 total (was 1687)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 37s		the patch passed
+1 💚	mvnsite	3m 18s		the patch passed
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	javadoc	2m 25s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	3m 33s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	6m 11s		the patch passed
+1 💚	shadedclient	23m 14s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	17m 34s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
-1 ❌	unit	229m 47s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 11s		The patch does not generate ASF License warnings.
		453m 21s

Reason	Tests
Failed junit tests	hadoop.ipc.TestIPC
	hadoop.hdfs.TestRollingUpgrade

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/1/artifact/out/Dockerfile
GITHUB PR	#3900
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux f289afe1c7f7 4.15.0-161-generic #169-Ubuntu SMP Fri Oct 15 13:41:54 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 5709090ce310fd2bd0c7d38a09e3fc0edad1364b
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/1/testReport/
Max. process+thread count	3158 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-01-19T15:56:28Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 41s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 5s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 42s		trunk passed
+1 💚	compile	22m 17s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	19m 29s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	3m 41s		trunk passed
+1 💚	mvnsite	3m 21s		trunk passed
+1 💚	javadoc	2m 27s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	3m 29s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	5m 48s		trunk passed
+1 💚	shadedclient	23m 9s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 27s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 14s		the patch passed
+1 💚	compile	21m 33s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javac	21m 33s		the patch passed
+1 💚	compile	19m 21s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	19m 21s		the patch passed
+1 💚	blanks	0m 1s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 31s	/results-checkstyle-root.txt	root: The patch generated 1 new + 201 unchanged - 0 fixed = 202 total (was 201)
+1 💚	mvnsite	3m 21s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	2m 24s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	3m 28s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	6m 12s		the patch passed
+1 💚	shadedclient	23m 15s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	17m 39s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
+1 💚	unit	231m 58s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 5s		The patch does not generate ASF License warnings.
		454m 1s

Reason	Tests
Failed junit tests	hadoop.ipc.TestIPC

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/2/artifact/out/Dockerfile
GITHUB PR	#3900
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux 9b1217eef5cf 4.15.0-161-generic #169-Ubuntu SMP Fri Oct 15 13:41:54 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / dc950618f3be2e7eb7c2dfe8794f731945e2e41f
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/2/testReport/
Max. process+thread count	3285 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-01-20T11:58:02Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 44s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 49s		trunk passed
+1 💚	compile	22m 14s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	19m 22s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	3m 40s		trunk passed
+1 💚	mvnsite	3m 22s		trunk passed
+1 💚	javadoc	2m 27s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	3m 31s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	5m 54s		trunk passed
+1 💚	shadedclient	23m 8s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 15s		the patch passed
+1 💚	compile	21m 33s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javac	21m 33s		the patch passed
+1 💚	compile	19m 37s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	19m 37s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 34s		the patch passed
+1 💚	mvnsite	3m 17s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	2m 25s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	3m 32s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	6m 15s		the patch passed
+1 💚	shadedclient	23m 13s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 26s		hadoop-common in the patch passed.
+1 💚	unit	226m 7s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 12s		The patch does not generate ASF License warnings.
		448m 55s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/3/artifact/out/Dockerfile
GITHUB PR	#3900
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux 39300c093785 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 9372f1cfa98274ecbad3ff1417d58a3c17261050
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/3/testReport/
Max. process+thread count	3121 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-01-21T17:55:58Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 38s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 51s		Maven dependency ordering for branch
+1 💚	mvninstall	22m 56s		trunk passed
+1 💚	compile	23m 4s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	20m 9s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	3m 42s		trunk passed
+1 💚	mvnsite	3m 26s		trunk passed
+1 💚	javadoc	2m 16s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	3m 21s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	5m 46s		trunk passed
+1 💚	shadedclient	22m 54s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 13s		the patch passed
+1 💚	compile	22m 10s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javac	22m 10s		the patch passed
+1 💚	compile	19m 23s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	19m 23s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 31s	/results-checkstyle-root.txt	root: The patch generated 1 new + 201 unchanged - 0 fixed = 202 total (was 201)
+1 💚	mvnsite	3m 21s		the patch passed
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	javadoc	2m 24s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	3m 31s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	6m 12s		the patch passed
+1 💚	shadedclient	23m 2s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	17m 30s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
-1 ❌	unit	226m 50s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 11s		The patch does not generate ASF License warnings.
		452m 8s

Reason	Tests
Failed junit tests	hadoop.ipc.TestIPC
	hadoop.hdfs.TestRollingUpgrade

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/4/artifact/out/Dockerfile
GITHUB PR	#3900
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux a22a64d3021a 4.15.0-161-generic #169-Ubuntu SMP Fri Oct 15 13:41:54 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 6b899099864d7caf35a06dfaba6601ffab44c31f
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/4/testReport/
Max. process+thread count	3364 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Hexiaoqiao

Thanks @MingXiangLi for your works here.
a. Leave some nit comments inline.
b. the failed unit test seems not related to this changes.
c. look forward to the new progress.

...ect/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/AutoCloseDataSetLock.java

hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java

...ect/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/AutoCloseDataSetLock.java

...ject/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/common/DataNodeLockManager.java

...ect/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataSetLockManager.java

hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml

…nd volume level lock

MingXiangLi · 2022-01-24T04:35:09Z

Thanks for the advice, I have repush the code.

hadoop-yetus · 2022-01-24T12:07:36Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 38s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 59s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 40s		trunk passed
+1 💚	compile	22m 10s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	19m 22s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	3m 38s		trunk passed
+1 💚	mvnsite	3m 13s		trunk passed
+1 💚	javadoc	2m 13s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	3m 22s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	5m 53s		trunk passed
+1 💚	shadedclient	22m 56s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 27s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 21s		the patch passed
+1 💚	compile	21m 27s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javac	21m 27s		the patch passed
+1 💚	compile	19m 23s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	19m 23s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	3m 34s		the patch passed
+1 💚	mvnsite	3m 18s		the patch passed
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	javadoc	2m 25s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	3m 32s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	6m 15s		the patch passed
+1 💚	shadedclient	23m 20s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 35s		hadoop-common in the patch passed.
+1 💚	unit	230m 46s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 11s		The patch does not generate ASF License warnings.
		452m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/5/artifact/out/Dockerfile
GITHUB PR	#3900
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell xml
uname	Linux dfc47343b9e0 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e28b21d`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/5/testReport/
Max. process+thread count	3606 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3900/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

Hexiaoqiao

LGTM. +1 from my side. Thanks @MingXiangLi for your works.
Will commit if no other furthermore comments while two days later.

…ataSetImpl. (apache#3900). Contributed by limingxiang. Signed-off-by: He Xiaoqiao <[email protected]>

MingXiangLi mentioned this pull request Jan 18, 2022

HDFS-15180.Split FsDatasetImpl lock to block Pool level #3889

Closed

MingXiangLi force-pushed the trunk-add-lock-manager branch from 5709090 to dc95061 Compare January 19, 2022 08:21

MingXiangLi force-pushed the trunk-add-lock-manager branch from dc95061 to 9372f1c Compare January 20, 2022 04:27

MingXiangLi force-pushed the trunk-add-lock-manager branch from 9372f1c to 6b89909 Compare January 21, 2022 10:22

Hexiaoqiao reviewed Jan 23, 2022

View reviewed changes

Hexiaoqiao added HDFS improvement labels Jan 23, 2022

HDFS-16429.Add DataSetLockManager to maintain block pool level lock a…

e28b21d

…nd volume level lock

MingXiangLi force-pushed the trunk-add-lock-manager branch from 6b89909 to e28b21d Compare January 24, 2022 04:33

Hexiaoqiao approved these changes Jan 25, 2022

View reviewed changes

Hexiaoqiao changed the title ~~HDFS-16429.Add DataSetLockManager to maintain block pool level lock a…~~ HDFS-16429. Add DataSetLockManager to manage fine-grain locks for FsDataSetImpl Jan 27, 2022

Hexiaoqiao merged commit e17c96a into apache:trunk Jan 27, 2022

yuanboliu pushed a commit to yuanboliu/hadoop that referenced this pull request Feb 21, 2022

HDFS-16429. Add DataSetLockManager to manage fine-grain locks for FsD…

200cb42

…ataSetImpl. (apache#3900). Contributed by limingxiang. Signed-off-by: He Xiaoqiao <[email protected]>

yuanboliu mentioned this pull request Feb 21, 2022

HDFS-16429. Add DataSetLockManager to manage fine-grain locks for FsD… #4005

Open

4 tasks

Hexiaoqiao mentioned this pull request Feb 23, 2022

HDFS-15382. Split one FsDatasetImpl lock to block pool grain locks. #3941

Closed

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022

HDFS-16429. Add DataSetLockManager to manage fine-grain locks for FsD…

bbcf6a8

…ataSetImpl. (apache#3900). Contributed by limingxiang. Signed-off-by: He Xiaoqiao <[email protected]>

LiuGuH pushed a commit to LiuGuH/hadoop that referenced this pull request Mar 26, 2024

HDFS-16429. Add DataSetLockManager to manage fine-grain locks for FsD…

ed742e8

…ataSetImpl. (apache#3900). Contributed by limingxiang. Signed-off-by: He Xiaoqiao <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-16429. Add DataSetLockManager to manage fine-grain locks for FsDataSetImpl #3900

HDFS-16429. Add DataSetLockManager to manage fine-grain locks for FsDataSetImpl #3900

MingXiangLi commented Jan 18, 2022

hadoop-yetus commented Jan 18, 2022

hadoop-yetus commented Jan 19, 2022

hadoop-yetus commented Jan 20, 2022

hadoop-yetus commented Jan 21, 2022

Hexiaoqiao left a comment

MingXiangLi commented Jan 24, 2022

hadoop-yetus commented Jan 24, 2022

Hexiaoqiao left a comment

HDFS-16429. Add DataSetLockManager to manage fine-grain locks for FsDataSetImpl #3900

HDFS-16429. Add DataSetLockManager to manage fine-grain locks for FsDataSetImpl #3900

Conversation

MingXiangLi commented Jan 18, 2022

hadoop-yetus commented Jan 18, 2022

hadoop-yetus commented Jan 19, 2022

hadoop-yetus commented Jan 20, 2022

hadoop-yetus commented Jan 21, 2022

Hexiaoqiao left a comment

Choose a reason for hiding this comment

MingXiangLi commented Jan 24, 2022

hadoop-yetus commented Jan 24, 2022

Hexiaoqiao left a comment

Choose a reason for hiding this comment