HDFS-17333. DFSClient supports lazy resolution from hostname to IP. #6430

KeeProMise · 2024-01-10T03:07:39Z

Description of PR

see also: HDFS-17333
Currently, When the dfsclient initializes, it will resolve all hosts of all namservices:
at DFSUtilClient#getAddresses(conf, null, addressKey)
at AbstractNNFailoverProxyProvider#getProxyAddresses(URI uri, String addressKey)
If the current environment where the dfsClient is located causes resolution of host->ip to be very slow, the existing logic will undoubtedly take a long time when there are too many nameservices.

Now, each dfsclient only needs the IPs of all namenodes of a certain nameservice at most. A better situation is that if the namenode selected by dfsclient for the first time can provide the required services normally, then the client only needs to know the IP of this namenode. Therefore, it is not necessary to resolve all namenodes of all nameservices in the configuration file, when the dfsclient initializes.

This patch supports lazy resolution of host->ip, which will only be resolved when the host needs to be accessed.

How was this patch tested?

new ut TestConfiguredFailoverProxyProvider#testLazyResolved() and TestDFSUtil#testLazyResolved()

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2024-01-10T11:24:18Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 38s		Maven dependency ordering for branch
+1 💚	mvninstall	35m 15s		trunk passed
+1 💚	compile	18m 11s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	16m 56s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	4m 39s		trunk passed
+1 💚	mvnsite	4m 23s		trunk passed
+1 💚	javadoc	3m 10s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	3m 26s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	9m 12s		trunk passed
+1 💚	shadedclient	38m 35s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 31s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 8s		the patch passed
+1 💚	compile	17m 46s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	17m 46s		the patch passed
+1 💚	compile	15m 10s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	15m 10s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	4m 1s		the patch passed
+1 💚	mvnsite	4m 7s		the patch passed
+1 💚	javadoc	3m 8s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	3m 19s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	9m 44s		the patch passed
+1 💚	shadedclient	40m 47s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 33s		hadoop-common in the patch passed.
+1 💚	unit	2m 43s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	219m 43s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 4s		The patch does not generate ASF License warnings.
		495m 21s

Reason	Tests
Failed junit tests	hadoop.tools.TestHdfsConfigFields

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/1/artifact/out/Dockerfile
GITHUB PR	#6430
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 9552b09511ff 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `3343470`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/1/testReport/
Max. process+thread count	4156 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-01-10T21:35:09Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 33s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 6s		Maven dependency ordering for branch
+1 💚	mvninstall	34m 23s		trunk passed
+1 💚	compile	17m 44s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	16m 29s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	4m 32s		trunk passed
+1 💚	mvnsite	4m 24s		trunk passed
+1 💚	javadoc	3m 18s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	3m 31s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	8m 53s		trunk passed
+1 💚	shadedclient	38m 1s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 30s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 58s		the patch passed
+1 💚	compile	16m 48s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	16m 48s		the patch passed
+1 💚	compile	15m 27s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	15m 27s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	4m 28s		the patch passed
+1 💚	mvnsite	4m 24s		the patch passed
+1 💚	javadoc	3m 12s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	3m 30s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	10m 9s		the patch passed
+1 💚	shadedclient	39m 22s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 25s		hadoop-common in the patch passed.
+1 💚	unit	2m 44s		hadoop-hdfs-client in the patch passed.
+1 💚	unit	215m 4s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 7s		The patch does not generate ASF License warnings.
		486m 33s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/2/artifact/out/Dockerfile
GITHUB PR	#6430
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 6f24688a3742 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `95d9fbe`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/2/testReport/
Max. process+thread count	3940 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-01-10T22:20:53Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 23s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 18s		trunk passed
+1 💚	compile	9m 16s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	8m 37s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	2m 15s		trunk passed
+1 💚	mvnsite	2m 38s		trunk passed
+1 💚	javadoc	2m 1s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	2m 22s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	4m 43s		trunk passed
+1 💚	shadedclient	20m 45s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 35s		the patch passed
+1 💚	compile	8m 1s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	8m 1s		the patch passed
+1 💚	compile	7m 29s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	7m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 57s		the patch passed
+1 💚	mvnsite	2m 17s		the patch passed
+1 💚	javadoc	1m 46s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	2m 14s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	4m 56s		the patch passed
+1 💚	shadedclient	20m 47s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	16m 34s		hadoop-common in the patch passed.
+1 💚	unit	2m 0s		hadoop-hdfs-client in the patch passed.
+1 💚	unit	184m 36s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		345m 43s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/3/artifact/out/Dockerfile
GITHUB PR	#6430
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux d05670ad082e 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `0bd0e84`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/3/testReport/
Max. process+thread count	4653 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

KeeProMise · 2024-01-14T16:52:25Z

@Hexiaoqiao hi, could you please help to review, thanks a lot!

KeeProMise · 2024-01-20T12:49:52Z

Can anyone help review it? Thanks.

KeeProMise · 2024-01-30T11:56:03Z

Hi @tasanuma @Hexiaoqiao @zhangshuyan0,
Please kindly review this PR as well if you have bandwidth, Thanks.

KeeProMise · 2024-02-27T13:07:20Z

Hi @tasanuma @Hexiaoqiao @zhangshuyan0, @slfan1989
Please kindly review this PR as well if you have bandwidth, Thanks.

tasanuma · 2024-03-01T14:12:42Z

@KeeProMise Sorry for being very late. This is a nice feature and the PR looks pretty good to me.

It's been a long time since Jenkins last ran, so I've triggered it to run again. If there are no issues, I'll merge this PR.
https://ci-hadoop.apache.org/blue/organizations/jenkins/hadoop-multibranch/detail/PR-6430/4/pipeline

hadoop-yetus · 2024-03-01T22:23:10Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	35m 33s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 34s		Maven dependency ordering for branch
+1 💚	mvninstall	34m 23s		trunk passed
+1 💚	compile	16m 33s		trunk passed
+1 💚	checkstyle	4m 44s		trunk passed
+1 💚	mvnsite	4m 38s		trunk passed
+1 💚	javadoc	3m 46s		trunk passed
-1 ❌	spotbugs	2m 50s	/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html	hadoop-common-project/hadoop-common in trunk has 1 extant spotbugs warnings.
-1 ❌	spotbugs	2m 50s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html	hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 💚	shadedclient	37m 4s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 7s		the patch passed
+1 💚	compile	16m 22s		the patch passed
+1 💚	javac	16m 22s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	4m 34s		the patch passed
+1 💚	mvnsite	4m 21s		the patch passed
+1 💚	xmllint	0m 0s		No new issues.
+1 💚	javadoc	3m 31s		the patch passed
+1 💚	spotbugs	9m 51s		the patch passed
+1 💚	shadedclient	38m 27s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	20m 3s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch passed.
+1 💚	unit	2m 42s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	228m 51s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 9s		The patch does not generate ASF License warnings.
		495m 3s

Reason	Tests
Failed junit tests	hadoop.crypto.TestCryptoStreamsWithOpensslSm4CtrCryptoCodec
	hadoop.service.launcher.TestServiceInterruptHandling
	hadoop.crypto.TestCryptoCodec
	hadoop.hdfs.server.datanode.TestLargeBlockReport
	hadoop.hdfs.protocol.TestBlockListAsLongs

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/4/artifact/out/Dockerfile
GITHUB PR	#6430
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 0d3228f82605 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `0bd0e84`
Default Java	Red Hat, Inc.-1.8.0_402-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/4/testReport/
Max. process+thread count	3694 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/4/console
versions	git=2.9.5 maven=3.6.3 spotbugs=4.2.2 xmllint=20901
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-03-02T08:04:17Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 31s		Maven dependency ordering for branch
+1 💚	mvninstall	31m 40s		trunk passed
+1 💚	compile	17m 1s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	compile	15m 37s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	checkstyle	4m 12s		trunk passed
+1 💚	mvnsite	4m 18s		trunk passed
+1 💚	javadoc	3m 19s		trunk passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	3m 31s		trunk passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
-1 ❌	spotbugs	2m 30s	/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html	hadoop-common-project/hadoop-common in trunk has 1 extant spotbugs warnings.
-1 ❌	spotbugs	2m 38s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html	hadoop-hdfs-project/hadoop-hdfs-client in trunk has 1 extant spotbugs warnings.
+1 💚	shadedclient	36m 24s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 31s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 53s		the patch passed
+1 💚	compile	17m 34s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javac	17m 34s		the patch passed
+1 💚	compile	16m 39s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	javac	16m 39s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	4m 37s		the patch passed
+1 💚	mvnsite	4m 11s		the patch passed
+1 💚	javadoc	3m 5s		the patch passed with JDK Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	3m 29s		the patch passed with JDK Private Build-1.8.0_392-8u392-ga-1~20.04-b08
+1 💚	spotbugs	10m 2s		the patch passed
+1 💚	shadedclient	40m 18s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 50s		hadoop-common in the patch passed.
+1 💚	unit	2m 40s		hadoop-hdfs-client in the patch passed.
-1 ❌	unit	229m 39s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 5s		The patch does not generate ASF License warnings.
		498m 2s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.datanode.TestLargeBlockReport
	hadoop.hdfs.protocol.TestBlockListAsLongs
	hadoop.hdfs.tools.TestDFSAdmin

Subsystem	Report/Notes
Docker	ClientAPI=1.44 ServerAPI=1.44 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/5/artifact/out/Dockerfile
GITHUB PR	#6430
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 9a32c982b58d 5.15.0-94-generic #104-Ubuntu SMP Tue Jan 9 15:25:40 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `b614849`
Default Java	Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.21+9-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_392-8u392-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/5/testReport/
Max. process+thread count	4536 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6430/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

KeeProMise · 2024-03-02T08:12:46Z

@KeeProMise Sorry for being very late. This is a nice feature and the PR looks pretty good to me.

It's been a long time since Jenkins last ran, so I've triggered it to run again. If there are no issues, I'll merge this PR. https://ci-hadoop.apache.org/blue/organizations/jenkins/hadoop-multibranch/detail/PR-6430/4/pipeline

@tasanuma Hi, thank you for reviewing this PR. I don't think the issues in unit testing and spotbugs are related to my code. Could you please help me review it again.

tasanuma

LGTM.

tasanuma · 2024-03-02T12:36:19Z

I made a slight change to the title to avoid including symbols, then merged it.
Thanks for your contribution, @KeeProMise!

KeeProMise · 2024-03-02T12:40:26Z

I made a slight change to the title to avoid including symbols, then merged it. Thanks for your contribution, @KeeProMise!

@tasanuma Thanks for your review and merge it!

…6430) Signed-off-by: Takanobu Asanuma <[email protected]> (cherry picked from commit a6aa292)

KeeProMise added 2 commits January 9, 2024 14:58

lazy resolve host -> ip

c97a88f

add ut

3343470

github-actions bot added HDFS trunk Common labels Jan 10, 2024

KeeProMise marked this pull request as draft January 10, 2024 03:09

KeeProMise marked this pull request as ready for review January 10, 2024 05:58

KeeProMise added 2 commits January 10, 2024 21:26

add dfs.client.failover.lazy.resolved to hdfs-default.xml

95d9fbe

add ut

0bd0e84

KeeProMise changed the title ~~HDFS-17333. Support lazy resolve host->ip.~~ HDFS-17333. DFSClient support lazy resolve host->ip. Jan 10, 2024

Merge remote-tracking branch 'upstream/trunk' into lazy_resolved

b614849

tasanuma approved these changes Mar 2, 2024

View reviewed changes

tasanuma changed the title ~~HDFS-17333. DFSClient support lazy resolve host->ip.~~ HDFS-17333. DFSClient supports lazy resolution from hostname to IP. Mar 2, 2024

tasanuma merged commit a6aa292 into apache:trunk Mar 2, 2024
1 of 5 checks passed

tasanuma pushed a commit that referenced this pull request Mar 2, 2024

HDFS-17333. DFSClient supports lazy resolution from hostname to IP. (#…

552452b

…6430) Signed-off-by: Takanobu Asanuma <[email protected]> (cherry picked from commit a6aa292)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-17333. DFSClient supports lazy resolution from hostname to IP. #6430

HDFS-17333. DFSClient supports lazy resolution from hostname to IP. #6430

KeeProMise commented Jan 10, 2024 •

edited

Loading

hadoop-yetus commented Jan 10, 2024

hadoop-yetus commented Jan 10, 2024

hadoop-yetus commented Jan 10, 2024

KeeProMise commented Jan 14, 2024

KeeProMise commented Jan 20, 2024

KeeProMise commented Jan 30, 2024

KeeProMise commented Feb 27, 2024

tasanuma commented Mar 1, 2024

hadoop-yetus commented Mar 1, 2024

hadoop-yetus commented Mar 2, 2024

KeeProMise commented Mar 2, 2024 •

edited

Loading

tasanuma left a comment

tasanuma commented Mar 2, 2024

KeeProMise commented Mar 2, 2024

HDFS-17333. DFSClient supports lazy resolution from hostname to IP. #6430

HDFS-17333. DFSClient supports lazy resolution from hostname to IP. #6430

Conversation

KeeProMise commented Jan 10, 2024 • edited Loading

Description of PR

How was this patch tested?

For code changes:

hadoop-yetus commented Jan 10, 2024

hadoop-yetus commented Jan 10, 2024

hadoop-yetus commented Jan 10, 2024

KeeProMise commented Jan 14, 2024

KeeProMise commented Jan 20, 2024

KeeProMise commented Jan 30, 2024

KeeProMise commented Feb 27, 2024

tasanuma commented Mar 1, 2024

hadoop-yetus commented Mar 1, 2024

hadoop-yetus commented Mar 2, 2024

KeeProMise commented Mar 2, 2024 • edited Loading

tasanuma left a comment

Choose a reason for hiding this comment

tasanuma commented Mar 2, 2024

KeeProMise commented Mar 2, 2024

KeeProMise commented Jan 10, 2024 •

edited

Loading

KeeProMise commented Mar 2, 2024 •

edited

Loading