HDFS-16400. Reconfig DataXceiver parameters for datanode #3843

tomscut · 2021-12-31T01:15:37Z

To avoid frequent rolling restarts of the DN, we should make DataXceiver parameters reconfigurable.

hadoop-yetus · 2021-12-31T09:38:35Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 58s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 32s		trunk passed
+1 💚	compile	1m 27s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	1m 21s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 28s		trunk passed
+1 💚	javadoc	1m 2s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 31s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 14s		trunk passed
+1 💚	shadedclient	22m 35s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 14s		the patch passed
+1 💚	compile	1m 17s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	1m 17s		the patch passed
+1 💚	compile	1m 12s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	1m 12s		the patch passed
+1 💚	blanks	0m 1s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 50s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 80 unchanged - 2 fixed = 81 total (was 82)
+1 💚	mvnsite	1m 16s		the patch passed
+1 💚	javadoc	0m 49s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 26s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	3m 14s		the patch passed
+1 💚	shadedclient	22m 16s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	402m 18s		hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 46s		The patch does not generate ASF License warnings.
		501m 41s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3843/1/artifact/out/Dockerfile
GITHUB PR	#3843
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 429e4d1c44d9 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `af3ad91`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3843/1/testReport/
Max. process+thread count	2760 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3843/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tomscut · 2022-01-11T01:49:23Z

Hi @tasanuma @Hexiaoqiao @ayushtkn @ferhui , could you please take a look at this. Thanks.

tasanuma · 2022-01-11T10:43:00Z

...-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java

+    String result;
+    try {
+      LOG.info("Reconfiguring {} to {}", property, newVal);
+      if (property.equals(DFS_DATANODE_MAX_RECEIVER_THREADS_KEY)) {


The if-statement seems redundant since it is already checked by case DFS_DATANODE_MAX_RECEIVER_THREADS_KEY .

Thanks @tasanuma for your review. I agree with your suggestion. There is only one parameter at present and it has been judged in advance. I will fix it.

virajjasani · 2022-01-11T12:04:09Z

...ject/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataXceiverServer.java

@@ -68,8 +68,7 @@
   * Enforcing the limit is required in order to avoid data-node
   * running out of memory.
   */
-  int maxXceiverCount =
-    DFSConfigKeys.DFS_DATANODE_MAX_RECEIVER_THREADS_DEFAULT;
+  volatile int maxXceiverCount;


Looks like we need volatile for sure right? There might be slightest chance for DataXceiverServer thread to throw limit exceeded error if this is not volatile?

Thanks @virajjasani for your review. IMO, to ensure consistency across threads, we need to make it volatile.

...p-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeReconfiguration.java

hadoop-yetus · 2022-01-12T10:22:18Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 0s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 58s		trunk passed
+1 💚	compile	1m 28s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	1m 19s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 31s		trunk passed
+1 💚	javadoc	1m 4s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 32s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 17s		trunk passed
+1 💚	shadedclient	22m 43s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 16s		the patch passed
+1 💚	compile	1m 20s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javac	1m 20s		the patch passed
+1 💚	compile	1m 11s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	1m 11s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 50s	/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 80 unchanged - 2 fixed = 81 total (was 82)
+1 💚	mvnsite	1m 22s		the patch passed
+1 💚	javadoc	0m 53s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 23s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	3m 16s		the patch passed
+1 💚	shadedclient	22m 18s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	412m 1s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 46s		The patch does not generate ASF License warnings.
		512m 14s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3843/2/artifact/out/Dockerfile
GITHUB PR	#3843
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 38df6a28e1cc 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `3e738af`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3843/2/testReport/
Max. process+thread count	2614 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3843/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tasanuma

LGTM.

tomscut · 2022-01-13T13:17:37Z

Hi @virajjasani , could you please review this again. Thank you.

virajjasani · 2022-01-13T15:57:24Z

One checkstyle warning is pending, else looks good.

tomscut · 2022-01-13T16:11:15Z

One checkstyle warning is pending, else looks good.

Thanks @virajjasani for your comments. This parameter maxXceiverCount is protected before, I think we don't need to change it. What do you think of this?

If we change maxXceiverCount to private, we might have to change all places used it before.

virajjasani · 2022-01-14T05:39:30Z

If we change maxXceiverCount to private, we might have to change all places used it before.

Yeah since it is not newly introduced and we are updating it to volatile, it's fine to leave it as is rather than making it private. I believe we can ignore this checkstyle warning in this case.
+1 (non-binding) from my side. Thanks @tomscut

tasanuma · 2022-01-14T05:47:17Z

Thanks for the review. I'm merging it.

tasanuma · 2022-01-14T05:48:33Z

Thanks for your contribution, @tomscut. Thanks for your review, @virajjasani.

tomscut · 2022-01-14T11:44:45Z

Thanks @tasanuma and @virajjasani for your review and merge.

Reviewed-by: Viraj Jasani <[email protected]> Signed-off-by: Takanobu Asanuma <[email protected]> (cherry picked from commit f02374d)

tasanuma · 2022-01-19T09:44:19Z

Cherry-picked it into branch-3.3.

Reviewed-by: Viraj Jasani <[email protected]> Signed-off-by: Takanobu Asanuma <[email protected]> (cherry picked from commit f02374d)

Reviewed-by: Viraj Jasani <[email protected]> Signed-off-by: Takanobu Asanuma <[email protected]>

HDFS-16400. Reconfig DataXceiver parameters for datanode

af3ad91

tasanuma reviewed Jan 11, 2022

View reviewed changes

virajjasani reviewed Jan 11, 2022

View reviewed changes

remove redundant if-statement, update assert

3e738af

tasanuma approved these changes Jan 12, 2022

View reviewed changes

tasanuma merged commit f02374d into apache:trunk Jan 14, 2022

tasanuma pushed a commit that referenced this pull request Jan 19, 2022

HDFS-16400. Reconfig DataXceiver parameters for datanode (#3843)

11fe527

Reviewed-by: Viraj Jasani <[email protected]> Signed-off-by: Takanobu Asanuma <[email protected]> (cherry picked from commit f02374d)

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022

HDFS-16400. Reconfig DataXceiver parameters for datanode (apache#3843)

6f270aa

Reviewed-by: Viraj Jasani <[email protected]> Signed-off-by: Takanobu Asanuma <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-16400. Reconfig DataXceiver parameters for datanode #3843

HDFS-16400. Reconfig DataXceiver parameters for datanode #3843

tomscut commented Dec 31, 2021 •

edited

Loading

hadoop-yetus commented Dec 31, 2021

tomscut commented Jan 11, 2022

tasanuma Jan 11, 2022

tomscut Jan 12, 2022 •

edited

Loading

virajjasani Jan 11, 2022

tomscut Jan 12, 2022

hadoop-yetus commented Jan 12, 2022

tasanuma left a comment

tomscut commented Jan 13, 2022

virajjasani commented Jan 13, 2022

tomscut commented Jan 13, 2022 •

edited

Loading

virajjasani commented Jan 14, 2022

tasanuma commented Jan 14, 2022

tasanuma commented Jan 14, 2022

tomscut commented Jan 14, 2022

tasanuma commented Jan 19, 2022

HDFS-16400. Reconfig DataXceiver parameters for datanode #3843

HDFS-16400. Reconfig DataXceiver parameters for datanode #3843

Conversation

tomscut commented Dec 31, 2021 • edited Loading

hadoop-yetus commented Dec 31, 2021

tomscut commented Jan 11, 2022

tasanuma Jan 11, 2022

Choose a reason for hiding this comment

tomscut Jan 12, 2022 • edited Loading

Choose a reason for hiding this comment

virajjasani Jan 11, 2022

Choose a reason for hiding this comment

tomscut Jan 12, 2022

Choose a reason for hiding this comment

hadoop-yetus commented Jan 12, 2022

tasanuma left a comment

Choose a reason for hiding this comment

tomscut commented Jan 13, 2022

virajjasani commented Jan 13, 2022

tomscut commented Jan 13, 2022 • edited Loading

virajjasani commented Jan 14, 2022

tasanuma commented Jan 14, 2022

tasanuma commented Jan 14, 2022

tomscut commented Jan 14, 2022

tasanuma commented Jan 19, 2022

tomscut commented Dec 31, 2021 •

edited

Loading

tomscut Jan 12, 2022 •

edited

Loading

tomscut commented Jan 13, 2022 •

edited

Loading