HDFS-15690. Add lz4-java as test dependency #2481

viirya · 2020-11-21T18:43:45Z

See https://issues.apache.org/jira/browse/HDFS-15690 for details.

TestFSImage.testNativeCompression failed with java.lang.NoClassDefFoundError: net/jpountz/lz4/LZ4Factory:

https://ci-hadoop.apache.org/job/PreCommit-HDFS-Build/305/testReport/junit/org.apache.hadoop.hdfs.server.namenode/TestFSImage/testNativeCompression/

We need to have lz4-java as test dependency.

viirya · 2020-11-21T18:43:53Z

cc @sunchao

sunchao · 2020-11-21T18:47:01Z

thanks for pinging @viirya . Can you change the JIRA project to HDFS?

viirya · 2020-11-21T18:52:13Z

thanks for pinging @viirya . Can you change the JIRA project to HDFS?

Oh, can I change that? Seems no such field to change. Or create new JIRA under HDFS?

viirya · 2020-11-21T18:54:03Z

@sunchao Okay. I moved this JIRA to HDFS. Thanks.

sunchao · 2020-11-21T18:59:03Z

Cool - I think you'll also need to change the PR title as well, and perhaps re-trigger the CI so that HDFS tests will be run.

viirya · 2020-11-21T19:08:42Z

Okay, don't know the JIRA id was changed too.

hadoop-yetus · 2020-11-21T21:43:41Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 31s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 48s		trunk passed
+1 💚	compile	1m 18s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	compile	1m 11s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	mvnsite	1m 19s		trunk passed
+1 💚	shadedclient	53m 8s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 54s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	1m 28s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 10s		the patch passed
+1 💚	compile	1m 10s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javac	1m 10s		the patch passed
+1 💚	compile	1m 9s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	javac	1m 9s		the patch passed
+1 💚	mvnsite	1m 13s		the patch passed
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	shadedclient	14m 50s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 49s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	1m 22s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
			_ Other Tests _
-1 ❌	unit	96m 18s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 46s		The patch does not generate ASF License warnings.
		178m 47s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestRollingUpgrade

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2481/1/artifact/out/Dockerfile
GITHUB PR	#2481
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml
uname	Linux f025f950cfb2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `740399a`
Default Java	Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2481/1/testReport/
Max. process+thread count	5147 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2481/1/console
versions	git=2.17.1 maven=3.6.0
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

viirya · 2020-11-21T22:21:18Z

This looks unrelated...

[ERROR] Tests run: 15, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 110.802 s <<< FAILURE! - in org.apache.hadoop.hdfs.TestRollingUpgrade
[ERROR] testRollback(org.apache.hadoop.hdfs.TestRollingUpgrade)  Time elapsed: 68.247 s  <<< ERROR!
java.net.BindException: Problem binding to [localhost:42009] java.net.BindException: Address already in use; For more details see:  http://wiki.apache.org/hadoop/BindException
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:908)
	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:809)
	at org.apache.hadoop.ipc.Server.bind(Server.java:640)
	at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:1225)
	at org.apache.hadoop.ipc.Server.<init>(Server.java:3118)
	at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:1055)
	at org.apache.hadoop.ipc.ProtobufRpcEngine2$Server.<init>(ProtobufRpcEngine2.java:435)
	at org.apache.hadoop.ipc.ProtobufRpcEngine2.getServer(ProtobufRpcEngine2.java:350)
	at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:853)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.initIpcServer(DataNode.java:1031)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:1452)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:513)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2868)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2774)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2818)
	at org.apache.hadoop.hdfs.MiniDFSCluster.restartDataNode(MiniDFSCluster.java:2516)
	at org.apache.hadoop.hdfs.TestRollingUpgrade.rollbackRollingUpgrade(TestRollingUpgrade.java:386)

sunchao

+1. testNativeCompression passed with this PR.

hadoop-yetus · 2020-11-21T22:40:54Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 34s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	34m 7s		trunk passed
+1 💚	compile	1m 19s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	compile	1m 15s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	mvnsite	1m 20s		trunk passed
+1 💚	shadedclient	53m 21s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 54s		trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	1m 27s		trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
			_ Patch Compile Tests _
+1 💚	mvninstall	1m 14s		the patch passed
+1 💚	compile	1m 12s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javac	1m 12s		the patch passed
+1 💚	compile	1m 6s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚	javac	1m 6s		the patch passed
+1 💚	mvnsite	1m 11s		the patch passed
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	shadedclient	14m 51s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 49s		the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚	javadoc	1m 22s		the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
			_ Other Tests _
-1 ❌	unit	96m 23s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 44s		The patch does not generate ASF License warnings.
		178m 55s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
	hadoop.hdfs.TestDecommissionWithBackoffMonitor
	hadoop.hdfs.server.namenode.ha.TestPipelinesFailover

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2481/2/artifact/out/Dockerfile
GITHUB PR	#2481
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml
uname	Linux 87dd067a2ce8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `740399a`
Default Java	Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2481/2/testReport/
Max. process+thread count	4130 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2481/2/console
versions	git=2.17.1 maven=3.6.0
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

sunchao · 2020-11-22T01:51:10Z

Merged to trunk. Thanks @viirya !

viirya · 2020-11-22T02:20:28Z

Thanks @sunchao

Add lz4-java as test dependency.

7993de6

viirya changed the title ~~HADOOP-17391. Add lz4-java as test dependency~~ HDFS-15690. Add lz4-java as test dependency Nov 21, 2020

Trigger Build

0443c54

sunchao approved these changes Nov 21, 2020

View reviewed changes

sunchao merged commit d730294 into apache:trunk Nov 22, 2020

sunchao pushed a commit that referenced this pull request Dec 29, 2020

HDFS-15690. Add lz4-java as test dependency (#2481)

0605942

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-15690. Add lz4-java as test dependency #2481

HDFS-15690. Add lz4-java as test dependency #2481

viirya commented Nov 21, 2020 •

edited

Loading

viirya commented Nov 21, 2020

sunchao commented Nov 21, 2020

viirya commented Nov 21, 2020 •

edited

Loading

viirya commented Nov 21, 2020

sunchao commented Nov 21, 2020

viirya commented Nov 21, 2020

hadoop-yetus commented Nov 21, 2020

viirya commented Nov 21, 2020

sunchao left a comment

hadoop-yetus commented Nov 21, 2020

sunchao commented Nov 22, 2020

viirya commented Nov 22, 2020

HDFS-15690. Add lz4-java as test dependency #2481

HDFS-15690. Add lz4-java as test dependency #2481

Conversation

viirya commented Nov 21, 2020 • edited Loading

viirya commented Nov 21, 2020

sunchao commented Nov 21, 2020

viirya commented Nov 21, 2020 • edited Loading

viirya commented Nov 21, 2020

sunchao commented Nov 21, 2020

viirya commented Nov 21, 2020

hadoop-yetus commented Nov 21, 2020

viirya commented Nov 21, 2020

sunchao left a comment

Choose a reason for hiding this comment

hadoop-yetus commented Nov 21, 2020

sunchao commented Nov 22, 2020

viirya commented Nov 22, 2020

viirya commented Nov 21, 2020 •

edited

Loading

viirya commented Nov 21, 2020 •

edited

Loading