HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from the endless CVE patches. #1876

jojochuang · 2020-03-05T00:10:02Z

NOTICE

Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute

hadoop-yetus · 2020-03-05T02:01:01Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	45m 44s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+1 💚	mvninstall	24m 39s	trunk passed
+1 💚	compile	0m 15s	trunk passed
+1 💚	mvnsite	0m 22s	trunk passed
+1 💚	shadedclient	42m 20s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 17s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 12s	the patch passed
+1 💚	compile	0m 10s	the patch passed
+1 💚	javac	0m 10s	the patch passed
+1 💚	mvnsite	0m 12s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 2s	The patch has no ill-formed XML file.
-1 ❌	shadedclient	17m 34s	patch has errors when building and testing our client artifacts.
+1 💚	javadoc	0m 13s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 13s	hadoop-project in the patch passed.
+1 💚	asflicense	0m 28s	The patch does not generate ASF License warnings.
		109m 40s

Subsystem	Report/Notes
Docker	Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1876/1/artifact/out/Dockerfile
GITHUB PR	#1876
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml
uname	Linux 35f475a87775 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `2649f8b`
Default Java	1.8.0_242
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1876/1/testReport/
Max. process+thread count	330 (vs. ulimit of 5500)
modules	C: hadoop-project U: hadoop-project
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1876/1/console
versions	git=2.7.4 maven=3.3.9
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

iwasakims · 2020-03-05T02:36:22Z

mvn test -Dtest=TestDelegationTokenAuthenticationHandlerWithMocks,TestWebDelegationToken reproducibly fails with the error like "java.lang.NoClassDefFoundError: com/fasterxml/jackson/core/exc/InputCoercionException" on my local. I'm looking into the cause.

iwasakims · 2020-03-05T03:04:15Z

looks like jackson2.version should be updated too.

jojochuang · 2020-03-05T03:52:00Z

Thanks. In that case it's much more complex. jackson 2.10.3 has extra dependency and it breaks Hadoop shading. I'll take a further look later.

hadoop-yetus · 2020-03-05T15:45:28Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 21s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+0 🆗	mvndep	0m 21s	Maven dependency ordering for branch
+1 💚	mvninstall	21m 33s	trunk passed
+1 💚	compile	17m 50s	trunk passed
+1 💚	mvnsite	0m 51s	trunk passed
+1 💚	shadedclient	56m 26s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 49s	trunk passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 20s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 49s	the patch passed
+1 💚	compile	17m 13s	the patch passed
+1 💚	javac	17m 13s	the patch passed
+1 💚	mvnsite	0m 52s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 3s	The patch has no ill-formed XML file.
+1 💚	shadedclient	15m 30s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 48s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 23s	hadoop-project in the patch passed.
+1 💚	unit	0m 24s	hadoop-client-runtime in the patch passed.
+1 💚	asflicense	0m 45s	The patch does not generate ASF License warnings.
		100m 33s

Subsystem	Report/Notes
Docker	Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1876/2/artifact/out/Dockerfile
GITHUB PR	#1876
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml
uname	Linux 00d13599f21d 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `004e955`
Default Java	1.8.0_242
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1876/2/testReport/
Max. process+thread count	308 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-client-modules/hadoop-client-runtime U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1876/2/console
versions	git=2.7.4 maven=3.3.9
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

iwasakims · 2020-03-06T01:39:55Z

@jojochuang Thanks for the update. Is reverting the change of hadoop-project/pom.xml in the second commit intentional?

…e endless CVE patches. Change-Id: I452a76bd9f71a0dd2ffb0f32df35c941d3b24aa3

Change-Id: I2915844b8e74f7f3e4fbb62bde8f0db76bcf1166

Change-Id: Ib375852c3372d61e9122db3ed48b40b73cd2ff15

jojochuang · 2020-03-06T02:36:43Z

I'm extremely sorry for that. Updated PR. The code compiles locally.

iwasakims

+1. mvn install without -DskipShade worked. I did not see relevant unit test fauilures on my local. No issue on mvn verify in hadoop-client-modules. Thanks, @jojochuang.

hadoop-yetus · 2020-03-06T04:21:02Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 33s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+0 🆗	mvndep	1m 12s	Maven dependency ordering for branch
+1 💚	mvninstall	22m 14s	trunk passed
+1 💚	compile	18m 12s	trunk passed
+1 💚	mvnsite	0m 56s	trunk passed
+1 💚	shadedclient	58m 41s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 54s	trunk passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 27s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 53s	the patch passed
+1 💚	compile	17m 23s	the patch passed
+1 💚	javac	17m 23s	the patch passed
+1 💚	mvnsite	0m 54s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	xml	0m 2s	The patch has no ill-formed XML file.
+1 💚	shadedclient	15m 35s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 53s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 24s	hadoop-project in the patch passed.
+1 💚	unit	0m 25s	hadoop-client-runtime in the patch passed.
+1 💚	asflicense	0m 45s	The patch does not generate ASF License warnings.
		103m 45s

Subsystem	Report/Notes
Docker	Client=19.03.7 Server=19.03.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1876/3/artifact/out/Dockerfile
GITHUB PR	#1876
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient xml
uname	Linux 833bbae1c1fa 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `004e955`
Default Java	1.8.0_242
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1876/3/testReport/
Max. process+thread count	310 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-client-modules/hadoop-client-runtime U: .
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1876/3/console
versions	git=2.7.4 maven=3.3.9
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

aajisaka · 2020-03-06T04:38:00Z

LGTM, +1. Thanks @jojochuang and @iwasakims

…e endless CVE patches. (apache#1876)

…e endless CVE patches. (apache#1876) (cherry picked from commit 69faaa1)

…e endless CVE patches. (apache#1876) (cherry picked from commit 69faaa1) Conflicts: hadoop-project/pom.xml

…e endless CVE patches. (apache#1876)

…e endless CVE patches. (apache#1876) (cherry picked from commit 69faaa1) Change-Id: I8bcb997e418f95445bbfd6f71b4ba6cde99dabac (cherry picked from commit 826c38c5c270e94823aa8bfcdc32a78f849f169e)

jojochuang added 3 commits March 5, 2020 18:36

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th…

a2ebe71

…e endless CVE patches. Change-Id: I452a76bd9f71a0dd2ffb0f32df35c941d3b24aa3

Shade javax/xml/bind

ea722ad

Change-Id: I2915844b8e74f7f3e4fbb62bde8f0db76bcf1166

Update jackson to the correct version.

a1f9995

Change-Id: Ib375852c3372d61e9122db3ed48b40b73cd2ff15

jojochuang force-pushed the HADOOP-16905 branch from e958eb7 to a1f9995 Compare March 6, 2020 02:36

iwasakims approved these changes Mar 6, 2020

View reviewed changes

iwasakims merged commit 69faaa1 into apache:trunk Mar 7, 2020

bilaharith pushed a commit to bilaharith/hadoop that referenced this pull request Mar 19, 2020

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th…

5df4453

…e endless CVE patches. (apache#1876)

RogPodge pushed a commit to RogPodge/hadoop that referenced this pull request Mar 25, 2020

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th…

3723609

…e endless CVE patches. (apache#1876)

jojochuang added a commit to jojochuang/hadoop that referenced this pull request Apr 23, 2020

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th…

61d5c94

…e endless CVE patches. (apache#1876) (cherry picked from commit 69faaa1)

jojochuang added a commit to jojochuang/hadoop that referenced this pull request Apr 23, 2020

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th…

d8ac8f2

…e endless CVE patches. (apache#1876) (cherry picked from commit 69faaa1)

aajisaka pushed a commit to aajisaka/hadoop that referenced this pull request Dec 3, 2021

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th…

d3e00a5

…e endless CVE patches. (apache#1876) (cherry picked from commit 69faaa1) Conflicts: hadoop-project/pom.xml

aajisaka mentioned this pull request Dec 3, 2021

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th… #3748

Merged

4 tasks

zhangxiping1 pushed a commit to zhangxiping1/hadoop that referenced this pull request Dec 13, 2022

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th…

7352462

…e endless CVE patches. (apache#1876)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from the endless CVE patches. #1876

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from the endless CVE patches. #1876

jojochuang commented Mar 5, 2020

hadoop-yetus commented Mar 5, 2020

iwasakims commented Mar 5, 2020

iwasakims commented Mar 5, 2020

jojochuang commented Mar 5, 2020

hadoop-yetus commented Mar 5, 2020

iwasakims commented Mar 6, 2020

jojochuang commented Mar 6, 2020

iwasakims left a comment

hadoop-yetus commented Mar 6, 2020

aajisaka commented Mar 6, 2020

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from the endless CVE patches. #1876

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from the endless CVE patches. #1876

Conversation

jojochuang commented Mar 5, 2020

NOTICE

hadoop-yetus commented Mar 5, 2020

iwasakims commented Mar 5, 2020

iwasakims commented Mar 5, 2020

jojochuang commented Mar 5, 2020

hadoop-yetus commented Mar 5, 2020

iwasakims commented Mar 6, 2020

jojochuang commented Mar 6, 2020

iwasakims left a comment

Choose a reason for hiding this comment

hadoop-yetus commented Mar 6, 2020

aajisaka commented Mar 6, 2020