HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th… #3748

aajisaka · 2021-12-03T15:14:27Z

…e endless CVE patches. (#1876)

(cherry picked from commit 69faaa1)

Conflicts:
hadoop-project/pom.xml

Description of PR

PR for testing by Jenkins. Backport HADOOP-16905 to branch-3.2. I'll backport HADOOP-17534 next.

How was this patch tested?

Not tested.

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

…e endless CVE patches. (apache#1876) (cherry picked from commit 69faaa1) Conflicts: hadoop-project/pom.xml

hadoop-yetus · 2021-12-03T17:06:45Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	11m 15s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ branch-3.2 Compile Tests _
+0 🆗	mvndep	3m 24s		Maven dependency ordering for branch
+1 💚	mvninstall	26m 17s		branch-3.2 passed
+1 💚	compile	15m 50s		branch-3.2 passed
+1 💚	mvnsite	0m 53s		branch-3.2 passed
+1 💚	javadoc	0m 48s		branch-3.2 passed
+1 💚	shadedclient	61m 50s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 37s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 41s		the patch passed
+1 💚	compile	14m 53s		the patch passed
+1 💚	javac	14m 53s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 50s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	0m 48s		the patch passed
+1 💚	shadedclient	15m 31s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 23s		hadoop-project in the patch passed.
+1 💚	unit	0m 23s		hadoop-client-runtime in the patch passed.
+1 💚	asflicense	0m 44s		The patch does not generate ASF License warnings.
		111m 21s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3748/1/artifact/out/Dockerfile
GITHUB PR	#3748
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml
uname	Linux 96e20edbf7cc 4.15.0-143-generic #147-Ubuntu SMP Wed Apr 14 16:10:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.2 / `d3e00a5`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~18.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3748/1/testReport/
Max. process+thread count	306 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-client-modules/hadoop-client-runtime U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3748/1/console
versions	git=2.17.1 maven=3.6.0
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

aajisaka · 2021-12-03T17:43:07Z

@brahmareddybattula Would you take a look?

virajjasani · 2021-12-04T14:16:36Z

Good to update some file in the top tree (outside of all modules) to trigger the full QA (e.g. adding a line etc)? Once full QA is done, we can revert that change.

This reverts commit 45f5595.

hadoop-yetus · 2021-12-06T08:50:43Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	8m 10s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ branch-3.2 Compile Tests _
+0 🆗	mvndep	3m 41s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 33s		branch-3.2 passed
+1 💚	compile	15m 9s		branch-3.2 passed
+1 💚	mvnsite	15m 42s		branch-3.2 passed
+1 💚	javadoc	6m 10s		branch-3.2 passed
+1 💚	shadedclient	78m 2s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 49s		Maven dependency ordering for patch
+1 💚	mvninstall	22m 33s		the patch passed
+1 💚	compile	15m 4s		the patch passed
+1 💚	javac	15m 4s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	13m 1s		the patch passed
+1 💚	xml	0m 4s		The patch has no ill-formed XML file.
+1 💚	javadoc	5m 51s		the patch passed
+1 💚	shadedclient	20m 35s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	237m 43s	/patch-unit-root.txt	root in the patch failed.
+1 💚	asflicense	1m 2s		The patch does not generate ASF License warnings.
		398m 53s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestBlockStoragePolicy

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3748/2/artifact/out/Dockerfile
GITHUB PR	#3748
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml
uname	Linux b5b5a9316c5c 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.2 / `4aedade`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~18.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3748/2/testReport/
Max. process+thread count	2995 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-client-modules/hadoop-client-runtime . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3748/2/console
versions	git=2.17.1 maven=3.6.0
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

aajisaka · 2021-12-06T10:17:01Z

Thank you @virajjasani for your comment. Updated the pom.xml to run the full unit tests. This precommit job didn't run all the tests because there is a test failure in hadoo-hdfs module and stopped testing.

aajisaka · 2021-12-06T10:41:33Z

Filed HADOOP-18035 to run all the unit tests.

virajjasani · 2021-12-06T10:42:53Z

The changelist available as part of build#2 is:

1. Modify a file to trigger the full QA (details / githubweb)
2. Revert "Modify a file to trigger the full QA" (details / githubweb)

Hence, looks like revert commit was also part of the build and that's why full QA was not triggered (as per logs). Hopefully test failure in one module would not prevent full QA based on what I have seen in recent PRs but it's still strange that full QA was not triggered. Maybe because after applying revert commit, build did not recognize any changed on the main pom file.

@aajisaka would you like to keep new line change in hadoop-main pom file for a while (~24 hr)? Once we get full QA results, we can revert it. Sounds good?

virajjasani · 2021-12-06T10:45:50Z

For instance, two build results had multiple test failures (including hadoop-hdfs modules), still hadoop-yarn modules tests were executed (PR #3749). Let me dig-in bit more to understand this behaviour on the current PR.

aajisaka · 2021-12-06T11:32:40Z

@virajjasani

Hence, looks like revert commit was also part of the build and that's why full QA was not triggered (as per logs).

It is not the reason why full QA was not triggered. I commented in HADOOP-18035 why the subsequent modules were not executed in branch-3.2 and branch-2.10.

virajjasani · 2021-12-06T11:36:48Z

I see, thanks @aajisaka

aajisaka · 2021-12-09T16:56:35Z

Now HADOOP-18040 is merged. Let me run the full QA again.

hadoop-yetus · 2021-12-10T06:24:07Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 41s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ branch-3.2 Compile Tests _
+0 🆗	mvndep	3m 43s		Maven dependency ordering for branch
+1 💚	mvninstall	24m 41s		branch-3.2 passed
+1 💚	compile	15m 52s		branch-3.2 passed
+1 💚	mvnsite	16m 25s		branch-3.2 passed
+1 💚	javadoc	6m 14s		branch-3.2 passed
+1 💚	shadedclient	80m 50s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 50s		Maven dependency ordering for patch
+1 💚	mvninstall	22m 33s		the patch passed
+1 💚	compile	14m 59s		the patch passed
+1 💚	javac	14m 59s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	13m 36s		the patch passed
+1 💚	xml	0m 4s		The patch has no ill-formed XML file.
+1 💚	javadoc	6m 16s		the patch passed
+1 💚	shadedclient	21m 10s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	647m 53s	/patch-unit-root.txt	root in the patch passed.
-1 ❌	asflicense	1m 23s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		805m 55s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestBlockStoragePolicy
	hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
	hadoop.hdfs.server.datanode.TestBPOfferService
	hadoop.yarn.sls.TestSLSRunner
	hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3748/3/artifact/out/Dockerfile
GITHUB PR	#3748
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml
uname	Linux 0bca3edce3d8 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.2 / `4aedade`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~18.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3748/3/testReport/
Max. process+thread count	2489 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-client-modules/hadoop-client-runtime . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3748/3/console
versions	git=2.17.1 maven=3.6.0
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

virajjasani · 2021-12-10T07:08:05Z

If TestSLSRunner and TestTimelineAuthFilterForV2 failures are not relevant, QA results look good IMHO.

aajisaka · 2021-12-10T07:21:12Z

Thanks. TestSLSRunner passed on my local. TestTimelineAuthFilterForV2 is always failing with and without the patch. I think it's okay to merge without binding +1 because this is a backport.

virajjasani · 2021-12-10T07:22:44Z

TestSLSRunner passed on my local. TestTimelineAuthFilterForV2 is always failing with and without the patch.

Sounds good. Otherwise it looks good anyways. +1 (non-binding)

aajisaka · 2021-12-10T07:24:24Z

Merged. Thank you @virajjasani for your review!

…e endless CVE patches (#3748) (cherry picked from commit 69faaa1) Conflicts: hadoop-project/pom.xml (cherry picked from commit bc68741)

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th…

d3e00a5

…e endless CVE patches. (apache#1876) (cherry picked from commit 69faaa1) Conflicts: hadoop-project/pom.xml

aajisaka requested a review from brahmareddybattula December 3, 2021 15:34

aajisaka added 2 commits December 6, 2021 11:10

Modify a file to trigger the full QA

45f5595

Revert "Modify a file to trigger the full QA"

4aedade

This reverts commit 45f5595.

aajisaka merged commit bc68741 into apache:branch-3.2 Dec 10, 2021

aajisaka deleted the HADOOP-16905-branch-3.2 branch December 10, 2021 07:24

asfgit pushed a commit that referenced this pull request Dec 16, 2021

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th…

a7a8529

…e endless CVE patches (#3748) (cherry picked from commit 69faaa1) Conflicts: hadoop-project/pom.xml (cherry picked from commit bc68741)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th… #3748

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th… #3748

aajisaka commented Dec 3, 2021

hadoop-yetus commented Dec 3, 2021

aajisaka commented Dec 3, 2021

virajjasani commented Dec 4, 2021 •

edited

Loading

hadoop-yetus commented Dec 6, 2021

aajisaka commented Dec 6, 2021

aajisaka commented Dec 6, 2021

virajjasani commented Dec 6, 2021

virajjasani commented Dec 6, 2021 •

edited

Loading

aajisaka commented Dec 6, 2021 •

edited

Loading

virajjasani commented Dec 6, 2021

aajisaka commented Dec 9, 2021

hadoop-yetus commented Dec 10, 2021

virajjasani commented Dec 10, 2021

aajisaka commented Dec 10, 2021 •

edited

Loading

virajjasani commented Dec 10, 2021

aajisaka commented Dec 10, 2021

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th… #3748

HADOOP-16905. Update jackson-databind to 2.10.3 to relieve us from th… #3748

Conversation

aajisaka commented Dec 3, 2021

Description of PR

How was this patch tested?

For code changes:

hadoop-yetus commented Dec 3, 2021

aajisaka commented Dec 3, 2021

virajjasani commented Dec 4, 2021 • edited Loading

hadoop-yetus commented Dec 6, 2021

aajisaka commented Dec 6, 2021

aajisaka commented Dec 6, 2021

virajjasani commented Dec 6, 2021

virajjasani commented Dec 6, 2021 • edited Loading

aajisaka commented Dec 6, 2021 • edited Loading

virajjasani commented Dec 6, 2021

aajisaka commented Dec 9, 2021

hadoop-yetus commented Dec 10, 2021

virajjasani commented Dec 10, 2021

aajisaka commented Dec 10, 2021 • edited Loading

virajjasani commented Dec 10, 2021

aajisaka commented Dec 10, 2021

virajjasani commented Dec 4, 2021 •

edited

Loading

virajjasani commented Dec 6, 2021 •

edited

Loading

aajisaka commented Dec 6, 2021 •

edited

Loading

aajisaka commented Dec 10, 2021 •

edited

Loading