YARN-10965. Centralize queue resource calculation based on CapacityVectors #3470

9uapaw · 2021-09-23T11:22:28Z

Description of PR

How was this patch tested?

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

(cherry picked from commit efc535e)

hadoop-yetus · 2021-10-29T20:56:04Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 41s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 5 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 50s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 17s		trunk passed
+1 💚	compile	9m 36s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	8m 24s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 40s		trunk passed
+1 💚	mvnsite	2m 18s		trunk passed
+1 💚	javadoc	1m 58s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 48s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	4m 13s		trunk passed
+1 💚	shadedclient	21m 8s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	21m 30s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 27s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 30s		the patch passed
+1 💚	compile	8m 57s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	8m 57s		the patch passed
+1 💚	compile	8m 14s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	8m 14s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 37s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 38 new + 146 unchanged - 0 fixed = 184 total (was 146)
+1 💚	mvnsite	2m 4s		the patch passed
+1 💚	javadoc	1m 44s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javadoc	0m 55s	/results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 2 new + 344 unchanged - 0 fixed = 346 total (was 344)
-1 ❌	spotbugs	2m 12s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	21m 18s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 8s		hadoop-yarn-api in the patch passed.
+1 💚	unit	96m 29s		hadoop-yarn-server-resourcemanager in the patch passed.
-1 ❌	asflicense	0m 53s	/results-asflicense.txt	The patch generated 6 ASF License warnings.
		237m 35s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
	Method org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.PercentageQueueCapacityCalculator.setup(CSQueue, String) seems to be useless At PercentageQueueCapacityCalculator.java:be useless At PercentageQueueCapacityCalculator.java:[line 18]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/1/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 9cfef34eebad 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `2bcb962`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/1/testReport/
Max. process+thread count	946 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-11-16T11:48:03Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 39s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 6 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 18s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 11s		trunk passed
+1 💚	compile	11m 4s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	9m 7s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 48s		trunk passed
+1 💚	mvnsite	2m 23s		trunk passed
+1 💚	javadoc	2m 4s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 54s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	4m 33s		trunk passed
+1 💚	shadedclient	23m 7s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	23m 29s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 30s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 39s		the patch passed
+1 💚	compile	10m 3s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	10m 3s		the patch passed
+1 💚	compile	9m 4s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	9m 4s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	1m 42s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 66 new + 166 unchanged - 0 fixed = 232 total (was 166)
+1 💚	mvnsite	2m 13s		the patch passed
+1 💚	javadoc	1m 51s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javadoc	0m 56s	/results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 2 new + 344 unchanged - 0 fixed = 346 total (was 344)
-1 ❌	spotbugs	2m 29s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	23m 23s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 11s		hadoop-yarn-api in the patch passed.
+1 💚	unit	99m 57s		hadoop-yarn-server-resourcemanager in the patch passed.
-1 ❌	asflicense	0m 55s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		253m 16s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
	org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.setChildQueues(Collection) does not release lock on all exception paths At ParentQueue.java:on all exception paths At ParentQueue.java:[line 300]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/2/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux bc192d57adc5 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `3dae1dc`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/2/testReport/
Max. process+thread count	948 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

…ulate the intermediate values that belong together

hadoop-yetus · 2021-11-18T12:56:24Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 43s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 6 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 57s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 26s		trunk passed
+1 💚	compile	10m 47s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	9m 8s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 44s		trunk passed
+1 💚	mvnsite	2m 24s		trunk passed
+1 💚	javadoc	2m 4s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 53s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	4m 37s		trunk passed
+1 💚	shadedclient	22m 31s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	22m 53s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 30s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 41s		the patch passed
+1 💚	compile	10m 5s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	10m 5s		the patch passed
+1 💚	compile	9m 10s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	9m 10s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 41s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 95 new + 166 unchanged - 0 fixed = 261 total (was 166)
+1 💚	mvnsite	2m 14s		the patch passed
+1 💚	javadoc	1m 44s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javadoc	0m 58s	/results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 3 new + 344 unchanged - 0 fixed = 347 total (was 344)
-1 ❌	spotbugs	2m 29s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	22m 29s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 6s		hadoop-yarn-api in the patch passed.
-1 ❌	unit	99m 24s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch passed.
-1 ❌	asflicense	0m 56s	/results-asflicense.txt	The patch generated 3 ASF License warnings.
		250m 34s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
	org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.setChildQueues(Collection) does not release lock on all exception paths At ParentQueue.java:on all exception paths At ParentQueue.java:[line 300]
Failed junit tests	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestMixedQueueResourceCalculation
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestUniformQueueResourceCalculation

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/3/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 36b5541a4301 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e549acc`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/3/testReport/
Max. process+thread count	944 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-11-18T18:01:11Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 52s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 6 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	16m 28s		Maven dependency ordering for branch
+1 💚	mvninstall	24m 27s		trunk passed
+1 💚	compile	10m 28s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	8m 45s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 38s		trunk passed
+1 💚	mvnsite	2m 2s		trunk passed
+1 💚	javadoc	1m 42s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 30s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	4m 9s		trunk passed
+1 💚	shadedclient	24m 5s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	24m 25s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 24s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 32s		the patch passed
+1 💚	compile	9m 46s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javac	9m 46s	/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 1 new + 401 unchanged - 1 fixed = 402 total (was 402)
+1 💚	compile	8m 42s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	8m 42s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 33s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 95 new + 166 unchanged - 1 fixed = 261 total (was 167)
+1 💚	mvnsite	1m 53s		the patch passed
+1 💚	javadoc	1m 31s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javadoc	0m 47s	/results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 3 new + 344 unchanged - 0 fixed = 347 total (was 344)
-1 ❌	spotbugs	2m 14s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	24m 26s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 0s		hadoop-yarn-api in the patch passed.
+1 💚	unit	114m 49s		hadoop-yarn-server-resourcemanager in the patch passed.
-1 ❌	asflicense	0m 46s	/results-asflicense.txt	The patch generated 3 ASF License warnings.
		268m 37s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
	org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.setChildQueues(Collection) does not release lock on all exception paths At ParentQueue.java:on all exception paths At ParentQueue.java:[line 300]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/4/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 00fa38f14614 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `6c629ea`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/4/testReport/
Max. process+thread count	936 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

…well

hadoop-yetus · 2021-11-22T12:03:35Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 6 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 45s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 10s		trunk passed
+1 💚	compile	11m 7s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	9m 20s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 39s		trunk passed
+1 💚	mvnsite	2m 21s		trunk passed
+1 💚	javadoc	2m 3s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 55s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	4m 37s		trunk passed
+1 💚	shadedclient	22m 23s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	22m 47s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 42s		the patch passed
+1 💚	compile	10m 10s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	10m 10s		the patch passed
+1 💚	compile	9m 8s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	9m 8s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 42s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 94 new + 166 unchanged - 0 fixed = 260 total (was 166)
+1 💚	mvnsite	2m 13s		the patch passed
+1 💚	javadoc	1m 52s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 46s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	4m 51s		the patch passed
+1 💚	shadedclient	22m 14s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 12s		hadoop-yarn-api in the patch passed.
+1 💚	unit	115m 54s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 46s		The patch does not generate ASF License warnings.
		266m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/5/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux ea4119581083 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `27e6344`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/5/testReport/
Max. process+thread count	951 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-11-22T14:38:04Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 6 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 54s		Maven dependency ordering for branch
+1 💚	mvninstall	25m 40s		trunk passed
+1 💚	compile	10m 57s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	9m 57s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 44s		trunk passed
+1 💚	mvnsite	2m 14s		trunk passed
+1 💚	javadoc	1m 53s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 46s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	4m 46s		trunk passed
+1 💚	shadedclient	24m 40s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	25m 1s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 25s		the patch passed
+1 💚	compile	10m 27s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	10m 27s		the patch passed
+1 💚	compile	9m 10s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	9m 10s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 44s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 93 new + 166 unchanged - 0 fixed = 259 total (was 166)
+1 💚	mvnsite	2m 14s		the patch passed
+1 💚	javadoc	1m 52s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 44s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	5m 4s		the patch passed
+1 💚	shadedclient	23m 37s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 13s		hadoop-yarn-api in the patch passed.
+1 💚	unit	99m 27s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 45s		The patch does not generate ASF License warnings.
		259m 12s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/6/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 164a3e35ba10 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e6609f2`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/6/testReport/
Max. process+thread count	945 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/6/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-11-29T17:56:44Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 2s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 6 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 36s		Maven dependency ordering for branch
+1 💚	mvninstall	28m 3s		trunk passed
+1 💚	compile	11m 35s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	9m 47s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	1m 47s		trunk passed
+1 💚	mvnsite	2m 13s		trunk passed
+1 💚	javadoc	1m 44s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 36s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	4m 45s		trunk passed
+1 💚	shadedclient	25m 50s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	26m 11s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 27s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 44s		the patch passed
+1 💚	compile	11m 19s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	11m 19s		the patch passed
+1 💚	compile	9m 46s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	9m 46s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 46s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 96 new + 167 unchanged - 0 fixed = 263 total (was 167)
+1 💚	mvnsite	2m 3s		the patch passed
+1 💚	javadoc	1m 39s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 26s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	4m 33s		the patch passed
+1 💚	shadedclient	24m 37s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 0s		hadoop-yarn-api in the patch passed.
+1 💚	unit	103m 27s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 57s		The patch does not generate ASF License warnings.
		266m 3s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/7/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux f6430b6151d9 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `784c1be`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/7/testReport/
Max. process+thread count	959 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/7/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

…pacity types

brumi1024

Thanks @9uapaw for this huge patch! The logic in general looks good to me, it's nicely done, however I haven't finished going through all of it in detail. In the meantime I posted a few minor comments.

...p-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/util/resource/ResourceUtils.java

...n/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractCSQueue.java

...adoop/yarn/server/resourcemanager/scheduler/capacity/AbsoluteResourceCapacityCalculator.java

...he/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java

brumi1024

Finished going through it. Apart from my minor comments I think this change looks good, it makes the capacity calculation more flexible.

...in/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ResourceVector.java

...hadoop/yarn/server/resourcemanager/scheduler/capacity/TestMixedQueueResourceCalculation.java

...rc/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java

...p-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/util/resource/ResourceUtils.java

...in/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ResourceVector.java

...op/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerQueueCapacityHandler.java

.../apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ResourceCalculationDriver.java

shuzirra · 2022-02-01T07:14:19Z

.../apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ResourceCalculationDriver.java

+    sumWeightsPerLabel.get(label).put(resourceName,
+        sumWeightsPerLabel.get(label).getOrDefault(resourceName, 0f) + value);


If this is called frequently we might consider using a mutable float here to reduce GC strain.

@9uapaw Is this comment fixed?

...e/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractQueueCapacityCalculator.java

hadoop-yetus · 2022-02-02T12:12:34Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	35m 45s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 1s		The patch appears to include 8 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	7m 46s		Maven dependency ordering for branch
-1 ❌	mvninstall	0m 22s	/branch-mvninstall-root.txt	root in trunk failed.
-1 ❌	compile	0m 23s	/branch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt	hadoop-yarn in trunk failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.
-1 ❌	compile	0m 23s	/branch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt	hadoop-yarn in trunk failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.
-0 ⚠️	checkstyle	0m 20s	/buildtool-branch-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	The patch fails to run checkstyle in hadoop-yarn
-1 ❌	mvnsite	0m 22s	/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt	hadoop-yarn-api in trunk failed.
-1 ❌	mvnsite	0m 23s	/branch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in trunk failed.
-1 ❌	javadoc	0m 23s	/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt	hadoop-yarn-api in trunk failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.
-1 ❌	javadoc	0m 24s	/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt	hadoop-yarn-server-resourcemanager in trunk failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.
-1 ❌	javadoc	1m 33s	/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt	hadoop-yarn-server-resourcemanager in trunk failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.
+1 💚	spotbugs	4m 32s		trunk passed
+1 💚	shadedclient	30m 56s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	31m 17s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	16m 31s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 40s		the patch passed
-1 ❌	compile	0m 45s	/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt	hadoop-yarn in the patch failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.
-1 ❌	javac	0m 45s	/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt	hadoop-yarn in the patch failed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.
-1 ❌	compile	0m 36s	/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt	hadoop-yarn in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.
-1 ❌	javac	0m 36s	/patch-compile-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt	hadoop-yarn in the patch failed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 33s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 262 new + 0 unchanged - 0 fixed = 262 total (was 0)
+1 💚	mvnsite	1m 34s		the patch passed
-1 ❌	javadoc	0m 31s	/results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt	hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 generated 99 new + 0 unchanged - 0 fixed = 99 total (was 0)
-1 ❌	javadoc	0m 41s	/results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt	hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 generated 99 new + 0 unchanged - 0 fixed = 99 total (was 0)
-1 ❌	javadoc	0m 38s	/results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt	hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 generated 346 new + 0 unchanged - 0 fixed = 346 total (was 0)
-1 ❌	spotbugs	2m 3s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	24m 7s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 49s		hadoop-yarn-api in the patch passed.
+1 💚	unit	100m 36s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 29s		The patch does not generate ASF License warnings.
		238m 33s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
	org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ResourceCalculationDriver.CALCULATOR_PRECEDENCE should be package protected At ResourceCalculationDriver.java: At ResourceCalculationDriver.java:[line 39]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/12/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 20fe1c32479c 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `faa9a9a`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/12/testReport/
Max. process+thread count	895 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/12/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-02-02T12:15:09Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	13m 13s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 8 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 17s		Maven dependency ordering for branch
+1 💚	mvninstall	22m 5s		trunk passed
+1 💚	compile	9m 43s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	8m 21s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 42s		trunk passed
+1 💚	mvnsite	2m 33s		trunk passed
+1 💚	javadoc	1m 59s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 48s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	4m 14s		trunk passed
+1 💚	shadedclient	21m 49s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	22m 10s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 33s		the patch passed
+1 💚	compile	8m 58s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
-1 ❌	javac	8m 58s	/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt	hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 generated 1 new + 392 unchanged - 1 fixed = 393 total (was 393)
+1 💚	compile	8m 14s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	8m 14s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 38s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 104 new + 157 unchanged - 0 fixed = 261 total (was 157)
+1 💚	mvnsite	2m 5s		the patch passed
+1 💚	javadoc	1m 45s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
-1 ❌	javadoc	0m 55s	/results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt	hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 generated 4 new + 342 unchanged - 0 fixed = 346 total (was 342)
-1 ❌	spotbugs	2m 16s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	21m 48s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 8s		hadoop-yarn-api in the patch passed.
+1 💚	unit	97m 24s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 53s		The patch does not generate ASF License warnings.
		253m 50s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
	org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ResourceCalculationDriver.CALCULATOR_PRECEDENCE should be package protected At ResourceCalculationDriver.java: At ResourceCalculationDriver.java:[line 39]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/11/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 42308a37fcce 4.15.0-161-generic #169-Ubuntu SMP Fri Oct 15 13:41:54 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `faa9a9a`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/11/testReport/
Max. process+thread count	949 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/11/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

shuzirra

I've reviewed the changes you made since my last request, and I think all of them are fine. I have a few suggestions to clean up the terminology, I think we are using the "child queue" terminology far too often, even in cases when we are only working on a single queue. But these are nice to have suggestions only.

I want to review the whole path as a whole again, especially focusing on the calculation parts, but if I don't find anythin there, I'll give you a +1. (However since the patch is complex, I'd like @szilard-nemeth to look at it as well before we merge it)

...e/hadoop/yarn/server/resourcemanager/scheduler/capacity/AbstractQueueCapacityCalculator.java

...ava/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CalculationContext.java

...op/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerQueueCapacityHandler.java

.../apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ResourceCalculationDriver.java

hadoop-yetus · 2022-02-18T13:29:45Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 2s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 8 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 30s		Maven dependency ordering for branch
+1 💚	mvninstall	25m 14s		trunk passed
+1 💚	compile	10m 38s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	compile	8m 47s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 41s		trunk passed
+1 💚	mvnsite	2m 1s		trunk passed
+1 💚	javadoc	1m 58s		trunk passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 47s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	4m 35s		trunk passed
+1 💚	shadedclient	27m 44s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	28m 5s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 26s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 45s		the patch passed
+1 💚	compile	11m 6s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
-1 ❌	javac	11m 6s	/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04.txt	hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 generated 1 new + 392 unchanged - 1 fixed = 393 total (was 393)
+1 💚	compile	8m 47s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	8m 47s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 37s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 99 new + 157 unchanged - 0 fixed = 256 total (was 157)
+1 💚	mvnsite	2m 7s		the patch passed
+1 💚	javadoc	1m 41s		the patch passed with JDK Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04
-1 ❌	javadoc	1m 29s	/results-javadoc-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07.txt	hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 generated 5 new + 342 unchanged - 0 fixed = 347 total (was 342)
-1 ❌	spotbugs	2m 38s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	27m 20s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	1m 8s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt	hadoop-yarn-api in the patch passed.
-1 ❌	unit	144m 49s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch passed.
-1 ❌	asflicense	0m 48s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		306m 57s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
	org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ResourceCalculationDriver.CALCULATOR_PRECEDENCE should be package protected At ResourceCalculationDriver.java: At ResourceCalculationDriver.java:[line 38]
Failed junit tests	hadoop.yarn.conf.TestYarnConfigurationFields
	hadoop.yarn.server.resourcemanager.TestApplicationACLs
	hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerQueueACLs
	hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
	hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriorityACLs
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
	hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
	hadoop.yarn.server.resourcemanager.TestClientRMService

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/13/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux d8b8787f5498 4.15.0-163-generic #171-Ubuntu SMP Fri Nov 5 11:55:11 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `c16c335`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.13+8-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/13/testReport/
Max. process+thread count	910 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/13/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-03-22T18:58:33Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 8 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 40s		Maven dependency ordering for branch
+1 💚	mvninstall	22m 58s		trunk passed
+1 💚	compile	9m 45s		trunk passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	8m 26s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 44s		trunk passed
+1 💚	mvnsite	2m 18s		trunk passed
+1 💚	javadoc	1m 59s		trunk passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 50s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	4m 13s		trunk passed
+1 💚	shadedclient	22m 13s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	22m 35s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 27s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 31s		the patch passed
+1 💚	compile	9m 2s		the patch passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javac	9m 2s	/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt	hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 generated 1 new + 392 unchanged - 1 fixed = 393 total (was 393)
+1 💚	compile	8m 18s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	8m 18s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 38s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 92 new + 155 unchanged - 0 fixed = 247 total (was 155)
+1 💚	mvnsite	2m 8s		the patch passed
+1 💚	javadoc	1m 46s		the patch passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 41s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 ❌	spotbugs	2m 13s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	21m 32s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 7s		hadoop-yarn-api in the patch passed.
+1 💚	unit	97m 12s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 51s		The patch does not generate ASF License warnings.
		241m 30s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
	org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ResourceCalculationDriver.CALCULATOR_PRECEDENCE should be package protected At ResourceCalculationDriver.java: At ResourceCalculationDriver.java:[line 38]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/14/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 91b2287df332 4.15.0-161-generic #169-Ubuntu SMP Fri Oct 15 13:41:54 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `785f2fa`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/14/testReport/
Max. process+thread count	995 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/14/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

shuzirra

@9uapaw thank you for the update, and for this big patch LGTM+1. But since this is a large patch, I'd like to wait for @szilard-nemeth 's review as well, before we move on to merging this in.

hadoop-yetus · 2022-04-01T17:35:58Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 8 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 40s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 5s		trunk passed
+1 💚	compile	9m 45s		trunk passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	8m 24s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 44s		trunk passed
+1 💚	mvnsite	2m 18s		trunk passed
+1 💚	javadoc	2m 0s		trunk passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 41s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	4m 46s		trunk passed
+1 💚	shadedclient	23m 34s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	23m 56s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 26s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 33s		the patch passed
+1 💚	compile	9m 26s		the patch passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javac	9m 26s	/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04.txt	hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 generated 1 new + 392 unchanged - 1 fixed = 393 total (was 393)
+1 💚	compile	8m 18s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	8m 18s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 38s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 92 new + 154 unchanged - 0 fixed = 246 total (was 154)
+1 💚	mvnsite	2m 6s		the patch passed
+1 💚	javadoc	1m 46s		the patch passed with JDK Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	1m 39s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
-1 ❌	spotbugs	2m 16s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	22m 9s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 10s		hadoop-yarn-api in the patch passed.
+1 💚	unit	97m 23s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 54s		The patch does not generate ASF License warnings.
		244m 36s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
	org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ResourceCalculationDriver.CALCULATOR_PRECEDENCE should be package protected At ResourceCalculationDriver.java: At ResourceCalculationDriver.java:[line 38]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/15/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 7798e76cc2fc 4.15.0-161-generic #169-Ubuntu SMP Fri Oct 15 13:41:54 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / ced18cbfdfe5978c4b8563cb1828e05582c88b9c
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.14+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/15/testReport/
Max. process+thread count	999 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/15/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2022-08-30T18:10:39Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 47s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 8 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 33s		Maven dependency ordering for branch
+1 💚	mvninstall	25m 19s		trunk passed
+1 💚	compile	11m 48s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	compile	9m 17s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	checkstyle	1m 59s		trunk passed
+1 💚	mvnsite	2m 43s		trunk passed
+1 💚	javadoc	2m 30s		trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	2m 33s		trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	5m 5s		trunk passed
+1 💚	shadedclient	22m 29s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	22m 55s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 38s		the patch passed
+1 💚	compile	9m 5s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
-1 ❌	javac	9m 5s	/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1.txt	hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 generated 2 new + 738 unchanged - 3 fixed = 740 total (was 741)
+1 💚	compile	8m 39s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	javac	8m 39s		hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~~20.04-b07 generated 0 new + 651 unchanged - 1 fixed = 651 total (was 652)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 49s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 92 new + 150 unchanged - 0 fixed = 242 total (was 150)
+1 💚	mvnsite	2m 32s		the patch passed
+1 💚	javadoc	2m 16s		the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚	javadoc	1m 58s		the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚	spotbugs	4m 48s		the patch passed
+1 💚	shadedclient	22m 19s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 36s		hadoop-yarn-api in the patch passed.
+1 💚	unit	99m 16s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	1m 9s		The patch does not generate ASF License warnings.
		259m 34s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/16/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 1e0930c998cc 4.15.0-191-generic #202-Ubuntu SMP Thu Aug 4 01:49:29 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `673a8c0`
Default Java	Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/16/testReport/
Max. process+thread count	944 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/16/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

...che/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerQueueManager.java

...op/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerQueueCapacityHandler.java

szilard-nemeth · 2023-01-10T21:12:49Z

.../apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ResourceCalculationDriver.java

+    sumWeightsPerLabel.get(label).put(resourceName,
+        sumWeightsPerLabel.get(label).getOrDefault(resourceName, 0f) + value);


@9uapaw Is this comment fixed?

szilard-nemeth · 2023-01-10T21:15:35Z

Hi @9uapaw ,
Awesome patch, really liked the quality of your code, congrats.
Have left 3 comments, please check them.
If you address those, I will +1 this and commit it as well.

Thanks.

szilard-nemeth · 2023-01-10T21:17:22Z

@9uapaw Also, we need a fresh Jenkins build so a rebase (or empty commit) is required.

hadoop-yetus · 2023-01-11T00:20:00Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 52s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 8 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 33s		Maven dependency ordering for branch
+1 💚	mvninstall	33m 46s		trunk passed
+1 💚	compile	10m 31s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	8m 50s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	1m 48s		trunk passed
+1 💚	mvnsite	2m 2s		trunk passed
-1 ❌	javadoc	0m 53s	/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-yarn-server-resourcemanager in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	1m 31s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	4m 22s		trunk passed
+1 💚	shadedclient	27m 17s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	27m 36s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 24s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 42s		the patch passed
+1 💚	compile	9m 49s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
-1 ❌	javac	9m 49s	/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 generated 2 new + 719 unchanged - 3 fixed = 721 total (was 722)
+1 💚	compile	8m 47s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	8m 47s		hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_352-8u352-ga-1~~20.04-b08 with JDK Private Build-1.8.0_352-8u352-ga-1~~20.04-b08 generated 0 new + 637 unchanged - 1 fixed = 637 total (was 638)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 37s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 92 new + 151 unchanged - 0 fixed = 243 total (was 151)
+1 💚	mvnsite	1m 54s		the patch passed
-1 ❌	javadoc	0m 50s	/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	1m 24s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	4m 28s		the patch passed
+1 💚	shadedclient	28m 9s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	1m 3s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt	hadoop-yarn-api in the patch passed.
+1 💚	unit	117m 29s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 48s		The patch does not generate ASF License warnings.
		288m 1s

Reason	Tests
Failed junit tests	hadoop.yarn.conf.TestYarnConfigurationFields

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/17/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 70b54edd9e1d 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `673a8c0`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/17/testReport/
Max. process+thread count	920 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/17/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2023-01-26T00:32:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 8 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 1s		Maven dependency ordering for branch
+1 💚	mvninstall	32m 1s		trunk passed
+1 💚	compile	10m 6s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	8m 55s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	1m 47s		trunk passed
+1 💚	mvnsite	2m 19s		trunk passed
-1 ❌	javadoc	0m 54s	/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-yarn-server-resourcemanager in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	1m 45s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	4m 35s		trunk passed
+1 💚	shadedclient	25m 32s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	25m 53s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 43s		the patch passed
+1 💚	compile	9m 19s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
-1 ❌	javac	9m 19s	/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 generated 2 new + 719 unchanged - 3 fixed = 721 total (was 722)
+1 💚	compile	8m 38s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	8m 38s		hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_352-8u352-ga-1~~20.04-b08 with JDK Private Build-1.8.0_352-8u352-ga-1~~20.04-b08 generated 0 new + 637 unchanged - 1 fixed = 637 total (was 638)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 37s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 92 new + 150 unchanged - 0 fixed = 242 total (was 150)
+1 💚	mvnsite	2m 4s		the patch passed
-1 ❌	javadoc	0m 53s	/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	1m 36s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	4m 33s		the patch passed
+1 💚	shadedclient	25m 56s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 9s		hadoop-yarn-api in the patch passed.
+1 💚	unit	99m 9s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 54s		The patch does not generate ASF License warnings.
		264m 42s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/18/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 27e245452ba0 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `2958698`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/18/testReport/
Max. process+thread count	938 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/18/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

szilard-nemeth · 2023-01-26T13:56:40Z

Added commit that fixes checkstyle issues.
Except some of those fall into these types:

Variable 'configuredMaxCapacityVectors' must be private and have accessor methods. [VisibilityModifier]
Redundant 'public' modifier. [RedundantModifier]
QueueUpdateWarning.java:48: public QueueUpdateWarning ofQueue(String queue) {:46: 'queue' hides a field. [HiddenField]

LGTM +1, waiting for Jenkins

hadoop-yetus · 2023-01-26T18:20:35Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 8 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 20s		Maven dependency ordering for branch
+1 💚	mvninstall	33m 16s		trunk passed
+1 💚	compile	10m 20s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	8m 29s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	1m 46s		trunk passed
+1 💚	mvnsite	2m 18s		trunk passed
-1 ❌	javadoc	1m 1s	/branch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-yarn-server-resourcemanager in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	1m 48s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	4m 24s		trunk passed
+1 💚	shadedclient	24m 34s		branch has no errors when building and testing our client artifacts.
-0 ⚠️	patch	24m 56s		Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 43s		the patch passed
+1 💚	compile	9m 4s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
-1 ❌	javac	9m 4s	/results-compile-javac-hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-yarn-project_hadoop-yarn-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 generated 2 new + 719 unchanged - 3 fixed = 721 total (was 722)
+1 💚	compile	8m 26s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	8m 26s		hadoop-yarn-project_hadoop-yarn-jdkPrivateBuild-1.8.0_352-8u352-ga-1~~20.04-b08 with JDK Private Build-1.8.0_352-8u352-ga-1~~20.04-b08 generated 0 new + 637 unchanged - 1 fixed = 637 total (was 638)
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 1 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-0 ⚠️	checkstyle	1m 39s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn.txt	hadoop-yarn-project/hadoop-yarn: The patch generated 25 new + 150 unchanged - 0 fixed = 175 total (was 150)
+1 💚	mvnsite	2m 6s		the patch passed
-1 ❌	javadoc	0m 56s	/patch-javadoc-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-yarn-server-resourcemanager in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	1m 40s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	4m 26s		the patch passed
+1 💚	shadedclient	25m 9s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 11s		hadoop-yarn-api in the patch passed.
+1 💚	unit	99m 14s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 54s		The patch does not generate ASF License warnings.
		263m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/19/artifact/out/Dockerfile
GITHUB PR	#3470
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 772d560d8fe9 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `ce2618d`
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/19/testReport/
Max. process+thread count	949 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3470/19/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

szilard-nemeth · 2023-01-26T18:47:32Z

Thanks @9uapaw for this huge contribution.
Latest patch LGTM, committed to trunk.
Thanks @brumi1024 , @shuzirra for the reviews.

9uapaw added 9 commits September 13, 2021 15:39

YARN-10930. Introduce universal capacity resource vector

86b32bd

YARN-10930. Cleanup for QueueCapacityVector

b935b29

YARN-10965. Introduce enhanced queue calculation

5f3c0da

YARN-10930. Cover additional test cases

6ee8dc2

YARN-10965. Extend

ca8016c

YARN-10965. Extend 2

4fb353c

YARN-10930. Fix review feedbacks

8df3901

(cherry picked from commit efc535e)

YARN-10965. Simplify calculation

8c6559f

YARN-10965. Simplify calculators

0b7adc8

9uapaw force-pushed the YARN-10965 branch from 42cac72 to 0b7adc8 Compare October 29, 2021 15:59

Merge remote-tracking branch 'origin/trunk' into YARN-10965

2bcb962

9uapaw added 2 commits November 15, 2021 14:23

YARN-10965. Simplify logic in QueueHandler

83aabd0

YARN-10965. Implement strict resource iteration order

3dae1dc

9uapaw changed the title ~~YARN-10965. Introduce enhanced queue calculation~~ YARN-10965. Centralize queue resource calculation based on CapacityVectors Nov 16, 2021

YARN-10965. Introduce driver concept to simplify the logic and encaps…

e549acc

…ulate the intermediate values that belong together

YARN-10965. Fix root driver child queue setting

6c629ea

9uapaw added 2 commits November 22, 2021 08:32

YARN-10965. Fix warnings and nits

27e6344

YARN-10965. Make Absolute calculator use remaining resource ratio as …

e6609f2

…well

YARN-10965. Create node label test

784c1be

YARN-10965. Make rounding strategies more flexible with regards to ca…

0793322

…pacity types

brumi1024 reviewed Dec 3, 2021

View reviewed changes

brumi1024 reviewed Dec 6, 2021

View reviewed changes

YARN-10965. Simplify test API

75bca46

shuzirra requested changes Feb 1, 2022

View reviewed changes

9uapaw added 4 commits February 1, 2022 11:14

YARN-10965. Introduce calculation iteration context

4deb72a

YARN-10965. Fix review feedbacks

044cf3b

Merge branch 'trunk' into YARN-10965

c648a82

YARN-10965. Remove unnecessary change in LeafQueue

faa9a9a

shuzirra reviewed Feb 17, 2022

View reviewed changes

YARN-10965. Fix review feedback

c16c335

9uapaw added 2 commits March 22, 2022 15:44

Merge branch 'trunk' into YARN-10965

415342d

YARN-10965. Fix checkstyle issues and nomenclature nits

785f2fa

shuzirra approved these changes Apr 1, 2022

View reviewed changes

9uapaw added 2 commits August 30, 2022 15:02

Merge branch 'trunk' into YARN-10965

edbf3da

YARN-10965. Fix javac and spotbugs errors

673a8c0

9uapaw force-pushed the YARN-10965 branch from ced18cb to 673a8c0 Compare August 30, 2022 13:49

szilard-nemeth reviewed Jan 10, 2023

View reviewed changes

Merge branch 'apache:trunk' into YARN-10965

2958698

Fix checkstyle

ce2618d

szilard-nemeth closed this Jan 26, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

YARN-10965. Centralize queue resource calculation based on CapacityVectors #3470

YARN-10965. Centralize queue resource calculation based on CapacityVectors #3470

9uapaw commented Sep 23, 2021

hadoop-yetus commented Oct 29, 2021

hadoop-yetus commented Nov 16, 2021

hadoop-yetus commented Nov 18, 2021

hadoop-yetus commented Nov 18, 2021

hadoop-yetus commented Nov 22, 2021

hadoop-yetus commented Nov 22, 2021

hadoop-yetus commented Nov 29, 2021

brumi1024 left a comment

brumi1024 left a comment

shuzirra Feb 1, 2022

szilard-nemeth Jan 10, 2023

hadoop-yetus commented Feb 2, 2022

hadoop-yetus commented Feb 2, 2022

shuzirra left a comment

hadoop-yetus commented Feb 18, 2022

hadoop-yetus commented Mar 22, 2022

shuzirra left a comment

hadoop-yetus commented Apr 1, 2022

hadoop-yetus commented Aug 30, 2022

szilard-nemeth Jan 10, 2023

szilard-nemeth commented Jan 10, 2023

szilard-nemeth commented Jan 10, 2023

hadoop-yetus commented Jan 11, 2023

hadoop-yetus commented Jan 26, 2023

szilard-nemeth commented Jan 26, 2023

hadoop-yetus commented Jan 26, 2023

szilard-nemeth commented Jan 26, 2023

		sumWeightsPerLabel.get(label).put(resourceName,
		sumWeightsPerLabel.get(label).getOrDefault(resourceName, 0f) + value);

YARN-10965. Centralize queue resource calculation based on CapacityVectors #3470

YARN-10965. Centralize queue resource calculation based on CapacityVectors #3470

Conversation

9uapaw commented Sep 23, 2021

Description of PR

How was this patch tested?

For code changes:

hadoop-yetus commented Oct 29, 2021

hadoop-yetus commented Nov 16, 2021

hadoop-yetus commented Nov 18, 2021

hadoop-yetus commented Nov 18, 2021

hadoop-yetus commented Nov 22, 2021

hadoop-yetus commented Nov 22, 2021

hadoop-yetus commented Nov 29, 2021

brumi1024 left a comment

Choose a reason for hiding this comment

brumi1024 left a comment

Choose a reason for hiding this comment

shuzirra Feb 1, 2022

Choose a reason for hiding this comment

szilard-nemeth Jan 10, 2023

Choose a reason for hiding this comment

hadoop-yetus commented Feb 2, 2022

hadoop-yetus commented Feb 2, 2022

shuzirra left a comment

Choose a reason for hiding this comment

hadoop-yetus commented Feb 18, 2022

hadoop-yetus commented Mar 22, 2022

shuzirra left a comment

Choose a reason for hiding this comment

hadoop-yetus commented Apr 1, 2022

hadoop-yetus commented Aug 30, 2022

szilard-nemeth Jan 10, 2023

Choose a reason for hiding this comment

szilard-nemeth commented Jan 10, 2023

szilard-nemeth commented Jan 10, 2023

hadoop-yetus commented Jan 11, 2023

hadoop-yetus commented Jan 26, 2023

szilard-nemeth commented Jan 26, 2023

hadoop-yetus commented Jan 26, 2023

szilard-nemeth commented Jan 26, 2023