HBASE-27892 Report memstore on-heap and off-heap size as jmx metrics #5308

jinggou · 2023-06-22T23:07:41Z

No description provided.

virajjasani

left a minor nit, otherwise looks good

virajjasani · 2023-06-22T23:21:52Z

...oop-compat/src/main/java/org/apache/hadoop/hbase/regionserver/MetricsRegionServerSource.java

@@ -238,6 +238,10 @@ public interface MetricsRegionServerSource extends BaseSource, JvmPauseMonitorSo
  String MAX_COMPACTED_STORE_FILE_REF_COUNT = "maxCompactedStoreFileRefCount";
  String MEMSTORE_SIZE = "memStoreSize";
  String MEMSTORE_SIZE_DESC = "Size of the memstore";
+  String MEMSTORE_ONHEAP_SIZE = "memStoreOnHeapSize";


let's keep memStoreHeapSize as Duo suggested and it is also referenced as memStoreHeapSize at multiple places?

Apache-HBase · 2023-06-22T23:44:31Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 50s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 46s	master passed
+1 💚	compile	2m 44s	master passed
+1 💚	checkstyle	0m 46s	master passed
+1 💚	spotless	0m 43s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 52s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 9s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 34s	the patch passed
+1 💚	compile	2m 41s	the patch passed
+1 💚	javac	2m 41s	the patch passed
+1 💚	checkstyle	0m 42s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	9m 2s	Patch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚	spotless	0m 41s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 4s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 20s	The patch does not generate ASF License warnings.
		34m 10s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5308
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 85d64bb19c56 5.4.0-152-generic #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `da171c3`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-hadoop-compat hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/1/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-06-23T03:10:42Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 18s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 46s	master passed
+1 💚	compile	1m 0s	master passed
+1 💚	shadedjars	4m 39s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 39s	the patch passed
+1 💚	compile	1m 0s	the patch passed
+1 💚	javac	1m 0s	the patch passed
+1 💚	shadedjars	4m 37s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 39s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 43s	hbase-hadoop-compat in the patch passed.
+1 💚	unit	216m 11s	hbase-server in the patch passed.
		240m 27s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5308
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b565cc5bc876 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `da171c3`
Default Java	Eclipse Adoptium-11.0.17+8
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/1/testReport/
Max. process+thread count	4671 (vs. ulimit of 30000)
modules	C: hbase-hadoop-compat hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-06-23T03:49:04Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 30s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 58s	master passed
+1 💚	compile	0m 51s	master passed
+1 💚	shadedjars	5m 21s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 32s	the patch passed
+1 💚	compile	0m 56s	the patch passed
+1 💚	javac	0m 56s	the patch passed
+1 💚	shadedjars	5m 44s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 41s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 36s	hbase-hadoop-compat in the patch passed.
-1 ❌	unit	252m 19s	hbase-server in the patch failed.
		278m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5308
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 6c9c20944eb9 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `da171c3`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/1/testReport/
Max. process+thread count	4605 (vs. ulimit of 30000)
modules	C: hbase-hadoop-compat hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/1/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-06-23T20:23:28Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	4m 42s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 48s	master passed
+1 💚	compile	2m 43s	master passed
+1 💚	checkstyle	0m 46s	master passed
+1 💚	spotless	0m 43s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 51s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 11s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 38s	the patch passed
+1 💚	compile	2m 40s	the patch passed
+1 💚	javac	2m 40s	the patch passed
+1 💚	checkstyle	0m 42s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	9m 0s	Patch does not cause any errors with Hadoop 3.2.4 3.3.5.
+1 💚	spotless	0m 43s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 8s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 20s	The patch does not generate ASF License warnings.
		38m 6s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#5308
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux 0ed6b62c38b1 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `66fd6db`
Default Java	Eclipse Adoptium-11.0.17+8
Max. process+thread count	79 (vs. ulimit of 30000)
modules	C: hbase-hadoop-compat hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/2/console
versions	git=2.34.1 maven=3.8.6 spotbugs=4.7.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-06-23T23:54:56Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 18s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 51s	master passed
+1 💚	compile	1m 33s	master passed
+1 💚	shadedjars	4m 56s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 32s	the patch passed
+1 💚	compile	1m 1s	the patch passed
+1 💚	javac	1m 1s	the patch passed
+1 💚	shadedjars	4m 45s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 42s	hbase-hadoop-compat in the patch passed.
-1 ❌	unit	222m 2s	hbase-server in the patch failed.
		249m 24s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#5308
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 853b2bb98ba6 5.4.0-148-generic #165-Ubuntu SMP Tue Apr 18 08:53:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `66fd6db`
Default Java	Eclipse Adoptium-11.0.17+8
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/2/testReport/
Max. process+thread count	4696 (vs. ulimit of 30000)
modules	C: hbase-hadoop-compat hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2023-06-24T00:18:32Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+0 🆗	mvndep	0m 13s	Maven dependency ordering for branch
+1 💚	mvninstall	2m 28s	master passed
+1 💚	compile	0m 47s	master passed
+1 💚	shadedjars	5m 11s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 35s	master passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s	Maven dependency ordering for patch
+1 💚	mvninstall	2m 20s	the patch passed
+1 💚	compile	0m 58s	the patch passed
+1 💚	javac	0m 58s	the patch passed
+1 💚	shadedjars	5m 54s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	the patch passed
		_ Other Tests _
+1 💚	unit	0m 36s	hbase-hadoop-compat in the patch passed.
-1 ❌	unit	248m 16s	hbase-server in the patch failed.
		273m 9s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#5308
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 16d6c1ba90f2 5.4.0-1101-aws #109~18.04.1-Ubuntu SMP Mon Apr 24 20:40:49 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `66fd6db`
Default Java	Temurin-1.8.0_352-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/2/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/2/testReport/
Max. process+thread count	4656 (vs. ulimit of 30000)
modules	C: hbase-hadoop-compat hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-5308/2/console
versions	git=2.34.1 maven=3.8.6
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

jinggou · 2023-06-27T00:40:16Z

I ran the tests locally and they passed.

…5308) Signed-off-by: Viraj Jasani <[email protected]>

…metrics (apache#5308)" This reverts commit 7c49300.

…metrics (#5308)" (#5311) This reverts commit 7c49300.

…pache#5308) Signed-off-by: Viraj Jasani <[email protected]> (cherry picked from commit 7c49300) Change-Id: I5111265d8b203cbb77a754aee3d1609af01dface

…metrics (apache#5308)" (apache#5311) This reverts commit 7c49300. (cherry picked from commit c0df943) Change-Id: I6e6aafda6ef5781761cf0bb5a3461b3060f6b94b

HBASE-27892 Report memstore on-heap and off-heap size as jmx metrics

ed75c38

jinggou mentioned this pull request Jun 22, 2023

HBASE-27948 Report memstore on-heap and off-heap size as jmx metrics in sub=Memory bean #5293

Merged

virajjasani reviewed Jun 22, 2023

View reviewed changes

rename memStoreOnHeapSize to memStoreHeapSize

b16b932

virajjasani approved these changes Jun 23, 2023

View reviewed changes

virajjasani requested a review from bbeaudreault June 23, 2023 22:54

virajjasani merged commit 0637bbc into apache:master Jun 27, 2023

virajjasani pushed a commit that referenced this pull request Jun 27, 2023

HBASE-27892 Report memstore on-heap and off-heap size as jmx metrics (#…

58275bb

…5308) Signed-off-by: Viraj Jasani <[email protected]>

virajjasani pushed a commit that referenced this pull request Jun 27, 2023

HBASE-27892 Report memstore on-heap and off-heap size as jmx metrics (#…

267d621

…5308) Signed-off-by: Viraj Jasani <[email protected]>

virajjasani pushed a commit that referenced this pull request Jun 27, 2023

HBASE-27892 Report memstore on-heap and off-heap size as jmx metrics (#…

60580b8

…5308) Signed-off-by: Viraj Jasani <[email protected]>

virajjasani pushed a commit that referenced this pull request Jun 27, 2023

HBASE-27892 Report memstore on-heap and off-heap size as jmx metrics (#…

7c49300

…5308) Signed-off-by: Viraj Jasani <[email protected]>

petersomogyi added a commit to petersomogyi/hbase that referenced this pull request Jun 27, 2023

Revert "HBASE-27892 Report memstore on-heap and off-heap size as jmx …

83a516f

…metrics (apache#5308)" This reverts commit 7c49300.

petersomogyi added a commit that referenced this pull request Jun 27, 2023

Revert "HBASE-27892 Report memstore on-heap and off-heap size as jmx …

c0df943

…metrics (#5308)" (#5311) This reverts commit 7c49300.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HBASE-27892 Report memstore on-heap and off-heap size as jmx metrics #5308

HBASE-27892 Report memstore on-heap and off-heap size as jmx metrics #5308

jinggou commented Jun 22, 2023

virajjasani left a comment

virajjasani Jun 22, 2023

Apache-HBase commented Jun 22, 2023

Apache-HBase commented Jun 23, 2023

Apache-HBase commented Jun 23, 2023

Apache-HBase commented Jun 23, 2023

Apache-HBase commented Jun 23, 2023

Apache-HBase commented Jun 24, 2023

jinggou commented Jun 27, 2023 •

edited

Loading

HBASE-27892 Report memstore on-heap and off-heap size as jmx metrics #5308

HBASE-27892 Report memstore on-heap and off-heap size as jmx metrics #5308

Conversation

jinggou commented Jun 22, 2023

virajjasani left a comment

Choose a reason for hiding this comment

virajjasani Jun 22, 2023

Choose a reason for hiding this comment

Apache-HBase commented Jun 22, 2023

Apache-HBase commented Jun 23, 2023

Apache-HBase commented Jun 23, 2023

Apache-HBase commented Jun 23, 2023

Apache-HBase commented Jun 23, 2023

Apache-HBase commented Jun 24, 2023

jinggou commented Jun 27, 2023 • edited Loading

jinggou commented Jun 27, 2023 •

edited

Loading