Info on compressed ordinary object pointers #15489

jasontedor · 2015-12-16T19:03:33Z

This commit adds to JvmInfo the status of whether or not compressed
ordinary object pointers are enabled. Additionally, logging of the max
heap size and the status of the compressed ordinary object pointers flag
are provided on startup.

Relates #13187, relates elastic/elasticsearch-definitive-guide#455

jasontedor · 2015-12-16T19:08:09Z

This provides log messages on startup that look like

[2015-12-16 13:53:33,417][INFO ][env                      ] [Illyana Rasputin] heap size [989.8mb], compressed ordinary object pointers [true]

and

[2015-12-16 13:54:25,051][INFO ][env                      ] [Everyman] heap size [31.9gb], compressed ordinary object pointers [false]

and provides a using_compressed_ordinary_object_pointers field in the jvm field of the nodes info API response.

polyfractal · 2015-12-16T19:12:57Z

Woo! The log message LGTM, but I don't think I'm qualified to comment on the JvmInfo code itself :)

Should this be added to the /_cat/nodes output too as an optional column? Is it possible to backport this to the 2.x series, or something technical which disallows it?

nik9000 · 2015-12-16T19:13:40Z

core/src/main/java/org/elasticsearch/monitor/jvm/JvmInfo.java

+            VMOption usingCompressedOopsVmOption = hotSpotDiagnosticMXBean.getVMOption("UseCompressedOops");
+            info.usingCompressedOops = Boolean.parseBoolean(usingCompressedOopsVmOption.getValue());
+        } catch (Throwable t) {
+            info.usingCompressedOops = false;


false or "unknown"?

@nik9000 I considered that and I don't mind going either way.

I think I'd prefer if it were just a string/enum/whatever that made the logs and api spit out "true"/"false"/"unknown".

@nik9000 Done in c20d5e41b12105045ee19834bc3b18ea906dddd3.

jasontedor · 2015-12-16T19:22:34Z

Should this be added to the /_cat/nodes output too as an optional column?

@polyfractal I considered that and I ultimately decided against; the reason that I decided against is because nearly all of the information that is in JvmInfo is not provided in the cat nodes API. That's just my thinking; I'm certainly open to other thoughts.

Is it possible to backport this to the 2.x series, or something technical which disallows it?

It's possible.

polyfractal · 2015-12-16T19:26:20Z

I considered that and I ultimately decided against; the reason that I decided against is because nearly all of the information that is in JvmInfo is not provided in the cat nodes API.

Makes sense. Someone would only need to check this once, and you'd never need to monitor it dynamically... no need to clutter up the cat endpoint with it :)

rmuir · 2015-12-16T20:10:19Z

Is reflection on this API really allowed in java 9? We should not add this if not.

jasontedor · 2015-12-16T20:19:10Z

Is reflection on this API really allowed in java 9? We should not add this if not.

@rmuir Compiled with the 9-ea+96-2015-12-10-032020.javare.4030.nc compiler, and run on that JVM, it's still available and works.

                "using_compressed_ordinary_object_pointers": true,
                "version": "9-ea",
                "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
                "vm_vendor": "Oracle Corporation",
                "vm_version": "9-ea+96-2015-12-10-032020.javare.4030.nc"

nik9000 · 2015-12-16T20:32:50Z

LGTM if @rmuir is ok with it.

rmuir · 2015-12-16T21:08:28Z

that is not a jigsaw build.

jasontedor · 2015-12-16T21:21:26Z

that is not a jigsaw build.

@rmuir Oops, I had JAVA_HOME set to the wrong build; thanks for catching that. That said, it works against the latest Jigsaw build 9-ea+96-jigsaw-nightly-h4080-20151215:

                "using_compressed_ordinary_object_pointers": true,
                "version": "9-ea",
                "vm_name": "Java HotSpot(TM) 64-Bit Server VM",
                "vm_vendor": "Oracle Corporation",
                "vm_version": "9-ea+96-jigsaw-nightly-h4080-20151215"

rmuir · 2015-12-16T21:55:50Z

OK, looks good, thanks for checking, like the other management apis we do this trick for, it is a public api (just not guaranteed in all jvms) and the reflection is correct.

This commit adds to JvmInfo the status of whether or not compressed ordinary object pointers are enabled. Additionally, logging of the max heap size and the status of the compressed ordinary object pointers flag are provided on startup. Relates #13187, relates elastic/elasticsearch-definitive-guide#455

Info on compressed ordinary object pointers

jasontedor · 2015-12-16T22:04:58Z

Thanks for reviewing @nik9000, @rmuir, and @polyfractal.

This commit backports commits cfc46da and 12e241f from master to 2.x. Relates #15489

kimchy · 2015-12-16T23:04:14Z

core/src/main/java/org/elasticsearch/env/NodeEnvironment.java

+        ByteSizeValue maxHeapSize = JvmInfo.jvmInfo().getMem().getHeapMax();
+        Boolean usingCompressedOops = JvmInfo.jvmInfo().usingCompressedOops();
+        String usingCompressedOopsStatus = usingCompressedOops == null ? "unknown" : Boolean.toString(usingCompressedOops);
+        logger.info("heap size [{}], compressed ordinary object pointers [{}]", maxHeapSize, usingCompressedOopsStatus);


Sorry I am late, but I think this should go under debug logging, as by default, this will be logged each time a node starts, and I don't think it is important enough to log each time a node starts...

mmm, maybe heap size I guess? was that the aim here?

Sorry I am late, but I think this should go under debug logging, as by default, this will be logged each time a node starts, and I don't think it is important enough to log each time a node starts...

@kimchy We log mount points, disk space metrics, whether or not the disks are maybe spinning and the underlying file system types at the info level; I opted for consistency with that.

jasontedor added >enhancement review v5.0.0-alpha1 labels Dec 16, 2015

nik9000 reviewed Dec 16, 2015
View reviewed changes

jasontedor mentioned this pull request Dec 16, 2015

Modify wording relating to HotSpot JVM compressed oops elastic/elasticsearch-definitive-guide#455

Merged

jasontedor added a commit that referenced this pull request Dec 16, 2015

Merge pull request #15489 from jasontedor/compressed-oops

cfc46da

Info on compressed ordinary object pointers

jasontedor merged commit cfc46da into elastic:master Dec 16, 2015

jasontedor deleted the compressed-oops branch December 16, 2015 22:00

jasontedor added a commit that referenced this pull request Dec 16, 2015

Info on compressed ordinary object pointers

175e1b5

This commit backports commits cfc46da and 12e241f from master to 2.x. Relates #15489

kimchy reviewed Dec 16, 2015
View reviewed changes

jasontedor mentioned this pull request Dec 17, 2015

Simpler using compressed oops flag representation #15509

Merged

jasontedor added v2.2.0 and removed review labels Dec 17, 2015

rmuir mentioned this pull request Dec 26, 2015

Add new bundled signatures for disallowing internal runtime APIs + fix class loading order bugs policeman-tools/forbidden-apis#91

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Info on compressed ordinary object pointers #15489

Info on compressed ordinary object pointers #15489

jasontedor commented Dec 16, 2015

jasontedor commented Dec 16, 2015

polyfractal commented Dec 16, 2015

nik9000 Dec 16, 2015

jasontedor Dec 16, 2015

nik9000 Dec 16, 2015

jasontedor Dec 16, 2015

jasontedor commented Dec 16, 2015

polyfractal commented Dec 16, 2015

rmuir commented Dec 16, 2015

jasontedor commented Dec 16, 2015

nik9000 commented Dec 16, 2015

rmuir commented Dec 16, 2015

jasontedor commented Dec 16, 2015

rmuir commented Dec 16, 2015

jasontedor commented Dec 16, 2015

kimchy Dec 16, 2015

kimchy Dec 16, 2015

jasontedor Dec 16, 2015

Info on compressed ordinary object pointers #15489

Info on compressed ordinary object pointers #15489

Conversation

jasontedor commented Dec 16, 2015

jasontedor commented Dec 16, 2015

polyfractal commented Dec 16, 2015

nik9000 Dec 16, 2015

Choose a reason for hiding this comment

jasontedor Dec 16, 2015

Choose a reason for hiding this comment

nik9000 Dec 16, 2015

Choose a reason for hiding this comment

jasontedor Dec 16, 2015

Choose a reason for hiding this comment

jasontedor commented Dec 16, 2015

polyfractal commented Dec 16, 2015

rmuir commented Dec 16, 2015

jasontedor commented Dec 16, 2015

nik9000 commented Dec 16, 2015

rmuir commented Dec 16, 2015

jasontedor commented Dec 16, 2015

rmuir commented Dec 16, 2015

jasontedor commented Dec 16, 2015

kimchy Dec 16, 2015

Choose a reason for hiding this comment

kimchy Dec 16, 2015

Choose a reason for hiding this comment

jasontedor Dec 16, 2015

Choose a reason for hiding this comment