Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow collection of jvm gc section in node-stats telemetry device #644

Conversation

dliappis
Copy link
Contributor

We already collect the mem section under node-stats/jvm but GC stats
can also be very useful for understanding the jvm memory behavior during
a benchmark.

Add node-stats-include-gc telemetry parameter (enabled by default) and
add it to docs.

We already collect the mem section under node-stats/jvm but GC stats
can also be very useful for understanding the jvm memory behavior during
a benchmark.

Add `node-stats-include-gc` telemetry parameter (enabled by default) and
add it to docs.
Copy link
Member

@danielmitterdorfer danielmitterdorfer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder whether we really need to add a new flag for this or whether we should just start capturing it by default? Overall LGTM. Can you please also add the respective labels?

@dliappis
Copy link
Contributor Author

I tested this using

esrally --configuration-name=metricstore --runtime-jdk=8 --track=geonames --challenge=append-no-conflicts-index-only --telemetry=node-stats --telemetry-params="node-stats-include-thread-pools:false,node-stats-include-buffer-pools:false,node-stats-include-breakers:false,node-stats-include-mem:false,node-stats-include-gc:true,node-stats-include-network:false,node-stats-include-process:false" --pipeline=from-sources-complete

and an example node-stats doc looks like:

{
  "_index": "rally-metrics-2019-02",
  "_type": "metrics",
  "_id": "9TlU4WgBStDFtUU6amBA",
  "_version": 1,
  "_score": null,
  "_source": {
    "name": "node-stats",
    "jvm_gc_collectors_young_collection_count": 1457,
    "jvm_gc_collectors_young_collection_time_in_millis": 14798,
    "jvm_gc_collectors_old_collection_count": 81,
    "jvm_gc_collectors_old_collection_time_in_millis": 3323,
    "@timestamp": 1549968631332,
    "relative-time": 4259307,
    "trial-id": "81f7b3c4-f935-4ff5-bd1f-a2aa220f3cb9",
    "trial-timestamp": "20190212T104315Z",
    "environment": "local",
    "track": "geonames",
    "lap": 1,
    "challenge": "append-no-conflicts-index-only",
    "car": "defaults",
    "meta": {
      ...
}

@dliappis dliappis added enhancement Improves the status quo :Telemetry Telemetry Devices that gather additional metrics labels Feb 12, 2019
@dliappis dliappis added this to the 1.0.4 milestone Feb 12, 2019
@dliappis
Copy link
Contributor Author

I wonder whether we really need to add a new flag for this or whether we should just start capturing it by default?

It will be captured by default, but I thought that being able to control it via node-stats-include-gc as we can already do with node-stats-include-mem can be beneficial for users who want "less polluted/more targeted " benchmarks.

@danielmitterdorfer
Copy link
Member

Ok, then let's keep it as you've implemented it.

@dliappis dliappis merged commit 524b6fa into elastic:master Feb 12, 2019
@dliappis dliappis deleted the add-jvm-gc-stats-in-node-stats-telemetry-device branch February 12, 2019 11:05
@dliappis
Copy link
Contributor Author

Thanks for the quick review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Improves the status quo :Telemetry Telemetry Devices that gather additional metrics
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants