Es shard level metrics #1752
Merged
Es shard level metrics #1752
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
elafarge
force-pushed
the
etienne/es-shard-level-metrics
branch
3 times, most recently
from
d7cf065 to
29d99cf
Compare
| shard_role = "primary" | ||
| elif count_replicas: | ||
| replica_number += 1 | ||
| shard_name = 'R' + i_str + '_{0}'.format(replica_number) |
There was a problem hiding this comment.
you could write shard_name = 'R{0}_{1}'.format(i_str, replica_number) instead
elafarge
force-pushed
the
etienne/es-shard-level-metrics
branch
2 times, most recently
from
e6d0b3b to
a20a1dc
Compare
| # The "shard_level_metrics" flag enables metrics and service checks on a per- | ||
| # shard basis (all the information is fetched under the /_stats?level=shards | ||
| # endpoint). The metrics and service check sent for each shard are named as | ||
| # such: elasticsearch.shard.metric.name . |
There was a problem hiding this comment.
you could add a line about how the metrics are tagged, so that people know how to graph per shard.
There was a problem hiding this comment.
Isn't the one six lines below suitable ?
elafarge
force-pushed
the
etienne/es-shard-level-metrics
branch
from
a20a1dc to
7dfdecf
Compare
| @@ -297,3 +346,87 @@ def test_health_event(self): | |||
| tags=['host:localhost', 'port:9200'], | |||
| count=1 | |||
| ) | |||
|
|
|||
| def test_pshard_metrics(self): | |||
There was a problem hiding this comment.
Since you're testing for new metrics you should add a coverage_report at the end of this method.
There was a problem hiding this comment.
Very good remark indeed @hkaj .
|
@hkaj Test coverage report has been added to the new tests. |
elafarge
force-pushed
the
etienne/es-shard-level-metrics
branch
from
c6b7d99 to
13f580b
Compare
|
Awesome! You can rebase and 🚢 then. Congrats 🎉 |
added 3 commits
July 14, 2015 15:39
Added statistics over primary shards only to our check. They're basically retrieved under the `_stats` endpoint and are aggregated metrics on the primary shards. It means that they don't take into account metrics from replica shards. So for instance `primaries.docs.count` will contain the total number of documents in the cluster without replicas. Computing replica's would be as simple as summing the "standard" document count metric over all nodes in the cluster. A test for those new metrics in particular has been added.
This adds shard-level metrics to elasticsearch. The feature is disabled by default. A flag has to be set in elastic.yaml to enable it. It fetches, for every shard (primary or replica) of every index, a set of metrics restricted to those shards. It also create a `state` service check on a per-shard basis. Metrics and state are fetched from the `/_stats?level=shard` endpoint. A test has also been added. The metrics and service checks are sent with a bunch of new tags: `es_node:node_name`, `es_shard:shard_name`, `es_index:index_name`, `es_role:(primary|replica)` to make it easy to define a scope to monitor/graph in our backend. The `shard_specific` tag is also added to avoid confusion with other stats metrics, aggregated on elasticsearch's end.
Following Haissam's suggestion, I added a call to `AgentCheckTest.coverage_report()` at the end the new elastic tests since they involve testing the presence of new metrics.
elafarge
force-pushed
the
etienne/es-shard-level-metrics
branch
from
38a0707 to
0c7bbbb
Compare
|
Thanks, I just wait for Travis to run and I'll 🚢 it :) |
|
Looks like it's good to be shipped :) |
|
Can't wait for tomorrow to see how much want we can have with these new metrics on staging :) |
elafarge
pushed a commit
that referenced
this pull request
Aug 27, 2015
…metrics" This reverts commit b19f251, reversing changes made to 3e6c0b1. This reverts BOTH the addition of shard-level metrics to Elasticsearch as well as the addition of PShard-related statistics. Pshard statistics will be readded for the 5.5 release, shard level statistics will be discussed for 5.6.0 because right now, they obviously create way to much contexts in our backend.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add shard level metrics to Elasticsearch integration (Careful: branch has been created on top of etienne/es-pshard-metrics)