Scatter Plots

[tsasser]

We need to be able to create dot plots. One important troubleshooting tool is to plot response time over time and see every event for the period in question.

[rashidkpc]

Plotting every event is really not feasible in datasets with millions of events. We may be able to implement something like a scatter plot later down the line, but right now elasticsearch has no functionality that would support this type of plotting.

[ruckc]

Wouldn't a box plot be appropriate for this, and can't elasticsearch
provide max/min/mean/stddev numbers using the statistical facet? Its not
every single data point, but its made to aggregate that type of data into a
easy to read chart.

Curtis Ruck
Anytime: 210-857-1126

On Mon, Feb 24, 2014 at 12:12 PM, Rashid Khan [email protected]:

Plotting every event is really not feasible in datasets with millions of
events. We may be able to implement something like a scatter plot later
down the line, but right now elasticsearch has no functionality that would
support this type of plotting.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/978#issuecomment-35909658
.

[rashidkpc]

The issue here is so short, and the description so vague its hard to know if tsasser is asking for a literal dot plot, which is not useful for more than a couple dozen data points. It sounds more like tsasser asking for a scatter plot, given the requirement to plot every event, in which case a heatmap would make a better representation.

[ecbrodie]

It is interesting you say that there is no support for scatter plots because the ElasticSearch page on Kibana (http://www.elasticsearch.org/overview/kibana/) shows an example of such a graph.

[rashidkpc]

@ecbrodie Technically thats just a high granularity line chart, with lines turned off and points turned on, it is not plotting every single point

[ecbrodie]

I understand the difference now, it is a special type of histogram. What I'm wondering now is whether there exists a way to plot every single point on a graph and have the x-axis set to @timestamp or equivalent time-based field, which I guess is not going to be planned as a feature. However, such a graph could prove to be useful for detecting outliers in the dataset, even when viewing more than a dozen data-points like you said above. Real-life example of the usefulness of outliers would be for detecting high latencies in metered client-server communication time for a particular device type.

[tsasser]

ecbrodie explained the idea.

This does work for millions of data points; I do it in gnuplot all the time. However, that means we have to have a separate tool that performs the query and Kibana should be able to do it.

I made a support request that includes a graph for you to look at.

[rashidkpc]

I'm going to close this as scatterplots simply are not feasible with the way Elasticsearch and Kibana work. The better solution for this would be heat maps as mentioned in the issue above.

[AlexIoannides]

Technically thats just a high granularity line chart, with lines turned off and points turned on, it is not plotting every single point

@rashidkpc Could the ability to add/remove points and lines from line charts be added to Kibana 4 as well?

[bcavagnolo]

@rashidkpc: You mentioned that the scatter plot on the Kibana page is "just a high granularity line chart, with lines turned off and points turned on." Is there away to achieve this using Kibana 4? I can't find any UI for tuning this.

[rmoff]

@bcavagnolo click Options: