Count aggregation should show zero's in line chart

[spalger]

We have talked about this a few times, but I don't think we've every agreed to do something different for the count metric agg. I think we should.

Consider this vis:

I seem to recall that we decided not to zero-fill line charts because they are designed to show the path values take, and null/undefined is not the same thing as zero. That's not true when it comes to the "count" metric though. Unless we specify a "min_doc_count": 0 on bucket aggregations we don't get the bucket count for buckets that don't exist because they would be empty, meaning it's impossible to get a count of zero, which is a totally legitimate thing.

This is actually what that chart should look like:

[pybern]

Hi! Totally new to to the community, is there currently a workaround to show null at 0?

[spalger]

Welcome @pybern! Did you try setting the { "min_doc_count": 0 } JSON options?

[pybern]

Thanks for the warm welcome @spalger! I did. On the wrong bucket. It is working now that I have added the setting to the correct bucket. Thanks btw! Hope to learn more and contribute when I can =D

[ivanandrianto95]

The problem is min_doc_count works for count, but not for average

[Bargs]

@ivanandrianto95 can you provide an example, and what you expect it to look like?

[ivanandrianto95]

When{ "min_doc_count": 0 } is specified, there is a difference when we select count and average of a field in Y-Axis. If we choose to display count, it will look like the 2nd picture above. The chart goes down to zero if there is no value at the time. But when we choose to display average of a field and { "min_doc_count": 0 } is specified, the chart line will be broken (instead of down to zero) if there is no value at the time. For example with 5 minutes interval and following values
Time Average
09.00 -
09.05 2
09.10 -
09.15 3
The chart will show a dot (not line) at 09.05 and 09.15

[Bargs]

@ivanandrianto95 Based on my read of @spalger's original post, that's the intended behavior of the average metric. He said:

we decided not to zero-fill line charts because they are designed to show the path values take, and null/undefined is not the same thing as zero

Showing missing values as 0 in the context of the average metric could be misleading, because it implies that actual values exist and their average is 0.

If you want this behavior though, I think you might be able to accomplish it with the "missing" parameter: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html#_missing_value_12

Set both min_doc_count and missing to 0.

[ppisljar]

what should the solution for this one be ? if i understand correctly everything can be achieved with additional json input. should { "min_doc_count": 0 } be in command by default if count metrics is selected ?

[Bargs]

@ppisljar my initial thought was to add min_doc_count: 0 automatically to visualizations with count metrics. But that might cause issues for visualizations with say both a count and average metric. This might need to be done at the vis level, so that only count metrics get zero filled. This would help folks who want to have a visualization with mixed metrics, since the min_doc_count workaround wouldn't help them at the moment.

[thomasneirynck]

The default min_doc_count value changed to 0 in Elasticsearch 2.0 (https://www.elastic.co/guide/en/elasticsearch/reference/2.3/breaking-changes-2.0.html).

ES returns intuitively correct responses:

• count: empty buckets get a 0 value
• average, median, ...: empty buckets get a null value

Kibana handles these responses correctly. Which means that the visualizations look correct as well.

This 'could' be fixed for Kibana 4.1 (which uses pre 2.0 ES), but not sure if this issue is worth backporting. => closing.