High settings of max zoom make heat map not accurately reflect the data

[thomasneirynck]

the heat map visualization type in Kibana does not really show what we would expect. It should bring out areas of high incidence.

Consider:

Judging from this image we would expect the densest cluster of data to be at the Ohio-Indiana border.

However, the peakiest data is in the Philadelphia area.

This is because we are creating a heatmap based on aggregation data (at the geohash-cell), but not on the raw incident data. This causes us to overweigh the clump of geohashes in Indiana, but ignore the peaky incidence in Philadelphia.

Ironically, the 'Shaded Geohash Grid' map best approximates a "heat map", albeit a coarsely grained one.

There is no "quick" fix for this. Heat maps should be generated from raw incidence-data. Generating these from a derivative causes unintuitive results.

[trevan]

Change the "maximum zoom" value for the heat map settings to be equal to the actual zoom you are looking at. I think that will make it look as you expect. I've patched my instance to always do that and the heatmap looks a lot better.

[spalger]

Interesting @trevan, seems to produce better results here

[thomasneirynck]

thanks @trevan, that works indeed much better!

Not sure yet why we have that setting in there right now. My initial gut reaction is that maybe we should consider removing it.

[thomasneirynck]

Looked into this further.

The max-zoom is a Leaflet setting (https://github.com/Leaflet/Leaflet.heat). It describes at which zoom-level the points should have maximum intensity (ie. more red)

This setting makes sense where each point on the map is a single datapoint. This setting avoids that everything would look like a giant rd ed blob when zoomed out because all points are very close together then.

We do not have that problem in Kibana. Kibana does not map individual data points. It maps aggregations. It recalculates this aggregation, each time the geohash-precision level changes. A "geohash-precision" roughly corresponds to 2-3 zoom-levels. (ES can only aggregate on zoom level precisions)

Right now, Kibana creates a new L.HeatLayer, each time the precision changes. (at first glance, there may actually be a bug. Kibana seems to unnecessarily recreate this at each zoom-level change, which is too often).

I propose we:

• deprecate the maxZoom parameter
• have Kibana automatically set this parameter when creating a new heatlayer. It could set it to '0'. It could also set it to the lowest zoom level that correspond to the precision. (the effect would be the same).

[thomasneirynck]

we deprecated maxZoom parameter in 6.0