Integrate GeoHexGridAggregation with vector tiles API

docs/changelog/84553.yaml

@@ -0,0 +1,5 @@
+pr: 84553
+summary: Add `geohex_grid` aggregation to vector tiles API
+area: Geo
+type: enhancement
+issues: []

docs/reference/search/search-vector-tile-api.asciidoc

@@ -91,17 +91,19 @@ Internally, {es} translates a vector tile search API request into a
 * A <<query-dsl-geo-bounding-box-query,`geo_bounding_box`>> query on the
 `<field>`. The query uses the `<zoom>/<x>/<y>` tile as a bounding box.
 
-* A <<search-aggregations-bucket-geotilegrid-aggregation,`geotile_grid`>>
-aggregation on the `<field>`. The aggregation uses the `<zoom>/<x>/<y>` tile as
-a bounding box.
+* A <<search-aggregations-bucket-geotilegrid-aggregation,`geotile_grid`>> or
+<<search-aggregations-bucket-geohexgrid-aggregation,`geohex_grid`>> aggregation
+on the `<field>`. The `grid_agg` parameter determines the aggregation type. The
+aggregation uses the `<zoom>/<x>/<y>` tile as a bounding box.
 
 * Optionally, a
 <<search-aggregations-metrics-geobounds-aggregation,`geo_bounds`>> aggregation
 on the `<field>`. The search only includes this aggregation if the
 `exact_bounds` parameter is `true`.
 
-For example, {es} may translate a vector tile search API request with an
-`exact_bounds` argument of `true` into the following search:
+For example, {es} may translate a vector tile search API request with a
+`grid_agg` argument of `geotile` and an `exact_bounds` argument of `true`
+into the following search:
 
 [source,console]
 ----
@@ -159,21 +161,21 @@ Protobufs (PBF)]. By default, the tile contains three layers:
 * A `hits` layer containing a feature for each `<field>` value matching the
 `geo_bounding_box` query.
 
-*  An `aggs` layer containing a feature for each cell of the `geotile_grid`. You
-can use these cells as tiles for lower zoom levels. The layer only contains
-features for cells with matching data.
+*  An `aggs` layer containing a feature for each cell of the `geotile_grid` or
+`geohex_grid`. The layer only contains features for cells with matching data.
 
 * A `meta` layer containing:
 ** A feature containing a bounding box. By default, this is the bounding box of
 the tile.
-** Value ranges for any sub-aggregations on the `geotile_grid`.
+** Value ranges for any sub-aggregations on the `geotile_grid` or `geohex_grid`.
 ** Metadata for the search.
 
 The API only returns features that can display at its zoom level. For example,
 if a polygon feature has no area at its zoom level, the API omits it.
 
 The API returns errors as UTF-8 encoded JSON.
 
+[role="child_attributes"]
 [[search-vector-tile-api-query-params]]
 ==== {api-query-parms-title}
 
@@ -200,37 +202,89 @@ larger than the vector tile.
 square with equal sides. Defaults to `4096`.
 // end::extent-param[]
 
+// tag::grid-agg[]
+`grid_agg`::
+(Optional, string) Aggregation used to create a grid for the `<field>`.
++
+.Valid values for `grid_agg`
+[%collapsible%open]
+====
+`geotile` (Default)::
+<<search-aggregations-bucket-geotilegrid-aggregation,`geotile_grid`>>
+aggregation.
+
+`geohex`::
+<<search-aggregations-bucket-geohexgrid-aggregation,`geohex_grid`>> aggregation.
+If you specify this value, the `<field>` must be a <<geo-point,`geo_point`>>
+field.
+====
+// end::grid-agg[]
+
 // tag::grid-precision[]
 `grid_precision`::
-(Optional, integer) Additional zoom levels available through the `aggs` layer.
-For example, if `<zoom>` is `7` and `grid_precision` is `8`, you can zoom in up to
-level 15. Accepts `0`-`8`. Defaults to `8`. If `0`, results don't include the
-`aggs` layer.
-+
-This value determines the grid size of the `geotile_grid` as follows:
+(Optional, integer) Precision level for cells in the `grid_agg`. Accepts
+`0`-`8`. Defaults to `8`. If `0`, results don't include the `aggs` layer.
 +
+.Grid precision for `geotile`
+[%collapsible%open]
+====
+For a `grid_agg` of `geotile`, you can use cells in the `aggs` layer as tiles
+for lower zoom levels. `grid_precision` represents the additional zoom levels
+available through these cells. The final precision is computed by as
+follows:
+
+`<zoom> + grid_precision`
+
+For example, if `<zoom>` is `7` and `grid_precision` is `8`, then the
+`geotile_grid` aggregation will use a precision of `15`. The maximum final
+precision is `29`.
+
+The `grid_precision` also determines the number of cells for the grid as
+follows:
+
 `(2^grid_precision) x (2^grid_precision)`
-+
+
 For example, a value of `8` divides the tile into a grid of 256 x 256 cells. The
 `aggs` layer only contains features for cells with matching data.
+====
++
+.Grid precision for `geohex`
+[%collapsible%open]
+====
+For a `grid_agg` of `geohex`, `grid_precision` is used to calculate the
+https://h3geo.org/docs/core-library/restable[precision of the hexagonal cells].
+This is computed as follows:
+
+`(<zoom> + grid_precision - 1) / 2`
+
+The minimum final precision is `2`. The maximum final precision is `15`.
+
+Hexagonal cells don't align perfectly on a vector tile. Some cells may intersect
+more than one vector tile.
+====
 // end::grid-precision[]
 
 // tag::grid-type[]
 `grid_type`::
 (Optional, string) Determines the geometry type for features in the `aggs`
-layer. In the `aggs` layer, each feature represents a `geotile_grid` cell.
-Accepts:
-
-`grid` (Default):::
-Each feature is a `Polygon` of the cell's bounding box.
+layer. In the `aggs` layer, each feature represents a cell in the grid.
++
+.Valid values for `grid_type`
+[%collapsible%open]
+====
+`grid` (Default)::
+Each feature is a `Polygon` of the cell's geometry. For a `grid_agg` of
+`geotile`, the feature is the cell's bounding box. For a `grid_agg` of
+`geohex`, the feature is the hexagonal cell's boundaries.
 
-`point`:::
+`point`::
 Each feature is a `Point` that's the centroid of the cell.
 
-`centroid`:::
+`centroid`::
 Each feature is a `Point` that's the centroid of the data within the cell. For
 complex geometries, the actual centroid may be outside the cell. In these cases,
 the feature is set to the closest point to the centroid inside the cell.
+====
 // end::grid-type[]
 
 // tag::size[]
@@ -255,7 +309,7 @@ If `false`, the response does not include the total number of hits matching the
 
 `aggs`::
 (Optional, <<search-aggregations,aggregation object>>)
-<<run-sub-aggs,Sub-aggregations>> for the `geotile_grid`. Supports the following
+<<run-sub-aggs,Sub-aggregations>> for the `grid_agg`. Supports the following
 aggregation types:
 +
 * <<search-aggregations-metrics-avg-aggregation,`avg`>>
@@ -293,6 +347,8 @@ You can specify fields in the array as a string or object.
 include::search.asciidoc[tag=fields-param-props]
 ====
 
+include::search-vector-tile-api.asciidoc[tag=grid-agg]
+
 include::search-vector-tile-api.asciidoc[tag=grid-precision]
 
 include::search-vector-tile-api.asciidoc[tag=grid-type]
@@ -397,7 +453,7 @@ Field value. Only returned for fields in the `fields` parameter.
 ====
 
 `aggs`::
-(object) Layer containing results for the `geotile_grid` aggregation and its
+(object) Layer containing results for the `grid_agg` aggregation and its
 sub-aggregations.
 +
 .Properties of `aggs`
@@ -408,8 +464,7 @@ include::search-vector-tile-api.asciidoc[tag=extent]
 include::search-vector-tile-api.asciidoc[tag=version]
 
 `features`::
-(array of objects) Array of features. Contains a feature for each cell of the
-`geotile_grid`.
+(array of objects) Array of features. Contains a feature for each cell of the grid.
 +
 .Properties of `features` objects
 [%collapsible%open]
@@ -582,6 +637,7 @@ the `13/4207/2692` vector tile.
 ----
 GET museums/_mvt/location/13/4207/2692
 {
+  "grid_agg": "geotile",
   "grid_precision": 2,
   "fields": [
     "name",

x-pack/plugin/src/yamlRestTest/resources/rest-api-spec/test/vector-tile/10_basic.yml

@@ -88,6 +88,30 @@ setup:
         body:
           grid_type: point
 
+---
+"grid agg geotile":
+  - do:
+      search_mvt:
+        index: locations
+        field: location
+        x: 0
+        y: 0
+        zoom: 0
+        body:
+          grid_agg: geotile
+
+---
+"grid agg geohex":
+  - do:
+      search_mvt:
+        index: locations
+        field: location
+        x: 0
+        y: 0
+        zoom: 0
+        body:
+          grid_agg: geohex
+
 ---
 "grid type grid":
   - do:

x-pack/plugin/vector-tile/src/javaRestTest/java/org/elasticsearch/xpack/vectortile/VectorTileRestIT.java

@@ -317,10 +317,18 @@ public void testGridPrecision() throws Exception {
         }
     }
 
-    public void testGridType() throws Exception {
+    public void testGeoTileGrid() throws Exception {
+        doGridAggType(randomBoolean() ? "" : ", \"grid_agg\": \"geotile\"");
+    }
+
+    public void testGeoHexGrid() throws Exception {
+        doGridAggType(", \"grid_agg\": \"geohex\"");
+    }
+
+    private void doGridAggType(String gridAgg) throws Exception {
         {
             final Request mvtRequest = new Request(getHttpMethod(), INDEX_POINTS + "/_mvt/location/" + z + "/" + x + "/" + y);
-            mvtRequest.setJsonEntity("{\"size\" : 100, \"grid_type\": \"point\" }");
+            mvtRequest.setJsonEntity("{\"size\" : 100" + gridAgg + ",\"grid_type\": \"point\" }");
             final VectorTile.Tile tile = execute(mvtRequest);
             assertThat(tile.getLayersCount(), Matchers.equalTo(3));
             assertLayer(tile, HITS_LAYER, 4096, 33, 2);
@@ -330,7 +338,7 @@ public void testGridType() throws Exception {
         }
         {
             final Request mvtRequest = new Request(getHttpMethod(), INDEX_POINTS + "/_mvt/location/" + z + "/" + x + "/" + y);
-            mvtRequest.setJsonEntity("{\"size\" : 100, \"grid_type\": \"grid\" }");
+            mvtRequest.setJsonEntity("{\"size\" : 100" + gridAgg + ", \"grid_type\": \"grid\" }");
             final VectorTile.Tile tile = execute(mvtRequest);
             assertThat(tile.getLayersCount(), Matchers.equalTo(3));
             assertLayer(tile, HITS_LAYER, 4096, 33, 2);
@@ -340,7 +348,7 @@ public void testGridType() throws Exception {
         }
         {
             final Request mvtRequest = new Request(getHttpMethod(), INDEX_POINTS + "/_mvt/location/" + z + "/" + x + "/" + y);
-            mvtRequest.setJsonEntity("{\"size\" : 100, \"grid_type\": \"centroid\" }");
+            mvtRequest.setJsonEntity("{\"size\" : 100" + gridAgg + ", \"grid_type\": \"centroid\" }");
             final VectorTile.Tile tile = execute(mvtRequest);
             assertThat(tile.getLayersCount(), Matchers.equalTo(3));
             assertLayer(tile, HITS_LAYER, 4096, 33, 2);
@@ -354,6 +362,12 @@ public void testGridType() throws Exception {
             final ResponseException ex = expectThrows(ResponseException.class, () -> execute(mvtRequest));
             assertThat(ex.getResponse().getStatusLine().getStatusCode(), Matchers.equalTo(HttpStatus.SC_BAD_REQUEST));
         }
+        {
+            final Request mvtRequest = new Request(getHttpMethod(), INDEX_POINTS + "/_mvt/location/" + z + "/" + x + "/" + y);
+            mvtRequest.setJsonEntity("{\"grid_agg\": \"invalid_agg\" }");
+            final ResponseException ex = expectThrows(ResponseException.class, () -> execute(mvtRequest));
+            assertThat(ex.getResponse().getStatusLine().getStatusCode(), Matchers.equalTo(HttpStatus.SC_BAD_REQUEST));
+        }
     }
 
     public void testInvalidAggName() {

x-pack/plugin/vector-tile/src/main/java/org/elasticsearch/xpack/vectortile/feature/FeatureFactory.java

@@ -13,6 +13,8 @@
 import com.wdtinc.mapbox_vector_tile.adapt.jts.UserDataIgnoreConverter;
 import com.wdtinc.mapbox_vector_tile.build.MvtLayerProps;
 
+import org.elasticsearch.common.geo.GeoPoint;
+import org.elasticsearch.common.geo.SimpleFeatureFactory;
 import org.elasticsearch.common.geo.SphericalMercatorUtils;
 import org.elasticsearch.geometry.Circle;
 import org.elasticsearch.geometry.Geometry;
@@ -37,6 +39,7 @@
 import org.locationtech.jts.geom.TopologyException;
 import org.locationtech.jts.simplify.TopologyPreservingSimplifier;
 
+import java.io.IOException;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Comparator;
@@ -56,6 +59,8 @@ public class FeatureFactory {
     private final CoordinateSequenceFilter sequenceFilter;
     // pixel precision of the tile in the mercator projection.
     private final double pixelPrecision;
+    // optimization for points and rectangles
+    private final SimpleFeatureFactory simpleFeatureFactory;
     // size of the buffer in pixels for the clip envelope. we choose a values that makes sure
     // we have values outside the tile for polygon crossing the tile so the outline of the
     // tile is not part of the final result.
@@ -73,8 +78,33 @@ public FeatureFactory(int z, int x, int y, int extent) {
         this.builder = new JTSGeometryBuilder(geomFactory);
         this.clipTile = geomFactory.toGeometry(clipEnvelope);
         this.sequenceFilter = new MvtCoordinateSequenceFilter(tileEnvelope, extent);
+        this.simpleFeatureFactory = new SimpleFeatureFactory(z, x, y, extent);
     }
 
+    /**
+     * Returns a {@code byte[]} containing the mvt representation of the provided point
+     */
+    public byte[] point(double lon, double lat) throws IOException {
+        return simpleFeatureFactory.point(lon, lat);
+    }
+
+    /**
+     * Returns a {@code byte[]} containing the mvt representation of the provided rectangle
+     */
+    public byte[] box(double minLon, double maxLon, double minLat, double maxLat) throws IOException {
+        return simpleFeatureFactory.box(minLon, maxLon, minLat, maxLat);
+    }
+
+    /**
+     * Returns a {@code byte[]} containing the mvt representation of the provided points
+     */
+    public byte[] points(List<GeoPoint> multiPoint) {
+        return simpleFeatureFactory.points(multiPoint);
+    }
+
+    /**
+     * Returns a List {@code byte[]} containing the mvt representation of the provided geometry
+     */
     public List<byte[]> getFeatures(Geometry geometry) {
         // Get geometry in spherical mercator
         final org.locationtech.jts.geom.Geometry jtsGeometry = geometry.visit(builder);