[Alerting] Add "Group By" support to ES Query alert type #89481
Comments
ymao1
added
Feature:Alerting
Team:ResponseOps
|
Pinging @elastic/kibana-alerting-services (Team:Alerting Services) |
|
For index threshold, we do do grouping via a top-N terms aggregation. Would we do the same here? IIRC, it's fairly a fairly straight-forward implementation, and seems pretty simple to understand for customers. Not sure if we'd need to do more than that though. Opening up the possibility of other bucket aggregations seems super-interesting, and kind of in line with this alert's "lower-level access" to ES bits. We'd need to discuss what the instance id's would be - if it's a top-N terms agg, it's simple - just the term, for other aggs we'd need to check to see how useful the grouping keys would be. |
|
I think a top-N terms aggregation like the Index Threshold would make the most sense since the simple ES query implementation is comparing hit count to a threshold condition. We could extend that to comparing the count of each bucket to a threshold condition. We could review other aggregation types to see if they would fit within this paradigm. We should also think about what "hits" we'd be returning if an aggregation is selected. Would we want to return the top N hits within each bucket as well? |
|
+1 |
gmmorris
added
the
Feature:Alerting/RuleTypes
gmmorris
added
enhancement
ersin-erdal
moved this from Todo
to In Progress
in AppEx: ResponseOps - Execution & Connectors
ersin-erdal
moved this from In Progress
to Todo
in AppEx: ResponseOps - Execution & Connectors
ymao1
moved this from Todo
to In Progress
in AppEx: ResponseOps - Execution & Connectors
ymao1
moved this from In Progress
to In Review
in AppEx: ResponseOps - Execution & Connectors
#144689) Resolves #89481 ## Summary Adds group by options to the ES query rule type, both DSL and KQL options. This is the same limited group by options that are offered in the index threshold rule type so I used the same UI components and rule parameter names. I moved some aggregation building code to `common` so they could be reused. All existing ES query rules are migrated to be `count over all` rules. ## To Verify * Create the following types of rules and verify they work as expected. Verify for both DSL query and KQL query * `count over all` rule - this should run the same as before, where it counts the number of documents that matches the query and applies the threshold condition to that value. `{{context.hits}}` is all the documents that match the query if the threshold condition is met. * `<metric> over all` rule - this calculates the specific aggregation metric and applies the threshold condition to the aggregated metric (for example, `avg event.duration`). `{{context.hits}}` is all the documents that match the query if the threshold condition is met. * `count over top N terms` - this will apply a term aggregation to the query and matches the threshold condition to each term bucket (for example, `count over top 10 event.action` will apply the threshold condition to the count of documents within each `event.action` bucket). `{{context.hits}}` is the result of the top hits aggregation within each term bucket if the threshold condition is met for that bucket. * `<metric> over top N terms` - this will apply a term aggregation and a metric sub-aggregation to the query and matches the threshold condition to the metric value within each term bucket (for example, `avg event.duration over top 10 event.action` will apply the threshold condition to the average value of `event.duration` within each `event.action` bucket). `{{context.hits}}` is the result of the top hits aggregation within each term bucket if the threshold condition is met for that bucket. * Verify the migration by creating a DSL and KQL query in an older version of Kibana and then upgrading to this PR. The rules should still continue running successfully. ### Checklist - [x] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios Co-authored-by: Kibana Machine <[email protected]> Co-authored-by: Lisa Cawley <[email protected]>
This PR introduced a basic ES query alert type that allows users to specify a query and a threshold condition for the number of matches against that query. We would like to enhance this alert type by adding the ability to group by a field within the index and then threshold against the hits within each group.
The text was updated successfully, but these errors were encountered: