Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ML] Support Composite aggregations in the datafeed #37757

Closed
davidkyle opened this issue Jan 23, 2019 · 2 comments
Closed

[ML] Support Composite aggregations in the datafeed #37757

davidkyle opened this issue Jan 23, 2019 · 2 comments
Assignees
Labels
>enhancement :ml Machine learning

Comments

@davidkyle
Copy link
Member

Composite aggregations provide a way to stream all buckets similar to scroll search. For Terms aggs this means high cardinality fields can be used in the datafeed without some of those terms being lost due to the limitation the size parameter.

@davidkyle davidkyle added >enhancement :ml Machine learning labels Jan 23, 2019
@benwtrent benwtrent self-assigned this Mar 2, 2021
@benwtrent
Copy link
Member

For supporting composite aggs, I am working on it with the following restrictions:

  • The composite agg MUST be the top level agg
  • Only ONE composite agg is allowed
  • The composite agg must contain exactly ONE date_histogram data source

The other aggregation restrictions of requiring a max agg on the timstamp field remains.

The other bucketing sources of the composite agg can be used as fields in the detectors. Meaning having a terms group by source, the terms field could be a partition field or influencer.

I am not 100% about supporting the geotile_grid data source for composite aggs. We can allow it, but there won't be any special handling yet. So the values would just be written out as x/y/zoom strings (just like the bucket key value).

@benwtrent
Copy link
Member

Composite aggs for datafeeds has been added as experimental #69970

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :ml Machine learning
Projects
None yet
Development

No branches or pull requests

2 participants