achieve unlimited bucket size aggregation

[jehutywong]

#27447
I am also facing the issue above, a limit on buckets number not seem to be an acceptable solution. Is there a way to achieve an unlimited bucket size aggregation, if i can accept a very long time running time?

[elasticmachine]

Pinging @elastic/es-analytics-geo

[polyfractal]

If you need all the buckets of an aggregation, you can use the composite aggregation. That allows "paging" over the results of an aggregation in a memory-friendly manner.

We can't allow unlimited buckets for non-composite aggregations because they have to store their working state in-memory...and "unlimited" is a very large number :) Composite agg works around this by breaking the agg up over multiple requests (one per "page") and having the client buffer the information client-side.

Also in the future, please open questions like this on the discuss forums. We try to keep the github issues for things like bug reports. Thanks!

[jehutywong]

Thanks for your advice @polyfractal, i have tried composite aggregation but facing this failure prompt:
“The first aggregation in buckets_path must be a multi-bucket aggregation”

I think my problem now is linked to this one #32692

[polyfractal]

Yeah, pipeline aggs don't work with composite right now. But do note that pipeline aggs are all operations that can be done client-side, so you could re-implement that functionality in your application (until we support pipelines + composite)