Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Lens] Better handling of "too many buckets" error #93998

Closed
wylieconlon opened this issue Mar 8, 2021 · 3 comments
Closed

[Lens] Better handling of "too many buckets" error #93998

wylieconlon opened this issue Mar 8, 2021 · 3 comments
Labels
enhancement New value added to drive a business result Feature:Lens impact:needs-assessment Product and/or Engineering needs to evaluate the impact of the change. Team:Visualizations Visualization editors, elastic-charts and infrastructure

Comments

@wylieconlon
Copy link
Contributor

This is an error that is hard to test using small datasets, it's only an issue with large and high-cardinality data. For this reason I haven't been able to reproduce it in a while, and I think we will need to generate special data to test this issue.

Users can already get unpredictable but frequent query failures due to the 65k search.max_buckets limit. For example, the query might fail on a large time range but succeed in a small time range because the large time range has more results. It will happen any time the query gets above 65k results, which could happen with a datatable:

  1. A histogram with 100 buckets (the default Kibana setting)
  2. A terms aggregation with 100 buckets
  3. A second terms aggregation with 100 buckets

When they get a failure, it looks like this:

Screen Shot 2021-03-08 at 3 39 03 PM

If this happens while using a dashboard, the user won't have any options to fix. I propose that we do two changes:

  1. When the user has multiple levels of aggregation that multiply together to cause too many buckets, we should warn them in the Lens editor even if the query will succeed. This will make users aware that there could be a problem.

  2. The Lens embeddable should offer inline controls to reduce the size of the Terms aggregation if the query fails. For example, we could offer to "try again with 10 top values" instead of 100.

This is related to #93912

@wylieconlon wylieconlon added enhancement New value added to drive a business result Team:Visualizations Visualization editors, elastic-charts and infrastructure Feature:Lens labels Mar 8, 2021
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app (Team:KibanaApp)

@wylieconlon
Copy link
Contributor Author

To test this I used metricbeat data and added 3 random-value scripted fields, using this script:

return BigDecimal.valueOf(Math.random())
    .setScale(1, RoundingMode.HALF_UP)
    .doubleValue();

@stratoula stratoula added the impact:needs-assessment Product and/or Engineering needs to evaluate the impact of the change. label Feb 15, 2023
@timductive
Copy link
Member

Closing this one as it is not a highly reported edge-case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New value added to drive a business result Feature:Lens impact:needs-assessment Product and/or Engineering needs to evaluate the impact of the change. Team:Visualizations Visualization editors, elastic-charts and infrastructure
Projects
None yet
Development

No branches or pull requests

4 participants