Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Last 'other' bucket on terms agg broken when first bucket is split on a term containing a dash (-) #42467

Closed
lukeelmers opened this issue Aug 1, 2019 · 2 comments · Fixed by #81981
Assignees
Labels
bug Fixes for quality problems that affect the customer experience Feature:Aggregations Aggregation infrastructure (AggConfig, esaggs, ...) impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. loe:medium Medium Level of Effort Project:LensDefault

Comments

@lukeelmers
Copy link
Member

lukeelmers commented Aug 1, 2019

Kibana version: tested on master & 7.2

The terms_other_bucket_helper we use for dealing with "other" and "missing" buckets in terms aggregations has a weird edge case where splitting on terms containing a dash/hyphen (-) can cause the other bucket to be missed.

I suspect this is due to the fact that we are splitting on dashes in bucket keys as a way to manage nested buckets:

const getAggResultBuckets = (aggConfigs, response, aggWithOtherBucket, key) => {
const keyParts = key.split('-');
let responseAgg = response;
for (const i in keyParts) {

For data that is expected to contain dashes, this can cause "other" buckets to fail to display in visualizations.

Steps to reproduce:

  1. Using the dev console, ingest documents from each of the two data sets todos1 and todos2 (listed below)
  2. Create index patterns todos1* and index pattern todos2*
  3. Create the same data table visualization against each index pattern (though this is reproducible on other vis types too):
    • split rows with terms aggregation on status.keyword, size 5
    • split rows with terms aggregation on task.keyword, size 1, "group other values" enabled
  4. Compare the two and notice that the data from todos1 is missing other buckets due to the dashes contained in the status.keyword data. Everything from todos2 behaves as expected.
  5. You'll notice the same issue is visible in the inspector data, since the other buckets are simply missing from the tabified data.

Visualization: todos1 (broken)
Screenshot 2019-08-01 12 23 50

Visualization: todos2 (works)
Screenshot 2019-08-01 12 23 32

Sample data: todos1 (broken)

POST todos1/_doc
{
  "status": "to-do",
  "task": "A"
}

POST todos1/_doc
{
  "status": "to-do",
  "task": "B"
}

POST todos1/_doc
{
  "status": "to-do",
  "task": "C"
}

POST todos1/_doc
{
  "status": "to-do",
  "task": "C"
}

POST todos1/_doc
{
  "status": "in-progress",
  "task": "D"
}

POST todos1/_doc
{
  "status": "in-progress",
  "task": "E"
}

POST todos1/_doc
{
  "status": "in-progress",
  "task": "F"
}

POST todos1/_doc
{
  "status": "done",
  "task": "G"
}

Sample data: todos2 (works)

POST todos2/_doc
{
  "status": "todo",
  "task": "A"
}

POST todos2/_doc
{
  "status": "todo",
  "task": "B"
}

POST todos2/_doc
{
  "status": "todo",
  "task": "C"
}

POST todos2/_doc
{
  "status": "todo",
  "task": "C"
}

POST todos2/_doc
{
  "status": "inprogress",
  "task": "D"
}

POST todos2/_doc
{
  "status": "inprogress",
  "task": "E"
}

POST todos2/_doc
{
  "status": "inprogress",
  "task": "F"
}

POST todos2/_doc
{
  "status": "done",
  "task": "G"
}
@lukeelmers lukeelmers added bug Fixes for quality problems that affect the customer experience Feature:Aggregations Aggregation infrastructure (AggConfig, esaggs, ...) Team:AppArch labels Aug 1, 2019
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app-arch

@lukeelmers lukeelmers self-assigned this Jan 28, 2020
@lukeelmers lukeelmers added impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. loe:medium Medium Level of Effort triaged labels Jun 16, 2020
@F-PMOR
Copy link

F-PMOR commented Jun 19, 2020

Hi all, Do you know when this issue will be resolved ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Fixes for quality problems that affect the customer experience Feature:Aggregations Aggregation infrastructure (AggConfig, esaggs, ...) impact:low Addressing this issue will have a low level of impact on the quality/strength of our product. loe:medium Medium Level of Effort Project:LensDefault
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants