Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect rate agg with custom _doc_count #77734

Closed
sorenlouv opened this issue Sep 14, 2021 · 2 comments · Fixed by #79346
Closed

Incorrect rate agg with custom _doc_count #77734

sorenlouv opened this issue Sep 14, 2021 · 2 comments · Fixed by #79346
Assignees
Labels
:Analytics/Aggregations Aggregations >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)

Comments

@sorenlouv
Copy link
Member

sorenlouv commented Sep 14, 2021

Versions affected: 7.15 and up

I'm seeing some odd results with the rate agg on documents that have a custom _doc_count and wondering if it's a bug, similar to #74617.

Reproduce problem

PUT rate-bug

POST rate-bug/_doc
{
  "_doc_count": 10,
  "@timestamp": "2021-09-14T22:33:37.477Z"
}

GET rate-bug/_search
{
  "size": 0,
  "aggs": {
    "timeseries": {
      "date_histogram": {
        "field": "@timestamp",
        "fixed_interval": "60s"
      },
      "aggs": {
        "throughput": {
          "rate": {
            "unit": "minute"
          }
        }
      }
    }
  }
}

Response
The search request will return:

{
  "aggregations" : {
    "timeseries" : {
      "buckets" : [
        {
          "key_as_string" : "2021-09-14T22:33:00.000Z",
          "key" : 1631658780000,
          "doc_count" : 10,
          "throughput" : {
            "value" : 1.0
          }
        }
      ]
    }
  }
}

Expectation
I'd expect that a bucket for 1 minute with 10 documents should have a rate of 10.

Actual
Rate is 1.0

@sorenlouv sorenlouv added >bug needs:triage Requires assignment of a team area label labels Sep 14, 2021
@iverase iverase added :Analytics/Aggregations Aggregations and removed needs:triage Requires assignment of a team area label labels Sep 15, 2021
@elasticmachine elasticmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Sep 15, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-analytics-geo (Team:Analytics)

@jpountz jpountz changed the title Incorrect rate agg when used on histogram fields Incorrect rate agg with custom _doc_count Sep 15, 2021
@jpountz
Copy link
Contributor

jpountz commented Sep 15, 2021

As per a discussion with @sqren, I updated the title and description to remove references to the histogram field since this bug can be reproduced with the rate agg and documents that have custom _doc_counts only.

@jimczi jimczi removed the blocker label Sep 16, 2021
csoulios added a commit that referenced this issue Oct 19, 2021
When running a rate aggregation without setting the field parameter, the result is computed based on the bucket doc_count.

This PR adds support for a custom _doc_count field.

Closes #77734
csoulios added a commit that referenced this issue Oct 19, 2021
Backports #79346 to 7.x

    When running a rate aggregation without setting the field parameter, the result is computed based on the bucket doc_count.

    This PR adds support for a custom _doc_count field.

    Closes #77734
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/Aggregations Aggregations >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants