Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure stable mappings for metrics #1209

Merged

Conversation

danielmitterdorfer
Copy link
Member

With this commit we disable dynamic detection of dates for documents in
the metrics store. By disabling this feature, any dynamic values (like
track parameters or user tags) that look like dates are mapped as
keywords instead of as dates. These values should only be treated as
keywords and are not meant to be used e.g. in range queries.

Furthermore, this ensures that we don't cause trouble when date formats
conflict in such dynamic values. Note that properties that are
explicitly mapped as dates (e.g. @timestamp) are not affected by this
change and will still be mapped as dates.

With this commit we disable dynamic detection of dates for documents in
the metrics store. By disabling this feature, any dynamic values (like
track parameters or user tags) that look like tags are mapped as
keywords instead of as dates. These values should only be treated as
keywords and are not meant to be used e.g. in range queries.
Furthermore, this ensures that we don't cause trouble when date formats
conflict in such dynamic values. Note that properties that are
explicitly mapped as dates (e.g. `@timestamp`) are not affected by this
change and will still be mapped as dates.
@danielmitterdorfer danielmitterdorfer added bug Something's wrong :Metrics How metrics are stored, calculated or aggregated labels Mar 15, 2021
@danielmitterdorfer danielmitterdorfer added this to the 2.1.0 milestone Mar 15, 2021
@danielmitterdorfer danielmitterdorfer self-assigned this Mar 15, 2021
Copy link
Contributor

@dliappis dliappis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, yay for less unpredictability in data types.

One question:

For existing monthly indices that have already ingested, say, a track param as a date, will this change break things due to type conflicts?

@danielmitterdorfer
Copy link
Member Author

For existing monthly indices that have already ingested, say, a track param as a date, will this change break things due to type conflicts?

No, because this only changes the index template which applies only to newly created indices. So the change then applies only to the new index.

@dliappis dliappis self-requested a review March 18, 2021 12:09
Copy link
Contributor

@dliappis dliappis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@danielmitterdorfer danielmitterdorfer merged commit df80732 into elastic:master Mar 22, 2021
@danielmitterdorfer danielmitterdorfer deleted the disable-dynamic-dates branch March 22, 2021 06:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something's wrong :Metrics How metrics are stored, calculated or aggregated
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants