[coordinator] Only honor default aggregation policies if not matched by mapping rule #2203
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…atch-configured-mapping-rules
martin-mao
approved these changes
Mar 11, 2020
Codecov Report
@@ Coverage Diff @@
## master #2203 +/- ##
======================================
Coverage 71.5% 71.5%
======================================
Files 1022 1022
Lines 88961 88961
======================================
Hits 63693 63693
Misses 20908 20908
Partials 4360 4360
Continue to review full report at Codecov.
|
arnikola
reviewed
Mar 11, 2020
| // NB(r): First apply mapping rules to see which storage policies | ||
| // have been applied, any that have been applied as part of | ||
| // mapping rules that exact match a default storage policy will | ||
| // skipped when applying default rules so to avoid storing |
There was a problem hiding this comment.
nit; be skipped, rules, so as
arnikola
reviewed
Mar 11, 2020
| debugLogMatchOptions{Meta: stagedMetadatas}) | ||
|
|
||
| // Only sample if going to actually aggregate | ||
| stagedMetadatsBeforeFilter := stagedMetadatas[:] |
There was a problem hiding this comment.
nit: stagedMetadataBeforeFilter
arnikola
reviewed
Mar 11, 2020
| for _, existing := range a.mappingRuleStoragePolicies { | ||
| if sp.Equivalent(existing) { | ||
| matchedByMappingRule = true | ||
| a.debugLogMatch("downsampler skipping default mapping rule storage policy", |
There was a problem hiding this comment.
Should this also log exactly which rule was skipped?
There was a problem hiding this comment.
Might get confusing since it could be a subsection of it - thankfully we print that out just as we start matching it (so its contextually close in the logs)
a.debugLogMatch("downsampler applying default mapping rule",
debugLogMatchOptions{Meta: stagedMetadatas})
…atch-configured-mapping-rules
…atch-configured-mapping-rules
notbdu
deleted the
r/skip-applying-default-mapping-rules-if-match-configured-mapping-rules
branch
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
After seeing mapping rules in use, it's become obvious that default aggregations should not honored if a matching mapping rule already specifies an aggregation for the same storage policy (retention + resolution). Letting both apply will cause race conditions on which aggregated value makes it to storage with the same metric name and tags.
This keeps the current behavior as is however will remove any applicable default aggregations for aggregated namespaces defined if a mapping rule has already been applied for the given storage policy (retention + resolution).
Special notes for your reviewer:
Does this PR introduce a user-facing and/or backwards incompatible change?:
Does this PR require updating code package or user-facing documentation?: