[agg] Use timestamp (not start aligned) for expiring forward versions #3922
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The forward_writer uses a version map indexed by timestamp, which is not start aligned. When the flusher passes the set of times to expire, it needs to convert the start aligned time to a timestamp. This manifests as a significant memory leak for sparse workloads where the timestamp of N != start aligned time of N+1. In those cases the versions are never GCd when they expire out of the buffer.
rallen090
approved these changes
Nov 16, 2021
rallen090
reviewed
Nov 16, 2021
| @@ -3275,5 +3286,9 @@ func verifyOnForwardedFlushResult(t *testing.T, expected, actual []testOnForward | |||
| require.Equal(t, len(expected), len(actual)) | |||
| for i := 0; i < len(expected); i++ { | |||
| require.True(t, expected[i].aggregationKey.Equal(actual[i].aggregationKey)) | |||
| require.Equal(t, len(expected[i].expiredTimes), len(actual[i].expiredTimes)) | |||
There was a problem hiding this comment.
and forgot to mention, this assertion would have exposed the bug (tests failed without the code change).
Codecov Report
@@ Coverage Diff @@
## master #3922 +/- ##
========================================
- Coverage 56.7% 56.7% -0.1%
========================================
Files 555 555
Lines 63347 63347
========================================
- Hits 35931 35919 -12
- Misses 24218 24232 +14
+ Partials 3198 3196 -2
Flags with carried forward coverage won't be shown. Click here to find out more. Continue to review full report at Codecov.
|
soundvibe
added a commit
that referenced
this pull request
Nov 16, 2021
* master: [agg] Use timestamp (not start aligned) for expiring forward versions (#3922) [tests] Add support for calls to label APIs in resources.Coordinator (#3916) [tests] Convert repair_and_replication Docker Integration Test to In-process (#3903) Always Close the conn if failed to write acks (#3855) [m3msg] Add receive and handle latency to consumers (#3920)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The forward_writer uses a version map indexed by timestamp, which is not
start aligned. When the flusher passes the set of times to expire, it
needs to convert the start aligned time to a timestamp.
This manifests as a significant memory leak for sparse workloads where
the timestamp of N != start aligned time of N+1. In those cases the
versions are never GCd when they expire out of the buffer.
What this PR does / why we need it:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing and/or backwards incompatible change?:
Does this PR require updating code package or user-facing documentation?: