-
Notifications
You must be signed in to change notification settings - Fork 455
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[m3ninx] Avoid cloning of roaring bitmap in conjunctionSearcher #3948
[m3ninx] Avoid cloning of roaring bitmap in conjunctionSearcher #3948
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3948 +/- ##
========================================
- Coverage 56.9% 56.7% -0.3%
========================================
Files 551 555 +4
Lines 63126 63406 +280
========================================
+ Hits 35947 35975 +28
- Misses 24003 24237 +234
- Partials 3176 3194 +18
Flags with carried forward coverage won't be shown. Click here to find out more. Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Indeed cloning looks necessary if we do at least one intersection/union/difference operation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM from the code perspective, but don't have deep understanding of this codebase.
What this PR does / why we need it:
conjunctionSearcher.Search
was always making a deep copy (Clone
) of the firstPostingsList
, even though it was not necessary when there was more than onePostingsList
in play. It was not necessary because bothIntersect
andDifference
construct a new roaring bitmap under the hood (there are no in-place implementations forIntersect
andDifference
).I have added a call to
CloneAsMutable
at the end ofconjunctionSearcher.Search
, it will only be executed if there were noIntersect
/Difference
calls made. I might have been too paranoid and could have avoided this becauseconjunctionSearcher.Search
returns immutableList
(so technically there would be no need to clone the input), but this all design does not fully embrace immutability (e.g. same object implements both mutable and immutable interfaces), which makes it very fragile, so I decided to err on the safe side.In practice, I expect this clone in the end to almost never happen. Because even for a very basic Prometheus query of the form
foo{bar="x"}
you already have an intersection of 2 search predicates, meaning that the cloning gets avoided.Note that the savings on heap size would not be very significant (I believe because those objects are mostly short lived):
But reduction of allocation/GC pressure in some use cases might be significant:
Special notes for your reviewer:
To make the immutable vs in-place operations more explicit, I have moved
Intersect
andDifference
fromMutableList
to the immutableList
interface and changed them to return the resulting value instead of doing a shallow mutation.Also, renamed
Union
/UnionMany
toUnionInPlace
/UnionManyInPlace
, to better reflect the semantics (and to match the names of the underlying bitmap methods used).Also, renamed
List.Clone
toList.CloneAsMutable
because it is returning aMutableList
, so that it is not a proper clone method.Does this PR introduce a user-facing and/or backwards incompatible change?:
NONE
Does this PR require updating code package or user-facing documentation?:
NONE