Java utilities to aid in accelerating aggregations on 128-bit types #10201
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jlowe
added
Java
jrhemstad
reviewed
Feb 2, 2022
Codecov Report
@@ Coverage Diff @@
## branch-22.04 #10201 +/- ##
================================================
+ Coverage 10.42% 10.47% +0.05%
================================================
Files 119 122 +3
Lines 20603 20501 -102
================================================
Hits 2148 2148
+ Misses 18455 18353 -102
Continue to review full report at Codecov.
|
jrhemstad
reviewed
Feb 3, 2022
revans2
reviewed
Feb 3, 2022
revans2
approved these changes
Feb 3, 2022
jlowe
added
5 - Ready to Merge
|
@gpucibot merge |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a couple of custom kernels for Java to help accelerate sum aggregations on 128-bit types and check for overflows. The first kernel extracts a 32-bit chunk from an 128-bit type which can be used to feed four 32-bit chunks into a sum aggregation. The second kernel takes the resulting upscaled 64-bit integer results and reassembles the parts into a 128-bit type column along with a boolean column to indicate whether the value overflowed.
By splitting the 128-bit type into 32-bit chunks, a sum aggregation on DECIMAL128 which is a sort-based aggregation can be turned into a hash-based aggregation on 32-bit integer inputs for improved performance. As a bonus, this approach can also check for overflow which is difficult to do when aggregating on DECIMAL128 sums directly.