Support for using tdigests to compute approximate percentiles. #8983
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…e their usage when calling the groupby object.
… in python bindings.
nvdbaranec
added
feature request
nvdbaranec
requested review from
mythrocks,
davidwendt,
charlesbluca and
rgsl888prabhu
github-actions
bot
added
CMake
nvdbaranec
added
non-breaking
github-actions
bot
added
CMake
jrhemstad
reviewed
Sep 23, 2021
jrhemstad
reviewed
Sep 23, 2021
jrhemstad
reviewed
Sep 23, 2021
jrhemstad
reviewed
Sep 23, 2021
jrhemstad
reviewed
Sep 23, 2021
jrhemstad
reviewed
Sep 23, 2021
jrhemstad
reviewed
Sep 23, 2021
jrhemstad
approved these changes
Sep 23, 2021
jrhemstad
requested changes
Sep 23, 2021
mythrocks
reviewed
Sep 23, 2021
Updated. Null percentile == null output. |
mythrocks
reviewed
Sep 23, 2021
mythrocks
approved these changes
Sep 23, 2021
jrhemstad
approved these changes
Sep 24, 2021
robertmaynard
approved these changes
Sep 24, 2021
|
@gpucibot merge |
rapids-bot bot
pushed a commit
that referenced
this pull request
Sep 24, 2021
This PR builds on #8983 and adds Java bindings. Authors: - Andy Grove (https://github.com/andygrove) - https://github.com/nvdbaranec Approvers: - Robert (Bobby) Evans (https://github.com/revans2) URL: #9094
rapids-bot bot
pushed a commit
that referenced
this pull request
Oct 18, 2021
Addresses comments from initial PR (#8983). Specifically implementing a tdigest_column_view for more cleanly accessing the various sub-columns of a tdigest column. Includes several bounds checking fixes for empty groups. Addresses an issue where entirely empty digests could potentially lead to an incorrect min/max values, which isn't technically _wrong_ but makes constructing test cases tricky. Authors: - https://github.com/nvdbaranec Approvers: - AJ Schmidt (https://github.com/ajschmidt8) - Robert (Bobby) Evans (https://github.com/revans2) - Jake Hemstad (https://github.com/jrhemstad) - MithunR (https://github.com/mythrocks) - Mike Wilson (https://github.com/hyperbolic2346) URL: #9403
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses #7170
Adds 3 pieces of new functionality:
TDIGESTaggregation which creates a tdigest column (https://arxiv.org/pdf/1902.04023.pdf) from a stream of input scalars.MERGE_TDIGESTaggregation which merges multiple tdigest columns into a new one.percentile_approxfunction which performs percentile queries on tdigest data.Also exposes several ::detail functions (
sort,merge,slice) in detail headers.Ready for review. I do need to add more tests though.