Generate group labels from offsets #10945
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ttnghia
added
feature request
This comment was marked as off-topic.
This comment was marked as off-topic.
devavret
reviewed
May 24, 2022
jrhemstad
reviewed
May 24, 2022
jrhemstad
reviewed
May 24, 2022
jrhemstad
reviewed
May 24, 2022
jrhemstad
reviewed
May 24, 2022
jrhemstad
reviewed
May 24, 2022
Signed-off-by: Nghia Truong <[email protected]>
|
Rerun tests. |
This reverts commit 15d036a.
Signed-off-by: Nghia Truong <[email protected]>
Signed-off-by: Nghia Truong <[email protected]>
Signed-off-by: Nghia Truong <[email protected]>
Signed-off-by: Nghia Truong <[email protected]>
Signed-off-by: Nghia Truong <[email protected]>
Signed-off-by: Nghia Truong <[email protected]>
jrhemstad
reviewed
May 31, 2022
jrhemstad
reviewed
May 31, 2022
Signed-off-by: Nghia Truong <[email protected]>
Signed-off-by: Nghia Truong <[email protected]>
jrhemstad
reviewed
May 31, 2022
jrhemstad
approved these changes
May 31, 2022
Signed-off-by: Nghia Truong <[email protected]>
devavret
approved these changes
Jun 1, 2022
|
@gpucibot merge |
This was referenced Jun 1, 2022
rapids-bot bot
pushed a commit
that referenced
this pull request
Jun 3, 2022
Given an array of integer values which may be the labels of some list elements, we want to generate an array of offsets so we can create a lists column from these offsets. For example: ``` input_labels = [ 0, 0, 0, 0, 1, 1, 4, 4, 4, 4 ] output = [ 0, 4, 6, 6, 6, 10 ] ``` This is basically the reverse operation of #10945. Closes #10955. Authors: - Nghia Truong (https://github.com/ttnghia) Approvers: - Devavret Makkar (https://github.com/devavret) - Jake Hemstad (https://github.com/jrhemstad) URL: #11017
rapids-bot bot
pushed a commit
that referenced
this pull request
Jul 26, 2022
This PR adds the following APIs for set operations: * `lists::have_overlap` * `lists::intersect_distinct` * `lists::union_distinct` * `lists::difference_distinct` ### Name Convention Except for the first API (`lists::have_overlap`) that returns a boolean column, the suffix `_distinct` of the rest APIs denotes that their results will be lists columns in which all list rows have been post-processed to remove duplicates. As such, their results are actually "set" columns in which each row is a "set" of distinct elements. --- Depends on: * #10945 * #11017 * NVIDIA/cuCollections#175 * #11052 * #11118 * #11100 * #11149 Closes #10409. Authors: - Nghia Truong (https://github.com/ttnghia) - Yunsong Wang (https://github.com/PointKernel) Approvers: - Michael Wang (https://github.com/isVoid) - AJ Schmidt (https://github.com/ajschmidt8) - Bradley Dice (https://github.com/bdice) - Yunsong Wang (https://github.com/PointKernel) URL: #11043
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a small (detail) API that generates group labels from a given offset array
offsets. The output will be an array containing consecutive groups of identical labels, the number of elements in each groupiis defined byoffsets[i+1] - offsets[i].Examples:
Note that the label values always start from
0. We can in fact add a parameter to allow specifying any starting value but we don't need it in now.Several places in cudf have been updated to adopt the new API immediately. These places have been tested extensively thus no unit tests for the new API is needed. In addition, I ran a benchmark for groupby aggregations and found no performance difference after adopting this.
Closes #10905 and unblocks #10409.