Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support MERGE_LISTS and MERGE_SETS in Java package [skip ci] #8516

Merged
merged 3 commits into from
Jun 24, 2021

Conversation

sperlingxx
Copy link
Contributor

@sperlingxx sperlingxx commented Jun 15, 2021

Closes #8445

This PR is to provide Java bindings for #8436.

@sperlingxx sperlingxx requested a review from a team as a code owner June 15, 2021 10:25
@sperlingxx sperlingxx added the 2 - In Progress Currently a work in progress label Jun 15, 2021
@github-actions github-actions bot added the Java Affects Java cuDF API. label Jun 15, 2021
@sperlingxx sperlingxx added non-breaking Non-breaking change feature request New feature or request labels Jun 15, 2021
@ttnghia ttnghia added the 3 - Ready for Review Ready for review by team label Jun 22, 2021
@sperlingxx sperlingxx changed the title [WIP] Support MERGE_LISTS and MERGE_SETS in Java package [skip ci] Support MERGE_LISTS and MERGE_SETS in Java package [skip ci] Jun 22, 2021
@sperlingxx sperlingxx removed the 2 - In Progress Currently a work in progress label Jun 22, 2021
@sperlingxx sperlingxx requested a review from ttnghia June 23, 2021 02:34
@sperlingxx
Copy link
Contributor Author

@gpucibot merge

@rapids-bot rapids-bot bot merged commit a73d3b3 into rapidsai:branch-21.08 Jun 24, 2021
@revans2
Copy link
Contributor

revans2 commented Jun 24, 2021

@sperlingxx did you test this? With this patch I am getting all kinds of errors with the lead/lag tests that don't exist before the patch.

@@ -56,10 +56,12 @@
ROW_NUMBER(17),
COLLECT_LIST(18),
COLLECT_SET(19),
LEAD(20),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@revans2 I assume that your problem is due to this.

Copy link
Contributor

@revans2 revans2 Jun 24, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No my problem is that @sperlingxx didn't even run the unit tests or he would have found this and there would have been no issue. I will post a patch shortly

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we arbitrarily define the value for aggregation enums, which is different from the enums defined in libcudf. I was talking about this before (probably with Jason). We should have some better way to automate this. Otherwise, we will continue to have issues with this in the future.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These just happen to match the values in libcudf. But in this case we never cast them so it does not have to match. This is the one place that we do it correctly. If we did cast them, then your patch that changed the values would have broken the java build, not this patch.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, it's my mistake! I apologized for my carelessness.

rapids-bot bot pushed a commit that referenced this pull request Jun 24, 2021
Fixes an issue introduced by #8516

Authors:
  - Robert (Bobby) Evans (https://github.com/revans2)

Approvers:
  - Jason Lowe (https://github.com/jlowe)

URL: #8600
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Ready for Review Ready for review by team feature request New feature or request Java Affects Java cuDF API. non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEA] provide Java bindings for aggregations: MERGE_LISTS and MERGE_SETS
3 participants