Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Support nested types in CreateNamedStruct #2436

Closed
revans2 opened this issue May 18, 2021 · 4 comments · Fixed by #2498
Closed

[FEA] Support nested types in CreateNamedStruct #2436

revans2 opened this issue May 18, 2021 · 4 comments · Fixed by #2498
Assignees
@revans2 @sperlingxx
Labels
feature request New feature or request P0 Must have for release
Milestone
May 10 - May 21

Comments

@revans2
Copy link
Collaborator

revans2 commented May 18, 2021

We have basic support for CreateNamedStruct but we cannot put nested types into a struct because cudf does not support scalars of nested types. That is slowly changing and we should add in support for Structs, Arrays, and Maps to be children of names structs.
@revans2 revans2 added feature request New feature or request ? - Needs Triage Need team to review and classify labels May 18, 2021
@sameerz sameerz removed the ? - Needs Triage Need team to review and classify label May 18, 2021
@sameerz sameerz added this to the May 10 - May 21 milestone May 18, 2021
@sameerz sameerz added the P0 Must have for release label May 18, 2021
@sperlingxx
Copy link
Collaborator

sperlingxx commented May 21, 2021 •
edited
Loading

Hi @revans2, we have the support of list_scalar and struct_scalar in libcudf. And I am working on support GpuLiteral on StructType. But for MapType, it seems not easy to support it as children of GpuCreateNamedStruct, if I am not wrong.

@revans2
Copy link
Collaborator Author

revans2 commented May 21, 2021

CUDF does not support maps directly. We emulate a map as a list of key, value structs. So if we have a Scalar List that can hold structs we can support scalar maps.

@sperlingxx sperlingxx mentioned this issue May 24, 2021
Merged
@revans2 revans2 self-assigned this May 24, 2021
@revans2
Copy link
Collaborator Author

revans2 commented May 24, 2021

I have a patch that I will put up tomorrow. Want to do a bit more testing on it.

@revans2
Copy link
Collaborator Author

revans2 commented May 24, 2021

Also struct literals are not supported just yet in cudf, but it should be very simple to add it in once that work is ready.

@revans2 revans2 mentioned this issue May 25, 2021
Merged
@revans2 revans2 mentioned this issue May 25, 2021
Closed
rapids-bot bot pushed a commit to rapidsai/cudf that referenced this issue May 26, 2021
@sperlingxx
Java: Support struct scalar (#8327)
2383193 
Current PR is to support struct scalar in Java package, which is required by spark-rapids ([issue link](NVIDIA/spark-rapids#2436)). In detail, current PR introduces three new features:
1. create struct scalar through Java API
2. get children of struct scalar through Java API
3. create struct column from (struct) scalar through Java API

Authors:
  - Alfred Xu (https://github.com/sperlingxx)

Approvers:
  - Robert (Bobby) Evans (https://github.com/revans2)

URL: #8327
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@revans2 revans2

@sperlingxx sperlingxx

Labels
feature request New feature or request P0 Must have for release
Projects
None yet
Milestone
May 10 - May 21
Development

Successfully merging a pull request may close this issue.

Add in basic support for scalar maps and allow nesting in named_struct revans2/spark-rapids
3 participants
@revans2 @sperlingxx @sameerz