-
Notifications
You must be signed in to change notification settings - Fork 916
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add list_scalar
API
#7584
Add list_scalar
API
#7584
Conversation
Codecov Report
@@ Coverage Diff @@
## branch-0.20 #7584 +/- ##
===============================================
+ Coverage 82.88% 82.91% +0.02%
===============================================
Files 103 103
Lines 17668 17658 -10
===============================================
- Hits 14645 14642 -3
+ Misses 3023 3016 -7
Continue to review full report at Codecov.
|
Co-authored-by: Nghia Truong <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This new class needs to be tested...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed my comments because they were a subset of @harrism 's review comments.
- No need for two separate
view()
andvalue()
methods. This will probably not return a host usable value like the rest of the scalar classes. Any such member can be added later when needed. - I'd vote against having validity as an enum.
Is this meant to support a list of list case? If so, you might want to use |
@devavret yes this should support nested lists case. Would love to have more clarification of |
Addressing @devavret and @jrhemstad 's comment, this PR will not introduce an
|
After an offline discussion, we figured we should return a |
We should be careful and not rush this. I don't understand all the details, but rushing this could result in painful churn, so let's figure out what the right thing to do is. If figuring that out is too big a task, then we can consider a halfway solution. |
@gpucibot merge |
…ssed on to construct validity data (#8072) Follow up of #7584 This PR fixes a bug when constructing `list_scalar`, `stream` and `mr` was not passed on to construct validity data in constructor. Currently only default stream and memory resource is used. Authors: - Michael Wang (https://github.com/isVoid) Approvers: - Nghia Truong (https://github.com/ttnghia) - Mark Harris (https://github.com/harrism) URL: #8072
This PR is to add the JNI support for scalar of list, along with building a ColumnVector from a list scalar. Since the PR #7584 inroduced the list scalar in cpp. Signed-off-by: Firestarman <[email protected]> Authors: - Liangcai Li (https://github.com/firestarman) Approvers: - Jason Lowe (https://github.com/jlowe) URL: #8077
Closes #5887
This PR adds
list_scalar
, a scalar for aLIST
type column. Alist_scalar
contains a validity field and a_data
field. The validity field describe whether the list_scalar is valid as a whole, and is accessible via derivedis_valid
member function. The_data
field is acudf::column
, owned by the scalar class. The data column can be either nested or non-nested type, depending on whatLIST
column it originate from.For example, a scalar from a
List<int>
column contains aint
column:A scalar from a
List<List<int>>
column would contain aList<int>
column instead:list_scalar
has aview
method, which returns a immutable view to the underlying data.