Add `lists.sort_values` API #7657

isVoid · 2021-03-19T19:39:47Z

Introduces list method list.sort_values. Sorts each list of a LIST column based on given criterion. This method signature is aligned with Series.sort_values. Example:

>>> s = cudf.Series([[4, 2, None, 9], [8, 8, 2], [2, 1]])
>>> s.list.sort_values(ascending=False, na_position="last")
0    [nan, 9.0, 4.0, 2.0]
1         [8.0, 8.0, 2.0]
2              [2.0, 1.0]
dtype: list

This PR also includes exposing ListMethods to docs and a small docstring fix to cudf.Series.

python/cudf/cudf/core/column/lists.py

python/cudf/cudf/_lib/cpp/lists/sorting.pxd

python/cudf/cudf/_lib/lists.pyx

python/cudf/cudf/_lib/cpp/lists/sorting.pxd

python/cudf/cudf/core/column/lists.py

Co-authored-by: GALI PREM SAGAR <[email protected]>

codecov · 2021-03-19T22:41:26Z

Codecov Report

Merging #7657 (8acf8f0) into branch-0.19 (7871e7a) will increase coverage by 0.61%.
The diff coverage is n/a.

@@               Coverage Diff               @@
##           branch-0.19    #7657      +/-   ##
===============================================
+ Coverage        81.86%   82.48%   +0.61%     
===============================================
  Files              101      101              
  Lines            16884    17426     +542     
===============================================
+ Hits             13822    14373     +551     
+ Misses            3062     3053       -9

Impacted Files	Coverage Δ
python/cudf/cudf/core/column/categorical.py	`91.97% <ø> (+0.58%)`	⬆️
python/cudf/cudf/core/column/column.py	`87.86% <ø> (+0.10%)`	⬆️
python/cudf/cudf/core/column/datetime.py	`89.63% <ø> (+0.54%)`	⬆️
python/cudf/cudf/core/column/decimal.py	`92.75% <ø> (-2.12%)`	⬇️
python/cudf/cudf/core/column/lists.py	`90.00% <ø> (-1.40%)`	⬇️
python/cudf/cudf/core/column/numerical.py	`94.83% <ø> (-0.20%)`	⬇️
python/cudf/cudf/core/column/string.py	`86.79% <ø> (+0.30%)`	⬆️
python/cudf/cudf/core/column/timedelta.py	`88.57% <ø> (+0.33%)`	⬆️
python/cudf/cudf/core/column_accessor.py	`96.01% <ø> (+0.70%)`	⬆️
python/cudf/cudf/core/dataframe.py	`90.90% <ø> (+0.43%)`	⬆️
... and 64 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d500142...8acf8f0. Read the comment docs.

kkraus14 · 2021-03-20T03:24:01Z

python/cudf/cudf/_lib/lists.pyx

@@ -58,3 +63,22 @@ def explode_outer(Table tbl, int explode_column_idx, bool ignore_index=False):
        column_names=tbl._column_names,
        index_names=None if ignore_index else tbl._index_names
    )
+
+
+def sort_lists(Column col, object order_enum, object null_order_enum):


Is there a way to more tightly type the enums here? I thought we had a pattern fo handling enum values like this elsewhere but I could be mistaken.

Unfortunately no. I think typically we just pass a Python object to Cython, where we then plumb it through to the appropriate C++ type. See for example https://github.com/rapidsai/cudf/blob/branch-0.19/python/cudf/cudf/_lib/sort.pyx#L26.

I'd suggest we do something similar here, where the Cython API shouldn't expect a Python enum, but rather something like a string or a bool. That encapsulates things a little better, making the use of an enum an implementation detail.

Apologies, @isVoid I got this backwards and you had it right from the beginning

I think this needs to be addressed before merging

Got it. Sorry misunderstood on this one!

kkraus14 · 2021-03-24T01:02:29Z

@gpucibot merge

isVoid added 2 commits March 19, 2021 11:05

Initial

6030947

remove indirection, na_pos check

6fe171a

isVoid requested a review from a team as a code owner March 19, 2021 19:39

isVoid requested review from cwharris and galipremsagar March 19, 2021 19:39

github-actions bot added the Python Affects Python cuDF API. label Mar 19, 2021

isVoid added 3 - Ready for Review Ready for review by team feature request New feature or request non-breaking Non-breaking change labels Mar 19, 2021

isVoid commented Mar 19, 2021

View reviewed changes

python/cudf/cudf/core/column/lists.py Show resolved Hide resolved

kkraus14 reviewed Mar 19, 2021

View reviewed changes

python/cudf/cudf/_lib/cpp/lists/sorting.pxd Outdated Show resolved Hide resolved

python/cudf/cudf/_lib/lists.pyx Outdated Show resolved Hide resolved

python/cudf/cudf/_lib/lists.pyx Outdated Show resolved Hide resolved

galipremsagar reviewed Mar 19, 2021

View reviewed changes

python/cudf/cudf/_lib/cpp/lists/sorting.pxd Outdated Show resolved Hide resolved

python/cudf/cudf/core/column/lists.py Show resolved Hide resolved

isVoid and others added 4 commits March 19, 2021 13:33

License year

b8b884d

Co-authored-by: GALI PREM SAGAR <[email protected]>

Use internal enum classes for order and null_order

56e06c8

capture cpp exception

8c1b763

Several docstring fixes, exposing lists API docs

574984a

Merge branch '7467' of github.com:isVoid/cudf into 7467

3d59f8f

isVoid requested review from kkraus14 and galipremsagar March 19, 2021 22:47

galipremsagar approved these changes Mar 19, 2021

View reviewed changes

Add nested list error msg

7b36e63

kkraus14 reviewed Mar 20, 2021

View reviewed changes

kkraus14 approved these changes Mar 20, 2021

View reviewed changes

isVoid mentioned this pull request Mar 22, 2021

Adds list.unique API #7664

Merged

isVoid added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team labels Mar 22, 2021

kkraus14 removed the 5 - Ready to Merge Testing and reviews complete, ready to merge label Mar 23, 2021

kkraus14 added the 0 - Waiting on Author Waiting for author to respond to review label Mar 23, 2021

isVoid added 3 commits March 23, 2021 09:57

Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into 7467

029c00d

style

e5ba902

Passing bool/str arg to cython

8acf8f0

kkraus14 approved these changes Mar 24, 2021

View reviewed changes

kkraus14 added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 0 - Waiting on Author Waiting for author to respond to review labels Mar 24, 2021

rapids-bot bot merged commit 1e9f8f8 into rapidsai:branch-0.19 Mar 24, 2021

randerzander mentioned this pull request Apr 21, 2021

[FEA] Support drop_duplicates on Series containing list objects #6784

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `lists.sort_values` API #7657

Add `lists.sort_values` API #7657

isVoid commented Mar 19, 2021 •

edited

Loading

codecov bot commented Mar 19, 2021 •

edited

Loading

kkraus14 Mar 20, 2021

shwina Mar 22, 2021

kkraus14 Mar 22, 2021

kkraus14 Mar 23, 2021

isVoid Mar 23, 2021

kkraus14 commented Mar 24, 2021

Add lists.sort_values API #7657

Add lists.sort_values API #7657

Conversation

isVoid commented Mar 19, 2021 • edited Loading

codecov bot commented Mar 19, 2021 • edited Loading

Codecov Report

kkraus14 Mar 20, 2021

Choose a reason for hiding this comment

shwina Mar 22, 2021

Choose a reason for hiding this comment

kkraus14 Mar 22, 2021

Choose a reason for hiding this comment

kkraus14 Mar 23, 2021

Choose a reason for hiding this comment

isVoid Mar 23, 2021

Choose a reason for hiding this comment

kkraus14 commented Mar 24, 2021

Add `lists.sort_values` API #7657

Add `lists.sort_values` API #7657

isVoid commented Mar 19, 2021 •

edited

Loading

codecov bot commented Mar 19, 2021 •

edited

Loading