Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interchange dataframe protocol #9071

Merged
merged 67 commits into from
Nov 17, 2021
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
8fc7e17
dataframe protocol implementation, support only int, float, categoric…
iskode Aug 19, 2021
4367d8f
refactor to call from_dataframe on cudf directly and __dataframe__()…
iskode Aug 19, 2021
331c696
remove commented monkeypatch
iskode Aug 19, 2021
c83b4d7
refactor test cases
iskode Aug 19, 2021
defcbc5
propagate nan_as_null from DataFrame to Column class
iskode Aug 19, 2021
7c19720
start missing value supports + int missing value tests
iskode Aug 26, 2021
89d00f2
apply protocol update changes
iskode Sep 13, 2021
8450d7e
missing values support for int, float and categorical
iskode Sep 13, 2021
ec842d6
add boolean support w/ missing values
iskode Sep 14, 2021
13e0b95
refactor 'convert_column_to_cupy_ndarray' and replace 'cp.array' to '…
iskode Sep 14, 2021
dfa02a2
change 'copy' to 'allow_copy' code wide
iskode Sep 14, 2021
8586daa
fix merging issues with rapidsai/cudf updates
iskode Sep 14, 2021
d0cd04c
make "from_dataframe" accessible cudf: cudf.from_dataframe
iskode Sep 22, 2021
f0aede4
fix merge with recent changes on cudf repo
iskode Sep 22, 2021
afdfeed
Merge branch 'branch-21.12' of https://github.com/rapidsai/cudf into …
iskode Sep 27, 2021
2e85f5d
minor corrections + remove 'unpackbits' in favor of cudf's own function
iskode Sep 30, 2021
9ed11a3
use cudf Column object as _CuDFColumn's _col attribute instead of cu…
iskode Oct 1, 2021
31ae630
Merge branch 'branch-21.12' of https://github.com/rapidsai/cudf into …
iskode Oct 5, 2021
ca9686d
use cudf buffer in _CuDFBuffer class
iskode Oct 6, 2021
a3e65e4
use buffer protocol instead of dlpack protocol as the latter doesn't …
iskode Oct 7, 2021
ae5fb81
string support of the protocol
iskode Oct 8, 2021
ed7130b
rename class attribute 'x' into 'buf'
iskode Oct 14, 2021
9a6f957
add thorough tests for on '_CuDFcolumn' and '_CuDFBuffer' classes
iskode Oct 15, 2021
f9ca94d
write 'dlpack' support tests
iskode Oct 15, 2021
1709bab
dlpack support ok
iskode Oct 15, 2021
25c4474
use '_from_data' to create dataframe from columns
iskode Oct 18, 2021
5f441c2
harmonize method names like 'convert_to_cudf_string'
iskode Oct 20, 2021
78741a9
Do the same for 'convert_to_cudf_categorical'
iskode Oct 22, 2021
3cc229b
remove unused methods
iskode Oct 22, 2021
6fa4566
reorganize code so that class mentions occur after their definitions
iskode Oct 22, 2021
c0f2bc3
replace '_ints' and similar to module level constants
iskode Oct 22, 2021
3a3a9dc
define module level Device class and remove device check with number (2)
iskode Oct 22, 2021
7ac4f27
rename methods + annotate return types
iskode Oct 22, 2021
c572c64
add type annotations
iskode Oct 23, 2021
c60cf5b
correct '__dlpack_device__' annotation
iskode Oct 29, 2021
4262fc7
correct 'dtype' method annotation
iskode Oct 29, 2021
a7fe287
mark 'Device' class as private
iskode Nov 1, 2021
f5aef73
Apply suggestions from @bdice code review
iskode Nov 1, 2021
c47ce43
fix test errors due to changes + remove commented code
iskode Nov 1, 2021
139ca5a
add string column to mixed type test + a mixed type test case with NA
iskode Nov 1, 2021
bcec52c
address remaining suggestions from @bdice
iskode Nov 1, 2021
3eefe0c
change bare exception into RuntimeError
iskode Nov 1, 2021
a11ddb6
fix flake8 style checks
iskode Nov 2, 2021
117e432
fix flake8 style checks for 'test_df_protocol.py' file
iskode Nov 2, 2021
5eb7642
isort formatting
iskode Nov 2, 2021
e164540
run 'black' to format code.
iskode Nov 2, 2021
8b34a86
fix test errors.
iskode Nov 2, 2021
eba1cdd
minor style changes
iskode Nov 2, 2021
44c84d2
Merge branch 'df-protocol' of github.com:iskode/cudf into df-protocol
iskode Nov 2, 2021
5cffc2f
remove incorrect comment.
iskode Nov 3, 2021
5f27e66
fix xome mypy check errors.
iskode Nov 3, 2021
ac1ceb8
fix mypy errors
iskode Nov 9, 2021
e274ea2
fix last mypy errors
iskode Nov 9, 2021
7aa325e
rerun black,isort, flake8
iskode Nov 9, 2021
80bf86a
run isort on dataframe.py
iskode Nov 9, 2021
9a3549c
Remove _DTypeKind alias _k
iskode Nov 10, 2021
b76d419
Remove _DTypeKind alias _k
iskode Nov 10, 2021
a34f118
Remove _DTypeKind alias _k
iskode Nov 10, 2021
99ca31d
Remove _DTypeKind alias _k
iskode Nov 10, 2021
87bba32
Remove _DTypeKind alias _k
iskode Nov 10, 2021
535e56e
add space to multi-line comment.
iskode Nov 10, 2021
b421a29
Remove _DTypeKind alias _k
iskode Nov 10, 2021
6ae5ee0
remove remaining _DtypeKind aliases
iskode Nov 10, 2021
581153f
import DataFrameObject from df_protocol
iskode Nov 10, 2021
c1231cb
rename assertion methods
iskode Nov 10, 2021
043333c
Merge remote-tracking branch 'upstream/branch-22.02' into df-protocol
bdice Nov 12, 2021
0423cc6
Merge branch 'branch-22.02' of https://github.com/rapidsai/cudf into …
iskode Nov 17, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion python/cudf/cudf/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
UInt64Index,
interval_range,
)
from cudf.core.dataframe import DataFrame, from_pandas, merge
from cudf.core.dataframe import DataFrame, from_pandas, merge, from_dataframe
from cudf.core.series import Series
from cudf.core.multiindex import MultiIndex
from cudf.core.cut import cut
Expand Down
13 changes: 12 additions & 1 deletion python/cudf/cudf/core/dataframe.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
is_string_dtype,
is_struct_dtype,
)
from cudf.core import column, reshape
from cudf.core import column, reshape, df_protocol
from cudf.core.abc import Serializable
from cudf.core.column import (
as_column,
Expand Down Expand Up @@ -7098,6 +7098,17 @@ def explode(self, column, ignore_index=False):

return super()._explode(column, ignore_index)

def __dataframe__(
self, nan_as_null: bool = False, allow_copy: bool = True
):
return df_protocol.__dataframe__(
self, nan_as_null=nan_as_null, allow_copy=allow_copy
)


def from_dataframe(df, allow_copy=False):
return df_protocol.from_dataframe(df, allow_copy=allow_copy)


def from_pandas(obj, nan_as_null=None):
"""
Expand Down
Loading