Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clarify dtype type in ColumnBuffers elements #87

Merged
merged 3 commits into from
Oct 12, 2022
Merged

Clarify dtype type in ColumnBuffers elements #87

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8003971
clarify ColumnBuffers.validity dtype type
flexatone Sep 24, 2022
cb391f7
clarify ColumnBuffers.validity dtype type
flexatone Sep 24, 2022
f446156
Update protocol/dataframe_protocol.py
flexatone Sep 26, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions protocol/dataframe_protocol.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,9 @@ class DtypeKind(enum.IntEnum):
CATEGORICAL = 23


Dtype = Tuple[DtypeKind, int, str, str] # see Column.dtype


class ColumnNullType(enum.IntEnum):
"""
Integer enum for null type representation.
Expand Down Expand Up @@ -86,18 +89,18 @@ class ColumnNullType(enum.IntEnum):
class ColumnBuffers(TypedDict):
# first element is a buffer containing the column data;
# second element is the data buffer's associated dtype
data: Tuple["Buffer", Any]
data: Tuple["Buffer", Dtype]

# first element is a buffer containing mask values indicating missing data;
# second element is the mask value buffer's associated dtype.
# None if the null representation is not a bit or byte mask
validity: Optional[Tuple["Buffer", Any]]
validity: Optional[Tuple["Buffer", Dtype]]

# first element is a buffer containing the offset values for
# variable-size binary data (e.g., variable-length strings);
# second element is the offsets buffer's associated dtype.
# None if the data buffer does not have an associated offsets buffer
offsets: Optional[Tuple["Buffer", Any]]
offsets: Optional[Tuple["Buffer", Dtype]]


class CategoricalDescription(TypedDict):
Expand Down Expand Up @@ -237,7 +240,7 @@ def offset(self) -> int:

@property
@abstractmethod
def dtype(self) -> Tuple[DtypeKind, int, str, str]:
def dtype(self) -> Dtype:
"""
Dtype description as a tuple ``(kind, bit-width, format string, endianness)``.

Expand Down