Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Cannot serialize a DecimalDtype column #8034

Closed
beckernick opened this issue Apr 22, 2021 · 0 comments · Fixed by #8041
Closed

[BUG] Cannot serialize a DecimalDtype column #8034

beckernick opened this issue Apr 22, 2021 · 0 comments · Fixed by #8041
Assignees
Labels
bug Something isn't working Python Affects Python cuDF API.

Comments

@beckernick
Copy link
Member

We can't currently serialize a Decimal64Dtype column. It looks like our serialization procedure expects a str attribute on the Dtype data structure to provide a typestring in line with https://numpy.org/doc/stable/reference/generated/numpy.dtype.html

import cudf
from cudf.core.dtypes import Decimal64Dtypes = cudf.Series([1.31]).astype(Decimal64Dtype(9,2))
s.serialize()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-99-80008fce367e> in <module>
      3 
      4 s = cudf.Series([1.31]).astype(Decimal64Dtype(9,2))
----> 5 s.serialize()

/raid/nicholasb/miniconda3/envs/rapids-gpubdb-20210421/lib/python3.8/site-packages/cudf/core/series.py in serialize(self)
    395         frames.extend(index_frames)
    396         header["index_frame_count"] = len(index_frames)
--> 397         header["column"], column_frames = self._column.serialize()
    398         header["type-serialized"] = pickle.dumps(type(self))
    399         frames.extend(column_frames)

/raid/nicholasb/miniconda3/envs/rapids-gpubdb-20210421/lib/python3.8/site-packages/cudf/core/column/column.py in serialize(self)
   1266         frames = []
   1267         header["type-serialized"] = pickle.dumps(type(self))
-> 1268         header["dtype"] = self.dtype.str
   1269 
   1270         if self.data is not None:

AttributeError: 'Decimal64Dtype' object has no attribute 'str'
conda list | grep "rapids\|dask\|pandas\|arrow\|numpy\|scipy"
# packages in environment at /raid/nicholasb/miniconda3/envs/rapids-gpubdb-20210421:
arrow-cpp                 1.0.1           py38hcb5322d_14_cuda    conda-forge
arrow-cpp-proc            3.0.0                      cuda    conda-forge
cudf                      0.20.0a210421   cuda_11.0_py38_gd501d2c0b9_179    rapidsai-nightly
cuml                      0.20.0a210421   cuda11.0_py38_g2870d59d8_80    rapidsai-nightly
dask                      2021.4.0           pyhd8ed1ab_0    conda-forge
dask-core                 2021.4.0           pyhd8ed1ab_0    conda-forge
dask-cuda                 0.20.0a210421           py38_17    rapidsai-nightly
dask-cudf                 0.20.0a210421   py38_gd501d2c0b9_179    rapidsai-nightly
libcudf                   0.20.0a210421   cuda11.0_gd501d2c0b9_179    rapidsai-nightly
libcuml                   0.20.0a210421   cuda11.0_g2870d59d8_80    rapidsai-nightly
libcumlprims              0.20.0a210408   cuda11.0_g7f19636_2    rapidsai-nightly
librmm                    0.20.0a210421   cuda11.0_g288e8be_17    rapidsai-nightly
numpy                     1.19.5           py38h18fd61f_1    conda-forge
pandas                    1.2.4            py38h1abd341_0    conda-forge
pyarrow                   1.0.1           py38h3e2403a_14_cuda    conda-forge
rmm                       0.20.0a210421   cuda_11.0_py38_g288e8be_17    rapidsai-nightly
scipy                     1.6.2            py38h7b17777_0    conda-forge
ucx                       1.9.0+gcd9efd3       cuda11.0_0    rapidsai-nightly
ucx-proc                  1.0.0                       gpu    rapidsai-nightly
ucx-py                    0.20.0a210419   py38_gcd9efd3_5    rapidsai-nightly

cc @randerzander

@beckernick beckernick added bug Something isn't working Needs Triage Need team to review and classify labels Apr 22, 2021
@randerzander randerzander added Python Affects Python cuDF API. and removed Needs Triage Need team to review and classify labels Apr 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Python Affects Python cuDF API.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants