Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Broken serialization of empty cudf.DataFrame #149

Closed
pentschev opened this issue Jan 3, 2024 · 1 comment · Fixed by #154
Closed

Broken serialization of empty cudf.DataFrame #149

pentschev opened this issue Jan 3, 2024 · 1 comment · Fixed by #154

Comments

@pentschev
Copy link
Member

pentschev commented Jan 3, 2024 •
edited
Loading

rapidsai/cudf#14615 broke serialization of empty cudf.DataFrame, causing

ucxx/python/distributed-ucxx/distributed_ucxx/tests/test_ucxx.py

Line 183 in c84c0f9

lambda cudf: cudf.DataFrame([], dtype=object),
to fail. In an offline discussion @wence- pointed should add serialize/deserialize to ColumnAccessor would probably be the best option, and @mroeschke agreed to it.

For now, I'm xfaililng that test to unblock CI, but it should be properly addressed in cuDF soon.
@pentschev
Copy link
Member Author

cc @charlesbluca

@mroeschke mroeschke mentioned this issue Jan 3, 2024
Merged
3 tasks
@pentschev pentschev mentioned this issue Jan 8, 2024
Merged
rapids-bot bot pushed a commit to rapidsai/cudf that referenced this issue Jan 9, 2024
@mroeschke
Ensure DataFrame column types are preserved during serialization (#14705
d81ca78 
)

xref rapidsai/ucxx#149

Ensures column subclass e.g `RangeIndex` and data types e.g. `int8` are preserved during serialization.

I think this should be backwards compatible since we're just adding keys to the the serialized `header` dict and the deserialization checks if those new keys exist

Authors:
  - Matthew Roeschke (https://github.com/mroeschke)
  - Charles Blackmon-Luca (https://github.com/charlesbluca)

Approvers:
  - Peter Andreas Entschev (https://github.com/pentschev)
  - Charles Blackmon-Luca (https://github.com/charlesbluca)

URL: #14705
@rapids-bot rapids-bot bot closed this as completed in #154 Jan 9, 2024
rapids-bot bot pushed a commit that referenced this issue Jan 9, 2024
@pentschev
Remove xfail mark from cuDF empty dataframe test (#154)
02c85e3 
Empty cuDF dataframe serialization was fixed in rapidsai/cudf#14705 .

Closes #149 .

Authors:
  - Peter Andreas Entschev (https://github.com/pentschev)

Approvers:
  - Lawrence Mitchell (https://github.com/wence-)

URL: #154
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove xfail mark from cuDF empty dataframe test pentschev/ucxx
1 participant
@pentschev