Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add to/from arrow for fixed point columns #7609

Merged
merged 48 commits into from
Apr 12, 2021
Merged
Changes from 1 commit
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
c7f35e8
Reuse initializer list in test
shwina Mar 16, 2021
09c7054
Add fixed point from arrow test
shwina Mar 16, 2021
18f77e7
Support DECIMAL conversion + cleanup
codereport Mar 16, 2021
d460ab1
Fix test
shwina Mar 16, 2021
c4085ad
SFINAE + temporary/initial changes
codereport Mar 17, 2021
31b4a56
Progress
shwina Mar 17, 2021
64a15c9
Changes
codereport Mar 17, 2021
552d316
Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into a…
shwina Mar 22, 2021
8494746
Merge branch 'add-from-to-arrow-fp' of github.com:shwina/cudf into ad…
shwina Mar 22, 2021
ce6e756
Add failing null test
shwina Mar 22, 2021
9db4246
Add null logic
codereport Mar 22, 2021
54d7164
Remove TODO
shwina Mar 22, 2021
8dd82be
Add more scales to test
shwina Mar 22, 2021
65e6b07
Add Large Test (no nulls)
codereport Mar 22, 2021
736fd15
Add Large Unit Test with Nulls + Cleanup
codereport Mar 22, 2021
54acb0c
Remove header
codereport Mar 23, 2021
31358a2
Cleanup
codereport Mar 23, 2021
8eaaed4
Add failing to_arrow test
shwina Mar 23, 2021
dd25a2e
Rename to_arrow.cpp -> to_arrow.cu
shwina Mar 23, 2021
c432107
Half baked to_arrow
shwina Mar 23, 2021
e3f704e
Initial to_arrow decimal64 work
codereport Mar 23, 2021
3b5f0be
Changes
codereport Mar 26, 2021
04ed34b
Get things building again
shwina Mar 26, 2021
f29ff3a
Changes
codereport Mar 26, 2021
1b81752
to_arrow changes
codereport Mar 29, 2021
5e4cb03
Add failing null test
shwina Mar 29, 2021
d64092f
Unit tests
codereport Mar 29, 2021
34b286d
Fix FixPointTableLarge
codereport Mar 30, 2021
4bd322d
Commenting out tests
codereport Mar 30, 2021
ab3c3d1
Fix for failing to_arrow tests :D :D
codereport Mar 30, 2021
84b0f96
Remove DecimalColumn.to/from_arrow. Patch precision of decimal result
shwina Mar 30, 2021
38ea86f
Merge branch 'branch-0.19' into add-from-to-arrow-fp
codereport Mar 30, 2021
161ec7f
Switch from SFINAE to template specialization
codereport Mar 30, 2021
b68b703
Fix scale
codereport Mar 30, 2021
89a4c0c
Bye bye thrust::scatter, hello thrust::for_each
codereport Mar 30, 2021
398b462
Patch precision in from_arrow
shwina Mar 30, 2021
4e937fb
Fix from_arrow scale & tests
codereport Mar 30, 2021
9d128c3
Merge branch 'add-from-to-arrow-fp' of github.com:shwina/cudf into ad…
shwina Mar 30, 2021
296c5df
xfail pytests with explanation
shwina Mar 30, 2021
b66e83f
C++ cleanup
codereport Mar 30, 2021
847efb3
Merge branch 'add-from-to-arrow-fp' of https://github.com/shwina/cudf…
codereport Mar 30, 2021
20c5f96
Change cast->view and remove xfail
shwina Mar 31, 2021
57fca21
Merge branch 'add-from-to-arrow-fp' of github.com:shwina/cudf into ad…
shwina Mar 31, 2021
60e875c
Merge branch 'branch-0.19' of https://github.com/rapidsai/cudf into a…
shwina Mar 31, 2021
b6c1f35
Addressing PR comments
codereport Mar 31, 2021
7503e6c
Address PR comments
codereport Mar 31, 2021
133529c
Addressing PR comments
codereport Apr 6, 2021
316ca7b
Merge branch 'branch-0.20' into add-from-to-arrow-fp
codereport Apr 8, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Change cast->view and remove xfail
  • Loading branch information
shwina committed Mar 31, 2021
commit 20c5f96bf5e8b8a4273aa0d0a529a1904c006d13
2 changes: 1 addition & 1 deletion python/cudf/cudf/core/column/column.py
Original file line number Diff line number Diff line change
@@ -366,7 +366,7 @@ def to_arrow(self) -> pa.Array:
)["None"].chunk(0)

if isinstance(self.dtype, cudf.Decimal64Dtype):
result = result.cast(
result = result.view(
pa.decimal128(
scale=result.type.scale, precision=self.dtype.precision
)
9 changes: 1 addition & 8 deletions python/cudf/cudf/tests/test_decimal.py
Original file line number Diff line number Diff line change
@@ -20,14 +20,7 @@
[1],
[-1],
[1, 2, 3, 4],
pytest.param(
[42, 1729, 4104],
marks=pytest.mark.xfail(
reason="pyarrow bug that sometimes "
"doesn't allow casting to a higher precision "
"then back."
),
),
[42, 1729, 4104],
[1, 2, None, 4],
[None, None, None],
[],