Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[FEA] Decimal128 support in Parquet reader/writer #9566

Closed
2 tasks
vuule opened this issue Oct 29, 2021 · 1 comment · Fixed by #9765
Closed
2 tasks

[FEA] Decimal128 support in Parquet reader/writer #9566

vuule opened this issue Oct 29, 2021 · 1 comment · Fixed by #9765
Assignees
@vuule
Labels
cuIO cuIO issue feature request New feature or request
Milestone
IO Data Type Expa...

Comments

@vuule
Copy link
Contributor

vuule commented Oct 29, 2021

  • Support decode to decimal128 in the reader (opt-in, keep decimal64 as default);
  • Support encode of decimal128 columns in the writer.
@vuule vuule added feature request New feature or request cuIO cuIO issue labels Oct 29, 2021
@vuule vuule self-assigned this Oct 29, 2021
@Salonijain27 Salonijain27 mentioned this issue Oct 29, 2021
Closed
@beckernick beckernick added this to the IO Data Type Expansion milestone Nov 3, 2021
@vuule vuule mentioned this issue Nov 23, 2021
Merged
@github-actions
Copy link

github-actions bot commented Dec 3, 2021

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

@rapids-bot rapids-bot bot closed this as completed in #9765 Dec 8, 2021
rapids-bot bot pushed a commit that referenced this issue Dec 8, 2021
@vuule
Add decimal128 support to Parquet reader and writer (#9765)
ea3aff2 
Closes #9566
Depends on #9804

Read decimal columns as 128bit when the input width requires it.
Write decimal128 columns as `FIXED_LEN_BYTE_ARRAY`.
Use the smallest viable decimal size to read `FIXED_LEN_BYTE_ARRAY` (used to default to decimal64, even when 32bits are sufficient).
Removes `strict_decimal_types` option from Parquet reader, we can now always read using the exact decimal type.

Authors:
  - Vukasin Milovanovic (https://github.com/vuule)

Approvers:
  - GALI PREM SAGAR (https://github.com/galipremsagar)
  - Devavret Makkar (https://github.com/devavret)
  - MithunR (https://github.com/mythrocks)
  - Charles Blackmon-Luca (https://github.com/charlesbluca)
  - https://github.com/nvdbaranec

URL: #9765
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vuule vuule

Labels
cuIO cuIO issue feature request New feature or request
Projects
None yet
Milestone
IO Data Type Expansion
Development

Successfully merging a pull request may close this issue.

Add decimal128 support to Parquet reader and writer vuule/cudf
2 participants
@beckernick @vuule