Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Provide a way to specify the maximum allowable precision for integers/floats #10558

Open
shwina opened this issue Mar 31, 2022 · 2 comments
Assignees
Labels
feature request New feature or request Python Affects Python cuDF API.

Comments

@shwina
Copy link
Contributor

shwina commented Mar 31, 2022

Is your feature request related to a problem? Please describe.

GPU memory is a valuable resource, and using int64/float64 columns where int32/float32 would suffice means using 2x as much memory unnecessarily. As opposed to scientific computing, 32-bit data types (or lower) are sufficient for many data science applications.

Even only 32-bit data types as inputs, the resulting output can be a 64-bit type:

>>> cudf.Series([1, 2, 3], dtype="int32") + cudf.Scalar(1, dtype="float32")
0    2.0
1    3.0
2    4.0
dtype: float64

(this is consistent with Pandas and NumPy)

Describe the solution you'd like

It would be nice to be able to specify a maximum bitwidth for integer/floating types. If an operation would result in a value greater than could be accommodated, simply overflowing would be acceptable.

This could be another use case for cudf.config.

Describe alternatives you've considered

The user can carefully cast results back from 64bit to 32bit to reduce memory usage, but this is tedious and does not help with peak memory usage.

@shwina shwina added feature request New feature or request Needs Triage Need team to review and classify labels Mar 31, 2022
@shwina shwina added Python Affects Python cuDF API. and removed Needs Triage Need team to review and classify labels Mar 31, 2022
@github-actions
Copy link

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

@github-actions
Copy link

This issue has been labeled inactive-90d due to no recent activity in the past 90 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request Python Affects Python cuDF API.
Projects
Status: Todo
Development

No branches or pull requests

4 participants