Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Potential overflow of decimal32 when casting to int64_t #9287

Merged
merged 5 commits into from
Sep 24, 2021

Conversation

codereport
Copy link
Contributor

This bug fixes #9281.

The issue is that we should be using std::common_type_t for explicit casting between different types to avoid overflow. That fix has been applied.

@codereport codereport added bug Something isn't working libcudf Affects libcudf (C++/CUDA) code. 4 - Needs Review Waiting for reviewer to review or respond non-breaking Non-breaking change labels Sep 23, 2021
@codereport codereport self-assigned this Sep 23, 2021
@codereport codereport requested a review from a team as a code owner September 23, 2021 16:42
@codereport
Copy link
Contributor Author

rerun tests

@jrhemstad
Copy link
Contributor

Great job on the quick fix.

@codereport codereport added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 4 - Needs Review Waiting for reviewer to review or respond labels Sep 23, 2021
@codereport codereport added 0 - Waiting on Author Waiting for author to respond to review and removed 5 - Ready to Merge Testing and reviews complete, ready to merge labels Sep 23, 2021
@codereport
Copy link
Contributor Author

codereport commented Sep 23, 2021

@shwina Currently, Python tests are failing:

E   AssertionError: Series are different
E   
E   Series values are different (30.0 %)
E   [index]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
E   [left]:  [14.0, 2.0, nan, 0.0, 1844674399.0, nan, 94.0, 1844674295.0, 1844674399.0, nan]
E   [right]: [14.0, 2.0, nan, 0.0, 1.8446744073709552e+19, nan, 94.0, 1.8446744073709552e+19, 1.8446744073709552e+19, nan]

The input sequence is:

@pytest.mark.parametrize(
    "data",
    [
        cudf.Series(
            [
                14.12309,
                2.343942,
                np.nan,
                0.0,
                -8.302082, # failing
                np.nan,
                94.31308,
                -112.2364, # failing
                -8.029972, # failing
                np.nan,
            ]
        ),
    ],
)

So although it seems like the previous behaviour was consistent with arrow, negative numbers seem to not be working.

This looks like an orthogonal issue to the current bug. Do you agree? or is the previous behaviour expected?

Note: This was the PR that added these tests: #7450

@github-actions github-actions bot added the Python Affects Python cuDF API. label Sep 24, 2021
@codereport
Copy link
Contributor Author

@ashwin And I just had a short call and diagnosed the issue was that we were using NUMERIC_TYPES in the Python tests when we should have been using SIGNED_TYPES.

@codereport codereport added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 0 - Waiting on Author Waiting for author to respond to review labels Sep 24, 2021
@codereport codereport requested a review from shwina September 24, 2021 15:07
@codereport codereport requested a review from a team as a code owner September 24, 2021 15:15
@codereport
Copy link
Contributor Author

rerun tests

@codecov
Copy link

codecov bot commented Sep 24, 2021

Codecov Report

Merging #9287 (f71b51c) into branch-21.12 (ab4bfaa) will increase coverage by 0.01%.
The diff coverage is 0.00%.

❗ Current head f71b51c differs from pull request most recent head 248b69c. Consider uploading reports for the commit 248b69c to get more accurate results
Impacted file tree graph

@@               Coverage Diff                @@
##           branch-21.12    #9287      +/-   ##
================================================
+ Coverage         10.79%   10.80%   +0.01%     
================================================
  Files               116      116              
  Lines             18869    19321     +452     
================================================
+ Hits               2036     2087      +51     
- Misses            16833    17234     +401     
Impacted Files Coverage Δ
python/cudf/cudf/io/orc.py 0.00% <0.00%> (ø)
python/cudf/cudf/io/hdf.py 0.00% <0.00%> (ø)
python/cudf/cudf/_version.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/abc.py 0.00% <0.00%> (ø)
python/cudf/cudf/api/types.py 0.00% <0.00%> (ø)
python/cudf/cudf/io/dlpack.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/frame.py 0.00% <0.00%> (ø)
python/cudf/cudf/core/index.py 0.00% <0.00%> (ø)
python/cudf/cudf/io/feather.py 0.00% <0.00%> (ø)
... and 47 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ba76310...248b69c. Read the comment docs.

@codereport codereport changed the base branch from branch-21.10 to branch-21.12 September 24, 2021 18:18
@codereport
Copy link
Contributor Author

@gpucibot merge

@rapids-bot rapids-bot bot merged commit b4560f4 into rapidsai:branch-21.12 Sep 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge bug Something isn't working libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Casting DECIMAL32 with a positive scale to INT64 results in premature overflow
6 participants