Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] Casting decimal64 into decimal32 fails with some values #9000

Closed
isVoid opened this issue Aug 9, 2021 · 5 comments · Fixed by #9772
Closed

[BUG] Casting decimal64 into decimal32 fails with some values #9000

isVoid opened this issue Aug 9, 2021 · 5 comments · Fixed by #9772
Assignees
@codereport
Labels
feature request New feature or request libcudf Affects libcudf (C++/CUDA) code.
Milestone
Decimal data type...

Comments

@isVoid
Copy link
Contributor

isVoid commented Aug 9, 2021

Describe the bug
With some decimal64 input and decimal32 type combination, I'm not getting expected result from libcudf. See below.

Steps/Code to reproduce bug

// In cudf::tests
using namespace numeric;
using fp_wrapperA = cudf::test::fixed_point_column_wrapper<int32_t>;
using fp_wrapperB = cudf::test::fixed_point_column_wrapper<int64_t>;

auto const input    = fp_wrapperB{{141230900000L}, scale_type{-10}};
auto const expected = fp_wrapperA{{14123}, scale_type{-3}};
auto const result   = cudf::cast(input, make_fixed_point_data_type<decimal32>(-3));

CUDF_TEST_EXPECT_COLUMNS_EQUAL(expected, result->view());

Fails with:

/raid/wangm/dev/rapids/cudf/cpp/tests/utilities/column_utilities.cu:506: Failure
Failed
first difference: lhs[0] = 14.123, rhs[0] = -0.050
Google Test trace:
/raid/wangm/dev/rapids/cudf/cpp/tests/unary/cast_tests.cpp:862:  <--  line of failure

Expected behavior
See expected variable in example above.

Environment overview (please complete the following information)

  • Environment location: Rapids-Compose
  • Method of cuDF install: Rapids-Compose

Environment details

Click here to see environment details
 **git***
 commit f207f98684fd8103ceba9285d66dec0b6242d07e (HEAD -> 8853, branch-21.10)
 Author: David Wendt <[email protected]>
 Date:   Fri Aug 6 14:15:48 2021 -0400
 
 Support bracket syntax for cudf::strings::replace_with_backrefs group index values (#8841)
 
 Closes #8816
 
 The current `\d` syntax for the replacement template parameter will fail if a number immediately follows the index pattern as described in #8816. This PR adds support for the `${d}` pattern but only if the `\d` pattern is not found in the replacement string. This should minimize breaking any current templates already being used with this API.
 
 Authors:
 - David Wendt (https://github.com/davidwendt)
 
 Approvers:
 - Vukasin Milovanovic (https://github.com/vuule)
 - Christopher Harris (https://github.com/cwharris)
 
 URL: https://github.com/rapidsai/cudf/pull/8841
 **git submodules***
 
 ***OS Information***
 DISTRIB_ID=Ubuntu
 DISTRIB_RELEASE=18.04
 DISTRIB_CODENAME=bionic
 DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS"
 NAME="Ubuntu"
 VERSION="18.04.5 LTS (Bionic Beaver)"
 ID=ubuntu
 ID_LIKE=debian
 PRETTY_NAME="Ubuntu 18.04.5 LTS"
 VERSION_ID="18.04"
 HOME_URL="https://www.ubuntu.com/"
 SUPPORT_URL="https://help.ubuntu.com/"
 BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
 PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
 VERSION_CODENAME=bionic
 UBUNTU_CODENAME=bionic
 Linux dgx06 4.15.0-76-generic #86-Ubuntu SMP Fri Jan 17 17:24:28 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
 
 ***GPU Information***
 Mon Aug  9 12:06:54 2021
 +-----------------------------------------------------------------------------+
 | NVIDIA-SMI 460.39       Driver Version: 460.39       CUDA Version: 11.2     |
 |-------------------------------+----------------------+----------------------+
 | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
 | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
 |                               |                      |               MIG M. |
 |===============================+======================+======================|
 |   0  Tesla V100-SXM2...  On   | 00000000:85:00.0 Off |                    0 |
 | N/A   32C    P0    56W / 300W |    610MiB / 32510MiB |      0%      Default |
 |                               |                      |                  N/A |
 +-------------------------------+----------------------+----------------------+
 
 +-----------------------------------------------------------------------------+
 | Processes:                                                                  |
 |  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
 |        ID   ID                                                   Usage      |
 |=============================================================================|
 +-----------------------------------------------------------------------------+
 
 ***CPU***
 Architecture:        x86_64
 CPU op-mode(s):      32-bit, 64-bit
 Byte Order:          Little Endian
 CPU(s):              80
 On-line CPU(s) list: 0-79
 Thread(s) per core:  2
 Core(s) per socket:  20
 Socket(s):           2
 NUMA node(s):        2
 Vendor ID:           GenuineIntel
 CPU family:          6
 Model:               79
 Model name:          Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz
 Stepping:            1
 CPU MHz:             1839.133
 CPU max MHz:         3600.0000
 CPU min MHz:         1200.0000
 BogoMIPS:            4389.88
 Virtualization:      VT-x
 L1d cache:           32K
 L1i cache:           32K
 L2 cache:            256K
 L3 cache:            51200K
 NUMA node0 CPU(s):   0-19,40-59
 NUMA node1 CPU(s):   20-39,60-79
 Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowprefetch cpuid_fault epb cat_l3 cdp_l3 invpcid_single pti intel_ppin ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm cqm rdt_a rdseed adx smap intel_pt xsaveopt cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local dtherm ida arat pln pts md_clear flush_l1d
 
 ***CMake***
 /raid/wangm/dev/rapids/compose/etc/conda/cuda_11.2.0/envs/rapids/bin/cmake
 cmake version 3.21.0
 
 CMake suite maintained and supported by Kitware (kitware.com/cmake).
 
 ***g++***
 /usr/local/bin/g++
 g++ (Ubuntu 9.4.0-1ubuntu1~18.04) 9.4.0
 Copyright (C) 2019 Free Software Foundation, Inc.
 This is free software; see the source for copying conditions.  There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
 
 
 ***nvcc***
 /usr/local/bin/nvcc
 nvcc: NVIDIA (R) Cuda compiler driver
 Copyright (c) 2005-2021 NVIDIA Corporation
 Built on Sun_Feb_14_21:12:58_PST_2021
 Cuda compilation tools, release 11.2, V11.2.152
 Build cuda_11.2.r11.2/compiler.29618528_0
 
 ***Python***
 /raid/wangm/dev/rapids/compose/etc/conda/cuda_11.2.0/envs/rapids/bin/python
 Python 3.7.10
 
 ***Environment Variables***
 PATH                            : /raid/wangm/dev/rapids/compose/etc/conda/cuda_11.2.0/envs/rapids/bin:/raid/wangm/dev/rapids/compose/etc/conda/cuda_11.2.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/cuda/bin
 LD_LIBRARY_PATH                 : /raid/wangm/dev/rapids/compose/etc/conda/cuda_11.2.0/envs/rapids/lib:/raid/wangm/dev/rapids/compose/etc/conda/cuda_11.2.0/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/i386-linux-gnu:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64:/usr/local/lib:/raid/wangm/dev/rapids/rmm/build/release:/raid/wangm/dev/rapids/cudf/cpp/build/release:/raid/wangm/dev/rapids/raft/cpp/build/release:/raid/wangm/dev/rapids/cuml/cpp/build/release:/raid/wangm/dev/rapids/cugraph/cpp/build/release:/raid/wangm/dev/rapids/cuspatial/cpp/build/release
 NUMBAPRO_NVVM                   :
 NUMBAPRO_LIBDEVICE              :
 CONDA_PREFIX                    : /raid/wangm/dev/rapids/compose/etc/conda/cuda_11.2.0/envs/rapids
 PYTHON_PATH                     :
 
 ***conda packages***
 /raid/wangm/dev/rapids/compose/etc/conda/cuda_11.2.0/bin/conda
 # packages in environment at /raid/wangm/dev/rapids/compose/etc/conda/cuda_11.2.0/envs/rapids:
 #
 # Name                    Version                   Build  Channel
 _libgcc_mutex             0.1                 conda_forge    conda-forge
 _openmp_mutex             4.5                      1_llvm    conda-forge
 abseil-cpp                20210324.2           h9c3ff4c_0    conda-forge
 alabaster                 0.7.12                     py_0    conda-forge
 appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
 argon2-cffi               20.1.0           py37h5e8e339_2    conda-forge
 arrow-cpp                 4.0.1           py37h20531c0_4_cuda    conda-forge
 arrow-cpp-proc            3.0.0                      cuda    conda-forge
 async_generator           1.10                       py_0    conda-forge
 attrs                     21.2.0             pyhd8ed1ab_0    conda-forge
 aws-c-cal                 0.5.11               h95a6274_0    conda-forge
 aws-c-common              0.6.2                h7f98852_0    conda-forge
 aws-c-event-stream        0.2.7               h3541f99_13    conda-forge
 aws-c-io                  0.10.5               hfb6a706_0    conda-forge
 aws-checksums             0.1.11               ha31a3da_7    conda-forge
 aws-sdk-cpp               1.8.186              hb4091e7_3    conda-forge
 babel                     2.9.1              pyh44b312d_0    conda-forge
 backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
 backports                 1.0                        py_2    conda-forge
 backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
 binutils_impl_linux-64    2.36.1               h193b22a_1    conda-forge
 black                     19.10b0                  py37_0    conda-forge
 bleach                    3.3.0              pyh44b312d_0    conda-forge
 bokeh                     2.3.3            py37h89c1867_0    conda-forge
 brotlipy                  0.7.0           py37h5e8e339_1001    conda-forge
 bzip2                     1.0.8                h7f98852_4    conda-forge
 c-ares                    1.17.1               h7f98852_1    conda-forge
 ca-certificates           2021.5.30            ha878542_0    conda-forge
 cachetools                4.2.2              pyhd8ed1ab_0    conda-forge
 certifi                   2021.5.30        py37h89c1867_0    conda-forge
 cffi                      1.14.6           py37hc58025e_0    conda-forge
 cfgv                      3.3.0              pyhd8ed1ab_0    conda-forge
 chardet                   4.0.0            py37h89c1867_1    conda-forge
 clang                     11.0.0               ha770c72_2    conda-forge
 clang-11                  11.0.0          default_ha5c780c_2    conda-forge
 clang-tools               11.0.0          default_ha5c780c_2    conda-forge
 clangxx                   11.0.0          default_ha5c780c_2    conda-forge
 click                     8.0.1            py37h89c1867_0    conda-forge
 cloudpickle               1.6.0                      py_0    conda-forge
 cmake                     3.21.0               h8897547_0    conda-forge
 cmake-format              0.6.11             pyh9f0ad1d_0    conda-forge
 cmake_setuptools          0.1.3                      py_0    rapidsai
 colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
 commonmark                0.9.1                      py_0    conda-forge
 cryptography              3.4.7            py37h5d9358c_0    conda-forge
 cudatoolkit               11.2.0               h73cb219_8    nvidia
 cupy                      9.2.0            py37h3c5eebb_0    conda-forge
 cython                    0.29.24          py37hcd2ae1e_0    conda-forge
 cytoolz                   0.11.0           py37h5e8e339_3    conda-forge
 dask                      2021.7.1                 pypi_0    pypi
 dataclasses               0.8                pyhc8e2a94_1    conda-forge
 decorator                 5.0.9              pyhd8ed1ab_0    conda-forge
 defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
 distlib                   0.3.2              pyhd8ed1ab_0    conda-forge
 distributed               2021.6.2+37.g04b6be4f          pypi_0    pypi
 dlpack                    0.5                  h9c3ff4c_0    conda-forge
 docutils                  0.16             py37h89c1867_3    conda-forge
 double-conversion         3.1.5                h9c3ff4c_2    conda-forge
 editdistance-s            1.0.0            py37h2527ec5_1    conda-forge
 entrypoints               0.3             py37hc8dfbb8_1002    conda-forge
 execnet                   1.9.0              pyhd8ed1ab_0    conda-forge
 expat                     2.4.1                h9c3ff4c_0    conda-forge
 fastavro                  1.4.4            py37h5e8e339_0    conda-forge
 fastrlock                 0.6              py37hcd2ae1e_1    conda-forge
 filelock                  3.0.12             pyh9f0ad1d_0    conda-forge
 flake8                    3.8.3                      py_1    conda-forge
 freetype                  2.10.4               h0708190_1    conda-forge
 fsspec                    2021.7.0           pyhd8ed1ab_0    conda-forge
 future                    0.18.2           py37h89c1867_3    conda-forge
 gcc_impl_linux-64         9.3.0               h70c0ae5_19    conda-forge
 gflags                    2.2.2             he1b5a44_1004    conda-forge
 glog                      0.5.0                h48cff8f_0    conda-forge
 gmp                       6.2.1                h58526e2_0    conda-forge
 grpc-cpp                  1.38.1               h36ce80c_0    conda-forge
 heapdict                  1.0.1                      py_0    conda-forge
 huggingface_hub           0.0.13             pyhd8ed1ab_0    conda-forge
 hypothesis                6.14.3             pyhd8ed1ab_0    conda-forge
 identify                  2.2.10             pyhd8ed1ab_0    conda-forge
 idna                      2.10               pyh9f0ad1d_0    conda-forge
 imagesize                 1.2.0                      py_0    conda-forge
 importlib-metadata        4.6.1            py37h89c1867_0    conda-forge
 importlib_metadata        4.6.1                hd8ed1ab_0    conda-forge
 iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
 ipykernel                 5.5.5            py37h085eea5_0    conda-forge
 ipython                   7.25.0           py37h085eea5_1    conda-forge
 ipython_genutils          0.2.0                      py_1    conda-forge
 isort                     5.6.4                      py_0    conda-forge
 jbig                      2.1               h7f98852_2003    conda-forge
 jedi                      0.18.0           py37h89c1867_2    conda-forge
 jinja2                    3.0.1              pyhd8ed1ab_0    conda-forge
 joblib                    1.0.1              pyhd8ed1ab_0    conda-forge
 jpeg                      9d                   h36c2ea0_0    conda-forge
 jsonschema                3.2.0            py37hc8dfbb8_1    conda-forge
 jupyter_client            6.1.12             pyhd8ed1ab_0    conda-forge
 jupyter_core              4.7.1            py37h89c1867_0    conda-forge
 jupyterlab_pygments       0.1.2              pyh9f0ad1d_0    conda-forge
 kernel-headers_linux-64   2.6.32              h77966d4_13    conda-forge
 krb5                      1.19.1               hcc1bbae_0    conda-forge
 lcms2                     2.12                 hddcbb42_0    conda-forge
 ld_impl_linux-64          2.36.1               hea4e1c9_1    conda-forge
 lerc                      2.2.1                h9c3ff4c_0    conda-forge
 libblas                   3.9.0                     8_mkl    conda-forge
 libbrotlicommon           1.0.9                h7f98852_5    conda-forge
 libbrotlidec              1.0.9                h7f98852_5    conda-forge
 libbrotlienc              1.0.9                h7f98852_5    conda-forge
 libcblas                  3.9.0                     8_mkl    conda-forge
 libclang-cpp11            11.0.0          default_ha5c780c_2    conda-forge
 libcurl                   7.77.0               h2574ce0_0    conda-forge
 libdeflate                1.7                  h7f98852_5    conda-forge
 libedit                   3.1.20191231         he28a2e2_2    conda-forge
 libev                     4.33                 h516909a_1    conda-forge
 libevent                  2.1.10               hcdb4288_3    conda-forge
 libffi                    3.3                  h58526e2_2    conda-forge
 libgcc-devel_linux-64     9.3.0               h7864c58_19    conda-forge
 libgcc-ng                 9.3.0               h2828fa1_19    conda-forge
 libgomp                   9.3.0               h2828fa1_19    conda-forge
 liblapack                 3.9.0                     8_mkl    conda-forge
 libllvm10                 10.0.1               he513fc3_3    conda-forge
 libllvm11                 11.0.1               hf817b99_0    conda-forge
 libnghttp2                1.43.0               h812cca2_0    conda-forge
 libpng                    1.6.37               h21135ba_2    conda-forge
 libprotobuf               3.16.0               h780b84a_0    conda-forge
 libsodium                 1.0.18               h36c2ea0_1    conda-forge
 libssh2                   1.9.0                ha56f1ee_6    conda-forge
 libstdcxx-ng              9.3.0               h6de172a_19    conda-forge
 libthrift                 0.14.2               he6d91bd_1    conda-forge
 libtiff                   4.3.0                hf544144_1    conda-forge
 libutf8proc               2.6.1                h7f98852_0    conda-forge
 libuv                     1.41.1               h7f98852_0    conda-forge
 libwebp-base              1.2.0                h7f98852_2    conda-forge
 llvm-openmp               11.1.0               h4bd325d_1    conda-forge
 llvmlite                  0.36.0           py37h9d7f4d0_0    conda-forge
 locket                    0.2.0                      py_2    conda-forge
 lz4-c                     1.9.3                h9c3ff4c_0    conda-forge
 markdown                  3.3.4              pyhd8ed1ab_0    conda-forge
 markupsafe                2.0.1            py37h5e8e339_0    conda-forge
 matplotlib-inline         0.1.2              pyhd8ed1ab_2    conda-forge
 mccabe                    0.6.1                      py_1    conda-forge
 mimesis                   4.0.0              pyh9f0ad1d_0    conda-forge
 mistune                   0.8.4           py37h5e8e339_1004    conda-forge
 mkl                       2020.4             h726a3e6_304    conda-forge
 more-itertools            8.8.0              pyhd8ed1ab_0    conda-forge
 msgpack-python            1.0.2            py37h2527ec5_1    conda-forge
 mypy                      0.782                      py_0    conda-forge
 mypy_extensions           0.4.3            py37h89c1867_3    conda-forge
 nbclient                  0.5.3              pyhd8ed1ab_0    conda-forge
 nbconvert                 6.1.0            py37h89c1867_0    conda-forge
 nbformat                  5.1.3              pyhd8ed1ab_0    conda-forge
 nbsphinx                  0.8.6              pyhd8ed1ab_1    conda-forge
 ncurses                   6.2                  h58526e2_4    conda-forge
 nest-asyncio              1.5.1              pyhd8ed1ab_0    conda-forge
 ninja                     1.10.2               h4bd325d_0    conda-forge
 nodeenv                   1.6.0              pyhd8ed1ab_0    conda-forge
 notebook                  6.4.0              pyha770c72_0    conda-forge
 numba                     0.53.1           py37hb11d6e1_1    conda-forge
 numpy                     1.21.1           py37h038b26d_0    conda-forge
 numpydoc                  1.1.0                      py_1    conda-forge
 nvtx                      0.2.3            py37h5e8e339_0    conda-forge
 olefile                   0.46               pyh9f0ad1d_1    conda-forge
 openjpeg                  2.4.0                hb52868f_1    conda-forge
 openssl                   1.1.1k               h7f98852_0    conda-forge
 orc                       1.6.9                h58a87f1_0    conda-forge
 packaging                 21.0               pyhd8ed1ab_0    conda-forge
 pandas                    1.2.5            py37h219a48f_0    conda-forge
 pandoc                    1.19.2                        0    conda-forge
 pandocfilters             1.4.2                      py_1    conda-forge
 parquet-cpp               1.5.1                         1    conda-forge
 parso                     0.8.2              pyhd8ed1ab_0    conda-forge
 partd                     1.2.0              pyhd8ed1ab_0    conda-forge
 pathspec                  0.8.1              pyhd3deb0d_0    conda-forge
 pexpect                   4.8.0            py37hc8dfbb8_1    conda-forge
 pickleshare               0.7.5           py37hc8dfbb8_1002    conda-forge
 pillow                    8.3.1            py37h0f21c89_0    conda-forge
 pip                       21.1.3             pyhd8ed1ab_0    conda-forge
 pluggy                    0.13.1           py37h89c1867_4    conda-forge
 pre-commit                2.13.0           py37h89c1867_0    conda-forge
 pre_commit                2.13.0               hd8ed1ab_0    conda-forge
 prometheus_client         0.11.0             pyhd8ed1ab_0    conda-forge
 prompt-toolkit            3.0.19             pyha770c72_0    conda-forge
 protobuf                  3.16.0           py37hcd2ae1e_0    conda-forge
 psutil                    5.8.0            py37h5e8e339_1    conda-forge
 ptvsd                     4.3.2                    pypi_0    pypi
 ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
 py                        1.10.0             pyhd3deb0d_0    conda-forge
 py-cpuinfo                8.0.0              pyhd8ed1ab_0    conda-forge
 pyarrow                   4.0.1           py37hb63ea2f_4_cuda    conda-forge
 pycodestyle               2.6.0              pyh9f0ad1d_0    conda-forge
 pycparser                 2.20               pyh9f0ad1d_2    conda-forge
 pyflakes                  2.2.0              pyh9f0ad1d_0    conda-forge
 pygments                  2.9.0              pyhd8ed1ab_0    conda-forge
 pyopenssl                 20.0.1             pyhd8ed1ab_0    conda-forge
 pyorc                     0.4.0                    pypi_0    pypi
 pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
 pyrsistent                0.17.3           py37h5e8e339_2    conda-forge
 pysocks                   1.7.1            py37h89c1867_3    conda-forge
 pytest                    6.2.4            py37h89c1867_0    conda-forge
 pytest-benchmark          3.4.1              pyhd8ed1ab_0    conda-forge
 pytest-forked             1.3.0              pyhd3deb0d_0    conda-forge
 pytest-xdist              2.3.0              pyhd8ed1ab_0    conda-forge
 python                    3.7.10          hffdb5ce_100_cpython    conda-forge
 python-dateutil           2.8.1                      py_0    conda-forge
 python_abi                3.7                     2_cp37m    conda-forge
 pytorch                   1.7.1           cpu_py37hf1c21f6_2    conda-forge
 pytz                      2021.1             pyhd8ed1ab_0    conda-forge
 pyyaml                    5.4.1            py37h5e8e339_0    conda-forge
 pyzmq                     22.1.0           py37h336d617_0    conda-forge
 rapidjson                 1.1.0             he1b5a44_1002    conda-forge
 re2                       2021.06.01           h9c3ff4c_0    conda-forge
 readline                  8.1                  h46c0cb4_0    conda-forge
 recommonmark              0.7.1              pyhd8ed1ab_0    conda-forge
 regex                     2021.7.6         py37h5e8e339_0    conda-forge
 requests                  2.25.1             pyhd3deb0d_0    conda-forge
 rhash                     1.4.1                h7f98852_0    conda-forge
 s2n                       1.0.10               h9b69904_0    conda-forge
 sacremoses                0.0.43             pyh9f0ad1d_0    conda-forge
 send2trash                1.7.1              pyhd8ed1ab_0    conda-forge
 setuptools                49.6.0           py37h89c1867_3    conda-forge
 six                       1.16.0             pyh6c4a22f_0    conda-forge
 snappy                    1.1.8                he1b5a44_3    conda-forge
 snowballstemmer           2.1.0              pyhd8ed1ab_0    conda-forge
 sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
 spdlog                    1.8.5                h4bd325d_0    conda-forge
 sphinx                    4.1.1              pyh6c4a22f_0    conda-forge
 sphinx-copybutton         0.4.0              pyhd8ed1ab_0    conda-forge
 sphinx-markdown-tables    0.0.15             pyhd3deb0d_0    conda-forge
 sphinx_rtd_theme          0.5.2              pyhd8ed1ab_1    conda-forge
 sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
 sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
 sphinxcontrib-htmlhelp    2.0.0              pyhd8ed1ab_0    conda-forge
 sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
 sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
 sphinxcontrib-serializinghtml 1.1.5              pyhd8ed1ab_0    conda-forge
 sphinxcontrib-websupport  1.2.4              pyh9f0ad1d_0    conda-forge
 sqlite                    3.36.0               h9cd32fc_0    conda-forge
 streamz                   0.6.2              pyh44b312d_0    conda-forge
 sysroot_linux-64          2.12                h77966d4_13    conda-forge
 tblib                     1.7.0              pyhd8ed1ab_0    conda-forge
 terminado                 0.10.1           py37h89c1867_0    conda-forge
 testpath                  0.5.0              pyhd8ed1ab_0    conda-forge
 tk                        8.6.10               h21135ba_1    conda-forge
 tokenizers                0.10.1           py37hcb7a40c_0    conda-forge
 toml                      0.10.2             pyhd8ed1ab_0    conda-forge
 toolz                     0.11.1                     py_0    conda-forge
 tornado                   6.1              py37h5e8e339_1    conda-forge
 tqdm                      4.61.2             pyhd8ed1ab_1    conda-forge
 traitlets                 5.0.5                      py_0    conda-forge
 transformers              4.9.0              pyhd8ed1ab_0    conda-forge
 typed-ast                 1.4.3            py37h5e8e339_0    conda-forge
 typing-extensions         3.10.0.0             hd8ed1ab_0    conda-forge
 typing_extensions         3.10.0.0           pyha770c72_0    conda-forge
 urllib3                   1.26.6             pyhd8ed1ab_0    conda-forge
 virtualenv                20.4.7           py37h89c1867_0    conda-forge
 wcwidth                   0.2.5              pyh9f0ad1d_2    conda-forge
 webencodings              0.5.1                      py_1    conda-forge
 wheel                     0.36.2             pyhd3deb0d_0    conda-forge
 xz                        5.2.5                h516909a_1    conda-forge
 yaml                      0.2.5                h516909a_0    conda-forge
 zeromq                    4.3.4                h9c3ff4c_0    conda-forge
 zict                      2.0.0                      py_0    conda-forge
 zipp                      3.5.0              pyhd8ed1ab_0    conda-forge
 zlib                      1.2.11            h516909a_1010    conda-forge
 zstd                      1.5.0                ha95c52a_0    conda-forge

@isVoid isVoid added bug Something isn't working Needs Triage Need team to review and classify labels Aug 9, 2021
@isVoid isVoid added the libcudf Affects libcudf (C++/CUDA) code. label Aug 9, 2021
@isVoid
Copy link
Contributor Author

isVoid commented Aug 9, 2021

Possible cause: int64_t representation of 141230900000L overflows at int32_t:

>>> np.int64(141230900000).astype(np.int32)
-503020768

@isVoid isVoid mentioned this issue Aug 9, 2021
Merged
@harrism
Copy link
Member

harrism commented Aug 11, 2021 •
edited
Loading

If the value is too big to store in 32 bits, then I think overflow is expected. I don't think this would be a bug in that case.

You could use a binary op to compare to std::numeric_limits<int32_t>::max() to detect this before casting.

CC @codereport

@beckernick beckernick added feature request New feature or request and removed bug Something isn't working Needs Triage Need team to review and classify labels Aug 25, 2021
@github-actions
Copy link

This issue has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this issue if no further response or action is needed. Otherwise, please respond with a comment indicating any updates or changes to the original issue and/or confirm this issue still needs to be addressed. This issue will be labeled inactive-90d if there is no activity in the next 60 days.

@revans2
Copy link
Contributor

revans2 commented Nov 16, 2021

I think this has been fixed. But it might depend on some of the decimal128 work. I don't see this kind of issue currently.

@codereport
Copy link
Contributor

codereport commented Nov 24, 2021 •
edited
Loading

I think this has been fixed. But it might depend on some of the decimal128 work. I don't see this kind of issue currently.

This hasn't been "fixed" per se. As @harrism mentioned above, the issue is just that the int32_t is not large enough to store the int64_t value being casted from. The way cudf::cast works for decimal types with different underlying storage that it first does a cast from intXX_t to intYY_t and then does the rescaling. Technically, when YY < XX we could to the rescaling first and that could avoid some overflowing. I will experiment with this tomorrow and let you know.

Also, decimal128 doesn't fix this because whether the value above is stored as an int64_t or an int128_t, it will still overflow when converted to int32_t.

@codereport codereport self-assigned this Nov 24, 2021
@codereport codereport mentioned this issue Nov 24, 2021
Merged
@rapids-bot rapids-bot bot closed this as completed in #9772 Dec 1, 2021
rapids-bot bot pushed a commit that referenced this issue Dec 1, 2021
@codereport
Avoid overflow for fixed_point cudf::cast and performance optimiz…
677e632 
…ation (#9772)

This resolves #9000.

When using `cudf::cast` for a wider decimal type to a narrower decimal type, you can overflow. This PR modifies the code path for this specific use case so that the "rescale" happens for the type cast. A small perf improvement was added when you have identical scales to avoid rescaling.

CI depends on #9766

Authors:
  - Conor Hoekstra (https://github.com/codereport)

Approvers:
  - Nghia Truong (https://github.com/ttnghia)
  - Mike Wilson (https://github.com/hyperbolic2346)
  - Vukasin Milovanovic (https://github.com/vuule)

URL: #9772
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@codereport codereport

Labels
feature request New feature or request libcudf Affects libcudf (C++/CUDA) code.
Projects
None yet
Milestone
Decimal data type and operations
Development

Successfully merging a pull request may close this issue.

Avoid overflow for fixed_point cudf::cast and performance optimization codereport/cudf
5 participants
@harrism @revans2 @beckernick @isVoid @codereport