[BUG] Datetime data is being written incorrectly by orc writer #7355
Assignees
Comments
galipremsagar
added
bug
|
The nanoseconds (coded separately from seconds) are exactly off by 0x20000000 (1<<29) |
rapids-bot bot
pushed a commit
that referenced
this issue
Mar 15, 2021
Closes #7355 Use 64 bit variables/buffers to handle nanosecond values since nanosecond encode can overflow a 32bit value in some cases. Removed the overloaded `intrle_minmax` function, using templated `numeric_limits` functions instead (the alternative was to add another overload). Performance impact evaluation pending, but this fix seems unavoidable regardless of the impact. Authors: - Vukasin Milovanovic (@vuule) Approvers: - GALI PREM SAGAR (@galipremsagar) - Devavret Makkar (@devavret) - Kumar Aatish (@kaatish) URL: #7581
hyperbolic2346
pushed a commit
to hyperbolic2346/cudf
that referenced
this issue
Mar 25, 2021
Closes rapidsai#7355 Use 64 bit variables/buffers to handle nanosecond values since nanosecond encode can overflow a 32bit value in some cases. Removed the overloaded `intrle_minmax` function, using templated `numeric_limits` functions instead (the alternative was to add another overload). Performance impact evaluation pending, but this fix seems unavoidable regardless of the impact. Authors: - Vukasin Milovanovic (@vuule) Approvers: - GALI PREM SAGAR (@galipremsagar) - Devavret Makkar (@devavret) - Kumar Aatish (@kaatish) URL: rapidsai#7581
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
For a certain range of values, the DateTime/timestamp data is not being written correctly into orc file. Round-tripping these values exhibit corruption in sub-second data.
Steps/Code to reproduce bug
Expected behavior
Environment overview (please complete the following information)
Environment details
Please run and paste the output of the
cudf/print_env.shscript here, to gather any other relevant environment detailsClick here to see environment details
Additional context
Surfaced while running fuzz tests: #6001
The text was updated successfully, but these errors were encountered: