You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
mythrocks
changed the title
SPARK-10177: Timestamps read incorrectly. Refer to the test in ParquetHiveCompatibility.scala#L97
Add test for the timestamp error case described in SPARK-10177
Jul 11, 2023
This seems like the bug was in DateTimeUtils as evident from the comment here. Considering that is this still a valid test case that we need in parquet_write_test.py?
Hmm. In hindsight, probably not top priority. I'd still add it, just in case there's a similar bug in CUDF, since it can be tested with a single timestamp record.
Apache JIRA: https://issues.apache.org/jira/browse/SPARK-10177
Refer to
ParquetHiveCompatibilitySuite.scala#L97
.We just need to add a PySpark test to check that the following data is written/read the same on
spark-rapids
and Apache Spark:The text was updated successfully, but these errors were encountered: