Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrongly handling non-numeric values of double type during the conversion from Delta #524

Closed
3 of 4 tasks
emilie-wang opened this issue Aug 23, 2024 · 0 comments · Fixed by #526
Closed
3 of 4 tasks
Labels
bug Something isn't working

Comments

@emilie-wang
Copy link

Search before asking

  • I had searched in the issues and found no similar issues.

Please describe the bug 🐞

This error happened during the translation of column stats from Delta → internal table →Iceberg. There is an edge case, when the double type field’s value is non-numeric like "NaN" or "Infinity", the value type become String when processing the snapshot from Delta source.

...INFO org.apache.xtable.iceberg.IcebergColumnStatsConverter:76 - name=weight, fieldType=double, valueType=class java.lang.String, getMinValue=-Infinity 
...INFO org.apache.xtable.iceberg.IcebergColumnStatsConverter:80 - name=weight, fieldType=double, valueType=class java.lang.String, getMaxValue=NaN

Are you willing to submit PR?

  • I am willing to submit a PR!
  • I am willing to submit a PR but need help getting started!

Code of Conduct

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
1 participant