Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Iceberg table statistics are non-deterministic #9716

Closed
findepi opened this issue Oct 20, 2021 · 1 comment · Fixed by #9906
Closed

Iceberg table statistics are non-deterministic #9716

findepi opened this issue Oct 20, 2021 · 1 comment · Fixed by #9906
Assignees
@alexjo2144
Labels
bug Something isn't working maintenance Project maintenance task
Milestone
369

Comments

@findepi
Copy link
Member

findepi commented Oct 20, 2021 •
edited
Loading

Currently the results seem to be dependent on the ordering of files.
Also, missing information is not properly handled.

  • see TODOs in the code (e.g. being added in Fix NPE in Iceberg stats reader #9719)
  • add explicit tests for metadata with missing file stats (currently this depends on the writer, and Trino writer changes)
@findepi findepi added bug Something isn't working maintenance Project maintenance task labels Oct 20, 2021
@findepi
Copy link
Member Author

findepi commented Oct 20, 2021

cc @losipiuk @alexjo2144

This was referenced Oct 20, 2021
Merged
Closed
@findepi findepi changed the title Revamp Iceberg statistics reporting Iceberg table statistics are non-deterministic Oct 25, 2021
This was referenced Oct 28, 2021
Closed
Merged
@findepi findepi mentioned this issue Nov 4, 2021
Merged
@alexjo2144 alexjo2144 mentioned this issue Nov 8, 2021
Merged
@findepi findepi added this to the 369 milestone Jan 21, 2022
@alexjo2144 alexjo2144 mentioned this issue Jan 21, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alexjo2144 alexjo2144

Labels
bug Something isn't working maintenance Project maintenance task
Milestone
369
Development

Successfully merging a pull request may close this issue.

Refactor Iceberg table statistics to be deterministic alexjo2144/trino
2 participants
@findepi @alexjo2144