Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure statistics predicate only for base column in delta connector #17449

Merged
merged 1 commit into from
May 15, 2023

Conversation

krvikash
Copy link
Contributor

Follow up of #17085 (comment)

@cla-bot cla-bot bot added the cla-signed label May 10, 2023
@github-actions github-actions bot added the delta-lake Delta Lake connector label May 10, 2023
@krvikash krvikash requested review from findepi and findinpath May 11, 2023 10:11
@findepi findepi requested review from ebyhr and alexjo2144 and removed request for findepi May 12, 2023 09:18
@ebyhr
Copy link
Member

ebyhr commented May 12, 2023

Create statistics predicate only for base column in delta connector

We didn't create statistics predicate for complex types even without this change because DeltaLakeMetadata#canUseInPredicate returns false for such types. Is my understanding wrong? The current commit title is a little confusing for me. It would be nice to describe the motivation in the commit body.

@findinpath findinpath self-requested a review May 15, 2023 05:02
@findinpath
Copy link
Contributor

Undoing my approval to wait until the comment #17449 (comment) gets clarified.

tableHandle.getNonPartitionConstraint() can include the dereferenced field
if dereference field is used as a predicate. Since stats for dereferenced field
is not supported, so filter those fields before creating statistics predicate.
@krvikash krvikash force-pushed the trino-17085-follow-up branch from 8f8b5e0 to 49fede7 Compare May 15, 2023 05:38
@krvikash krvikash changed the title Create statistics predicate only for base column in delta connector Ensure statistics predicate only for base column in delta connector May 15, 2023
@krvikash
Copy link
Contributor Author

Thanks, @ebyhr | @findinpath for the review. I have updated the commit message and put the description in the commit body.

@ebyhr ebyhr merged commit db43ee8 into trinodb:master May 15, 2023
@github-actions github-actions bot added this to the 418 milestone May 15, 2023
@krvikash krvikash deleted the trino-17085-follow-up branch May 15, 2023 06:31
@colebow
Copy link
Member

colebow commented May 17, 2023

Does this need release notes? cc @ebyhr

@krvikash krvikash added the no-release-notes This pull request does not require release notes entry label May 17, 2023
@krvikash
Copy link
Contributor Author

No, It does not require release notes. As we didn't create statistics predicate for complex types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed delta-lake Delta Lake connector no-release-notes This pull request does not require release notes entry
Development

Successfully merging this pull request may close these issues.

4 participants