Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deserialize partition value by physical name for Delta connector #16388

Merged
merged 1 commit into from
Mar 13, 2023

Conversation

SchutteJan
Copy link
Member

Description

When supplying a partition predicate on a Delta table which has column mapping set to 'name' to the DeltaLakeSplitManager the values of the partition predicate are not resolved to the physical name but the name of the column.

Additional context and related issues

Without the above patch, any query with a where clause on the partition column will fail with the following stack-trace:
https://pastebin.com/PcK3WG8P

Release notes

( ) This is not user-visible or docs only and no release notes are required.
(x) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text:

@cla-bot
Copy link

cla-bot bot commented Mar 6, 2023

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Jan Schutte.
This is most likely caused by a git client misconfiguration; please make sure to:

  1. check if your git client is configured with an email to sign commits git config --list | grep email
  2. If not, set it up using git config --global user.email [email protected]
  3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

@github-actions github-actions bot added the delta-lake Delta Lake connector label Mar 6, 2023
@ebyhr ebyhr self-requested a review March 6, 2023 22:52
@SchutteJan SchutteJan force-pushed the physical-name-delta-reader branch from 6e747b0 to 12a6c5f Compare March 7, 2023 10:05
@cla-bot
Copy link

cla-bot bot commented Mar 7, 2023

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@ebyhr
Copy link
Member

ebyhr commented Mar 7, 2023

@cla-bot check

@cla-bot cla-bot bot added the cla-signed label Mar 7, 2023
@cla-bot
Copy link

cla-bot bot commented Mar 7, 2023

The cla-bot has been summoned, and re-checked this pull request!

@ebyhr ebyhr force-pushed the physical-name-delta-reader branch from 12a6c5f to 40760f5 Compare March 7, 2023 23:18
@SchutteJan
Copy link
Member Author

@ebyhr Is this PR ready for merging or is there still something missing?

@ebyhr
Copy link
Member

ebyhr commented Mar 10, 2023

/test-with-secrets sha=40760f5ff40a0cc1a41228e2e3d5b73389222b95

@github-actions
Copy link

The CI workflow run with tests that require additional secrets finished as failure: https://github.com/trinodb/trino/actions/runs/4382799260

@SchutteJan SchutteJan force-pushed the physical-name-delta-reader branch from 40760f5 to caf0df0 Compare March 10, 2023 11:30
@ebyhr
Copy link
Member

ebyhr commented Mar 10, 2023

/test-with-secrets sha=caf0df00ba179e6f5c0077ac42d8f7751246130d

@ebyhr ebyhr merged commit 329b728 into trinodb:master Mar 13, 2023
@github-actions github-actions bot added this to the 411 milestone Mar 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed delta-lake Delta Lake connector
Development

Successfully merging this pull request may close these issues.

2 participants