Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't overwrite reader and writer versions during COMMENT and ADD COLUMN in Delta #14612

Merged
merged 1 commit into from
Oct 14, 2022

Conversation

ebyhr
Copy link
Member

@ebyhr ebyhr commented Oct 13, 2022

Description

Fixes #14611

Non-technical explanation

Release notes

(x) Release notes are required, with the following suggested text:

# Delta Lake
* Don't overwrite reader and writer versions when executing `COMMENT` and `ALTER TABLE ... ADD COLUMN` statements. ({issue}`14611`)

@ebyhr ebyhr requested a review from findepi October 13, 2022 06:46
@@ -263,4 +263,32 @@ public void testTrinoAlterTablePreservesTableMetadata()
onTrino().executeQuery("DROP TABLE delta.default." + tableName);
}
}

@Test(groups = {DELTA_LAKE_DATABRICKS, DELTA_LAKE_OSS, PROFILE_SPECIFIC_TESTS})
public void testTrinoAlterTablePreservesReaderAndWriterVersions()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we good for inserts, checkpoints, etc.?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those operations look fine. Added some statements to this test.

@alexjo2144
Copy link
Member

Can we put the minWriterVersion and minReaderVersion in the DeltaLakeTableHandle?

Either way, thanks for spotting this

@ebyhr ebyhr force-pushed the ebi/delta-reader-writer-versions branch from 72524e2 to 5ab5ab4 Compare October 14, 2022 00:56
@ebyhr ebyhr changed the title Don't overwrite reader and writer versions during during COMMENT and ADD COLUMN in Delta Don't overwrite reader and writer versions during COMMENT and ADD COLUMN in Delta Oct 14, 2022
@ebyhr
Copy link
Member Author

ebyhr commented Oct 14, 2022

Can we put the minWriterVersion and minReaderVersion in the DeltaLakeTableHandle?

Actually, I considered this during fixing, but we avoid adding fields to DeltaLakeTableHandle as much as possible. Let me keep as-is because the affected operations are only COMMENT and ADD COLUMN statements.

@ebyhr ebyhr merged commit 1e0e2d9 into master Oct 14, 2022
@ebyhr ebyhr deleted the ebi/delta-reader-writer-versions branch October 14, 2022 03:47
@ebyhr ebyhr mentioned this pull request Oct 14, 2022
@github-actions github-actions bot added this to the 401 milestone Oct 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

COMMENT and ADD COLUMN statements change reader and writer versions in Delta Lake connector
4 participants