Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for writing partitionValues_parsed field in Delta Lake checkpoint #19586

Closed
ebyhr opened this issue Oct 31, 2023 · 0 comments · Fixed by #19662
Closed

Add support for writing partitionValues_parsed field in Delta Lake checkpoint #19586

ebyhr opened this issue Oct 31, 2023 · 0 comments · Fixed by #19662
Assignees
Labels
delta-lake Delta Lake connector

Comments

@ebyhr
Copy link
Member

ebyhr commented Oct 31, 2023

https://github.com/delta-io/delta/blob/master/PROTOCOL.md#checkpoints-1

partitionValues_parsed: In this struct, the column names correspond to the partition columns and the values are stored in their corresponding data type. This is a required field when the table is partitioned and the table property delta.checkpoint.writeStatsAsStruct is set to true. If the table is not partitioned, this column can be omitted. For example, for partition columns year, month and event with data types int, int and string respectively, the schema for this field will look like:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
delta-lake Delta Lake connector
2 participants