Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support column mapping #930

Open
dennyglee opened this issue Nov 10, 2022 · 2 comments
Open

Support column mapping #930

dennyglee opened this issue Nov 10, 2022 · 2 comments
Labels
enhancement New feature or request

Comments

@dennyglee
Copy link
Collaborator

Description

Per 1076, it would be great for delta-rs to support column mapping. With Delta tables that support column mapping, the altering or dropping of columns is a metadata only operation.

Use Case
As folks can enable column mapping for Delta tables, it would be great for delta-rs to support this feature.

Related Issue(s)
#1076, #1064

@polynomialherder
Copy link
Contributor

I've started working through this -- I'll reach out with updates and questions

@aersam
Copy link
Contributor

aersam commented Aug 2, 2023

On python side, I guess it's not possible to implement with the current to_pyarrow_dataset method, since Dataset does not support a distinction between physical and logical column names. I opened an issue about that: apache/arrow#36593

With pyarrow Table would be fairly straight forward, one could probably also expose a RecordBatchReader based on a dataset which uses physical column names.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants