Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Virtual column support parquet format #12425

Merged
merged 20 commits into from
Aug 18, 2023
Merged

Conversation

b41sh
Copy link
Member

@b41sh b41sh commented Aug 10, 2023

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

  • Virtual column support read parquet format
  • Using VirtualColumnReader to read virtual columns
  • Ignore reading source columns if all virtual columns are generated

Part of #6994


This change is Reviewable

@vercel
Copy link

vercel bot commented Aug 10, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
databend ⬜️ Ignored (Inspect) Visit Preview Aug 18, 2023 0:19am

@lichuang lichuang self-requested a review August 10, 2023 15:24
@b41sh b41sh force-pushed the virtual_column_parquet-b41sh branch from 3f5210a to 68c8ff2 Compare August 14, 2023 08:55
@b41sh b41sh force-pushed the virtual_column_parquet-b41sh branch from 68c8ff2 to 0a9aaaa Compare August 15, 2023 17:37
@b41sh b41sh requested review from sundy-li and RinChanNOWWW August 16, 2023 08:31
@b41sh b41sh marked this pull request as ready for review August 16, 2023 08:32
@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Aug 16, 2023
@RinChanNOWWW
Copy link
Contributor

This PR is to support store virtual columns as parquet format?

@b41sh
Copy link
Member Author

b41sh commented Aug 16, 2023

This PR is to support store virtual columns as parquet format?

The format of the virtual column block is the same as the corresponding table block format. Previously, only table of native type was supported, after this PR table of parquet type is also supported.

@BohuTANG
Copy link
Member

The unit test always failed in this PR

@b41sh b41sh merged commit b625033 into main Aug 18, 2023
@b41sh b41sh deleted the virtual_column_parquet-b41sh branch August 18, 2023 13:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants