-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Table Scan Enhancement Plan #944
Comments
Also, from my own perspective, I have some extensions to make:
|
thanks a lot for this list!
|
As a CatalogProvider implementation, I suppose. |
Here is my understanding based on @yjshen 's PRs so far:
I am guessing a hive metastore connector will need to touch both SchemaProvider and TableProvider? I do think that ObjectStore is too low level to implement hive metastore. |
Thanks @houqp for the detailed explanation. After checking delta-rs code as well as what we are doing currently in DataFusion. I have several thoughts:
|
Thanks @yjshen I plan to review this carefully tomorrow |
@yjshen this seems mostly complete now? |
I agree this is basically complete, so closing it down. Please feel free to reopen or file a new issue with any remaining work to be done |
To make DataFusion scan tables more flexibly and more efficiently, we can take several further steps. (I linked several issues I am aware of)
Capability:
Performance:
The text was updated successfully, but these errors were encountered: