-
Notifications
You must be signed in to change notification settings - Fork 995
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Reduce the access of filesystem when reading paimon table #2978
Comments
I submit an pr to cache table in catalog , #2971 |
we can also to cache manifest file to improve query speed, after all, https://docs.cloudera.com/cdw-runtime/cloud/iceberg-how-to/topics/iceberg-manifest-caching.html |
@zhangjun0x01 This is a separate issue. In theory, we can cache all the files. |
@wg1026688210 I think we have finished this issue, thanks! |
yes ,we can open other pr to cache manifest , like this apache/iceberg#4518 |
Is there any progress? This is a very important requirement. |
Search before asking
Motivation
Currently, the computation engine frequently reads the schema file and the size of the data files from filesystem when reading the Paimon table.This lead to additional access to the file system. The purpose of this issue is to reduce access to the file system by caching the schema ,paimon table and so on.
Solution
Anything else?
No response
Are you willing to submit a PR?
The text was updated successfully, but these errors were encountered: