Iceberg storage caching on ephemeral disks #18988
mayankasthana
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
We use iceberg connector with data stored in GCS. Trino runs on GCP VMs. GCP provides local-ssd which are ephemeral directly attached NVME SSD disks. These would be great for caching.
However, icerberg catalog doesn't provide storage caching that is present in hive connector.
What would be a good way to do some caching? Can we add this functionality to trino on iceberg-connector? And perhaps other connectors?
There is also alluxio that enables caching for iceberg on trino, but I am unsure how it will work with ephemeral storage.
I am also unsure if hive connector storage caching works well with ephemeral disks or not.
Beta Was this translation helpful? Give feedback.
All reactions