Trino Parquet Column Encryption

[shangxinli]

Parquet modular encryption provides a column encryption feature. It would be great to have that in Presto. Due to the slowness of the PresteoDB, I would like to start the feature in TrinoDB. The original issue in PrestoDB can be found here.

It is a large feature and it can be divided into three parts: 1) Common code path 2) Decryption 3) Encryption. The command code path change and decryption change draft can be found here.

[xiacongling]

Hi, @shangxinli . I noticed that the previous PR has been closed, is there any progress on Parquet column encryption?

[amoghmargoor]

We at Apple have been running PME for a while now and would like to contribute the decryption support in Trino for it. We will be raising the draft PR soon for this. cc @vgankidi

[amoghmargoor]

Design doc for the decryption support: https://docs.google.com/document/d/1hve3UvT2oa6ud1iWifI6KH2dJpSjpg62svy6ES-cjcM/edit?usp=sharing

[my7ym]

@amoghmargoor Do you folks have any timeline to deliver the PME support for Trino? Thanks!

[hashhar]

It seems this is no longer being actively worked on.

Feel free to re-open if needed.

[sopel39]

I thin we should reopen it. cc @amoghmargoor

[amoghmargoor]

I have just opened a new PR for this rebased, PR is WIP as we are trying to add test cases for the encryption. #23583. This will have many newer files compared to existing PRs due to parquet-hadoop module being decoupled.

[dfangs]

Hey folks, I just noticed that there has been some progress on Parquet Modular Encryption support for Trino since our last inquiry in 2023.

We actually ended up continuing the work in #20069 internally as our company had a goal to enable PME on Trino last year (decryption only, same as Presto), and it has been in production for around half a year.

I wonder what is the best way for us to contribute. Notably, I discovered some bugs/missing parts in the original Presto implementation (including a potential fix to prestodb/presto#22703) and did some refactoring on the encryption-related classes (e.g., InternalFileDecryptor) to make it much more readable.

We would like to push these changes to the upstream too if possible to keep our fork minimal (i.e. only limited to custom KMS implementation). I saw that there is a WIP in #24517; I'm happy to open a new PR too if that works better.

cc @sopel39 if you have some thoughts on collaboration, thanks!