Add Apache Ranger authorizer plugin

[mneethiraj]

Description

Added Apache Ranger authorizer plugin to authorize data access in Trino using Apache Ranger policies. Earlier version of this plugin is in Apache Ranger git repo. The plugin has been updated for the changes in SystemAccessControl interface in Trino master branch.

Additional context and related issues

• RangerSystemAccessControlFactory implements SystemAccessControlFactory
• RangerSystemAccessControl implements SystemAccessControl

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# Apache Ranger authorizer for Trino
* This plugin supports use of Apache Ranger policies to authorize data access in Trino - like operations on catalogs, schemas, tables, columns.
* Column-masking and row-filtering are supported in this plugin.
* Accesses authorized by the plugin are audited for compliance purposes.

## Requirements
* Access to an Apache Ranger instance having authorization policies to be enforced by this plugin
* Access to audit stores (Solr/Elasticsearch/S3/HDFS) to save access audit logs

## Configuration
Add following entries in /etc/trino/access-control.properties to configure Apache Ranger as the authorizer in Trino:

access-control.name=ranger
ranger.service.name=dev_trino
ranger.plugin.config.resource=/etc/trino/ranger-trino-security.xml,/etc/trino/ranger-trino-audit.xml,/etc/trino/ranger-trino-policymgr-ssl.xml

Apache Ranger plugin configurations for policy store and audit store should be updated in following configuration file:
/etc/trino/ranger-trino-security.xml
/etc/trino/ranger-trino-audit.xml
/etc/trino/ranger-trino-policymgr-ssl.xml```

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[cla-bot]

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

[mneethiraj]

This is my first PR in Trino. Looking for help with the failures:

• ci/check-commits-dispatcher: PR requires a rebase. Found: 4 merge commits. Should I create a new PR?
• Test failures seem unrelated to this PR

[mosabua]

@mneethiraj thanks for sending this PR. I am not sure how it compares to the linked PR from @dprophet but overall we would like to get a Ranger plugin merged.

In terms of other issues:

• You need to submit a signed CLA.
• You might need to rebase the PR. Currently there are no conflicts so rebase should be easy.
• The CI failures seems to be unrelated so you can ignore.

[mneethiraj]

@mosabua - thank you for the response.

I am not sure how it compares to the linked PR from @dprophet but overall we would like to get a Ranger plugin merged.

PR #13297 was created about 2 years back, built with Trino version 391-SNAPSHOT. Ranger plugin in this PR builds with the current version - 453-SNAPSHOT, updated for changes in authorization interface since then. Also, this PR builds with the most recent Apache Ranger version - 2.4.0.

• You need to submit a signed CLA.
• You might need to rebase the PR. Currently there are no conflicts so rebase should be easy.
• The CI failures seems to be unrelated so you can ignore.

• I got the confirmation email that my CLA was received today.
• PR has been rebased. CI is in progress

[lozbrown]

It would be amazing to get this merged.

Haveing to patch docker images to add the plugin in really slows our ability to keep up with upgrades

[mosabua]

Excellent @mneethiraj .. maybe @dprophet and his team can help with the review so we can get this over the finish line easier.

[mneethiraj]

maybe @dprophet and his team can help with the review so we can get this over the finish line easier.

@dprophet - can you please review this PR and help get this merged?

[lozbrown]

I guess we need to enhance the helm chart to support the Ranger config.

Additionally that config should be documented in the trino website

[mosabua]

Docs should be part of this PR.

Helm chart for running Ranger is a separate topic and potentially out of scope for the Trino chart. Relevant config for this plugin in the Helm chart might be possible with the existing chart or could be added after this is merge.

cc @nineinchnick

[mneethiraj]

Docs should be part of this PR.

@mosabua - documentation on configuring Trino to use Ranger plugin is included in this PR, at plugin/trino-ranger/README.md. And sample configurations to use Ranger plugin in docker setup are included in directory plugin/trino-ranger/src/main/resources/conf-docker. Please review and let me know if any updates are needed.

[mosabua]

The docs should really be user visible .. so you need to add a md file in https://github.com/trinodb/trino/tree/master/docs/src/main/sphinx/security and hook it in https://github.com/trinodb/trino/blob/master/docs/src/main/sphinx/security.md#access-control

Once you moved the content from the readme and updated it I can review

[lozbrown]

Docs should be part of this PR.

Helm chart for running Ranger is a separate topic and potentially out of scope for the Trino chart. Relevant config for this plugin in the Helm chart might be possible with the existing chart or could be added after this is merge.

cc @nineinchnick

Whilst an official ranger helm chart would be wonderful somewhere it's not really a trino problem.

I was specifically referring to enhancing the trino helm chart to support config for this plugin. Makeing the most used configurations configurable via the values files.

Some of the config for this plugin is xml which is a bit inconsistent / incongruent with all other trino config.

[ognjen-it]

Hi @mneethiraj ,

I'm so happy to see that ranger-trino plugin will be up-to-date now.

I just tried to test this plugin with ranger 2.4.0, but I'm always getting the same error, access deny. Audit:

{
    "repoType": 203,
    "repo": "trino",
    "reqUser": "trino",
    "evtTime": "2024-07-29 15:11:48.142",
    "access": "SetUser",
    "resource": "trino",
    "resType": "trinouser",
    "action": "impersonate",
    "result": 0,
    "agent": "trino",
    "policy": 24,
    "enforcer": "ranger-acl",
    "agentHost": "trino-coordinator-662b97674b-9bxst",
    "logType": "RangerAudit",
    "id": "f6914203-d6ba-4103-a601-84ae4ae978fa-7",
    "seq_num": 15,
    "event_count": 1,
    "event_dur_ms": 0,
    "tags": [],
    "cluster_name": ""
}

How did you get it working? Could you please share with us how to test it?

[mneethiraj]

Merged updates from upstream/master.

[mneethiraj]

except for removing the implementation of grant/revoke/deny methods, all other comments are addressed.

grant/revoke/deny methods updates are done as well.

[mneethiraj]

@kokosing, @ksobolew, @mosabua - are there any more items to address before this PR can be merged?

[ksobolew]

I think it looks good to me. Thank you!

[kokosing]

Looks good. A bunch of hopefully last comments.

[mneethiraj]

• updates to address review comments
• merged commits from latest upstream/master.

[mosabua]

@kokosing @ksobolew @mneethiraj .. is this ready now after a rebase? From my view it looks good to go.

[kokosing]

Thank you all for huge effort! I think it is an important milestone for Ranger and Trino. Great work!

[kokosing]

Please rebase as there were plenty of commits recently and a release. It would be nice to make sure the whole CI is passing.

Also please squash commits, and make sure that all the co-authors who contributed code here are mentioned in the commit message. My impression says it was a crowd effort :)

[mneethiraj]

merged commits from latest upstream/master.

[mneethiraj]

Please rebase as there were plenty of commits recently and a release. It would be nice to make sure the whole CI is passing.

Done. The most recent run has whole CI passing.

Also please squash commits, and make sure that all the co-authors who contributed code here are mentioned in the commit message. My impression says it was a crowd effort :)

Done. This PR includes contributions from @lozbrown and @kokosing; both have been listed as Co-authors in the commit message.

[kokosing]

Thank you! 🎉 👏 ❤️

[niteshy]

Congrats @mneethiraj 👏 🎉 ❤️

[ksobolew]

❤️

[mosabua]

Congratulations @mneethiraj and everyone involved .. I am looking forward to see this shipped in Trino 466. Also note that I might get a blog post about this going .. and at a minimum it will be part of the updates in the keynote for Trino Summit.

[mneethiraj]

My sincere thanks to @kokosing, @ksobolew, @mosabua for the most rigorous review I have been through! Without your time, effort and guidance, this PR wouldn't have gotten here. Special thanks to @kokosing for your patience, detailed comments and suggestions; I learned a lot through our interactions. @lozbrown - thank you for owning the documentation part of this PR, even before I could ask for help.

Trino community has built an amazing level of infrastructure to keep the bar very high for commits. Kudos to everyone here!

Looking forward to continued engagement with Trino community. There is a lot to do..but for now, let me relish my first commit:-)