💧 Work through data-engineering-database-access repo for projects to Lake Formation Tags

[bagg3rs]

User Story

As a Analytical Platform team
I want to know how we currently manage database access
So that I can understand how these relate to maintaining access permissions for our users to query tables with Lake Formation

Value / Purpose

Database-access is the current method of granting access to databases with Athena. With the move to Lake Formation and the ability to use Tags we can improve how access is granted and speed up onboarding users to access the data they need without the need for pull requests

Useful Contacts

James, Julia, Michael, RichB, Data Engineering (Gwion, Soumaya)

Hypothesis

If we understand how we currently permission tables for 294 Projects
Then we can better spend our time developing AP-UI features that our users need

Proposal

"Dummy Migrate" a set of existing Projects over to use LF-Tags, this should be the target databases's we have for QuickSight MVP #4655

• How projects are currently organised see repo
• Identify key attributes used for access control (e.g. domain, department, data sensitivity... or project)

Map projects to tags

• For each project, determine which tags could apply
• Create a mapping document or spreadsheet there is an example here

Review current permissions

• Analyse existing access controls
• Note which project attributes determine access levels

Define tag-based policies

• Create Lake Formation policies using the new tags
• Ensure these policies mirror existing access controls

Validation and testing

• Compare tag-based access to current permissions
• Test to ensure equivalent access levels

PreReq'ish

• Projects have already been selected as part of 📝 Identify and document datasets to be used in in Quicksight MVP #4655

Definition of Done

• Projects have examined and permissions reviewed
• Document the migration and create permissions mapping document
• Give team overview and create any additional issues required

[YvanMOJdigital]

@bagg3rs need more info

[YvanMOJdigital]

Work with catalog team tags stuff