☁️ Complete Deployment of Bedrock into Airflow

[julialawrence]

User Story

Making Bedrock available in Airflow requires change in the underlying iam_builder module. Version https://github.com/moj-analytical-services/iam_builder/releases/tag/v4.6.0 of the module has now been released to pypi and includes a bedrock toggle which optionally adds bedrock permission to the DAG role.

As a release of a new version of the iam_buider requires bumping dependencies in the requirements-pulumi.txt used in the DAGs, an amount of local verification of the changes will be required.

Value / Purpose

Releasing this version of the iam_builder module will give Airflow users access to Bedrock, requested via a number of existing feature requests.

Useful Contacts

Julia Lawrence Jacob Woffenden Jacob Hamblin-Pyke

User Types

No response

Hypothesis

No response

Proposal

Run a test deployment from a developer machine to the dev environment using pulumi. You will need an administrator role in data-production to carry out the relevant pulumi operations.

• aws-vault into data-prod account
• Update requirements and install to venv
• pulumi stack select dev
• pulumi preview
• Select details
• Ensure policy changes are innocuous.

Additional Information

No response

Definition of Done

Example

• Documentation has been written / updated
• Testing carried out successfully
• Module released