feat: DBTP-1507 Setup codepipeline notifications #314

tony-griffin · 2025-01-16T09:22:47Z

Addresses DBTP-1507

Use platform-helper notify commands for notifications, similar to our other pipelines.

Code relied on John's codebase pipeline PR and so is branched from that.
Codebase module files changed in this PR:
codebase-pipelines/buildspec-deploy.yml
codebase-pipelines/codepipeline.tf
codebase-pipelines/locals.tf
codebase-pipelines/codepipeline.tf
codebase-pipelines/tests/unit.tftest.hcl
codebase-pipelines/iam.tf

Checklist:

Title:

Scope included as per conventional commits
Ticket reference included (unless it's a quick out of ticket thing)

Description:

Link to ticket included (unless it's a quick out of ticket thing)
Includes tests (or an explanation for why it doesn't)
Includes any applicable changes to the documentation in this code base
Includes link(s) to any applicable changes to the documentation in the DBT Platform Documentation (can be to a pull request)

Tasks:

Trigger the pull request regression tests for this branch and confirm that they are passing

…eline-notifications

codecov-commenter · 2025-01-17T11:53:03Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 81.68%. Comparing base (a886ed1) to head (5b28002).
Report is 30 commits behind head on DBTP-1503-manual-pipeline.

✅ All tests successful. No failed tests found.

Additional details and impacted files

@@                    Coverage Diff                     @@
##           DBTP-1503-manual-pipeline     #314   +/-   ##
==========================================================
  Coverage                      81.68%   81.68%           
==========================================================
  Files                              7        7           
  Lines                            868      868           
==========================================================
  Hits                             709      709           
  Misses                           159      159

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…eline-notifications

…om:uktrade/terraform-platform-modules into DBTP-1507-setup-codepipeline-notifications

antroy-madetech

A few minor comments.

codebase-pipelines/buildspec-deploy.yml

antroy-madetech · 2025-01-21T13:47:38Z

codebase-pipelines/buildspec-deploy.yml

+      - aws ecr get-login-password --region ${AWS_REGION} | docker login --username AWS --password-stdin ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com
+
+      # construct Slack message env vars
+      - |


I think in general, these multiline commands should be reserved for blocks, such as if/then/else clauses, function definitions while loops etc. So I'd put line 30 in it's own single line item.

antroy-madetech · 2025-01-21T13:48:18Z

codebase-pipelines/buildspec-deploy.yml

+          echo "Image contains no timestamp label"
+          exit 1
+        fi
+        echo "Found image timestamp $SLACK_REF"


Similarly this echo should be it's own item

antroy-madetech · 2025-01-21T13:49:21Z

codebase-pipelines/buildspec-deploy.yml

+
+        # Extract the pipeline name from CODEBUILD_INITIATOR default env var
+      - |
+        PIPELINE_NAME=""


Not needed - the conditional will either add the PIPELINE_NAME or exit

antroy-madetech · 2025-01-21T13:49:49Z

codebase-pipelines/buildspec-deploy.yml

+        fi
+
+      # Construct the pipeline execution URL
+      - |


Split this into 2 single line items.

antroy-madetech · 2025-01-21T13:50:00Z

codebase-pipelines/buildspec-deploy.yml

+        echo "Pipeline execution URL: ${PIPELINE_EXECUTION_URL}"
+
+      - |
+        BUILD_ID_PREFIX=$(echo $CODEBUILD_BUILD_ID | cut -d':' -f1)


antroy-madetech · 2025-01-21T13:50:46Z

codebase-pipelines/buildspec-deploy.yml

+        BUILD_ID_PREFIX=$(echo $CODEBUILD_BUILD_ID | cut -d':' -f1)
+        echo "BUILD_ID_PREFIX - ${BUILD_ID_PREFIX}"
+
+      - |


This can be just a single line item. - MESSAGE=...

I tried adding as a single line comment but there was a syntax error because of the use of the ":" which only resolved when using multiline

FYI - you should be able to just surround the line in single quotes. It's because it thinks it's a map rather than a string. Surrounding in single quotes will make the YAML parser treat it as a string.

But happy for you to leave it as a multi-line quote in this case as it's probably neater.

antroy-madetech · 2025-01-21T13:51:39Z

codebase-pipelines/buildspec-deploy.yml

+            STATUS_TEXT="STOP_REQUESTED/STOPPED/PENDING/IN_PROGRESS"
+            ;;
+        esac
+        echo "Status emoji set to ${STATUS_EMOJI}"


The echo should be a command on it's own

antroy-madetech · 2025-01-21T13:54:24Z

codebase-pipelines/buildspec-deploy.yml

+        esac
+        echo "Status emoji set to ${STATUS_EMOJI}"
+
+      - |


don't need the block scalar indicator here as it's just a single line.

Co-authored-by: Anthony Roy <[email protected]>

codebase-pipelines/iam.tf

codebase-pipelines/tests/unit.tftest.hcl

Co-authored-by: John Stainsby <[email protected]>

antroy-madetech

Looking better. A minor formatting thing left, but approved.

antroy-madetech · 2025-01-22T11:21:19Z

codebase-pipelines/buildspec-deploy.yml

+
+      # Construct the pipeline execution URL
+      - PIPELINE_EXECUTION_URL="https://${AWS_REGION}.console.aws.amazon.com/codesuite/codepipeline/pipelines/${PIPELINE_NAME}/executions/${PIPELINE_EXECUTION_ID}" 
+


Maybe just remove some of these random empty lines in this section.

antroy-madetech · 2025-01-22T11:23:53Z

codebase-pipelines/buildspec-deploy.yml

+        BUILD_ID_PREFIX=$(echo $CODEBUILD_BUILD_ID | cut -d':' -f1)
+        echo "BUILD_ID_PREFIX - ${BUILD_ID_PREFIX}"
+
+      - |


FYI - you should be able to just surround the line in single quotes. It's because it thinks it's a map rather than a string. Surrounding in single quotes will make the YAML parser treat it as a string.

But happy for you to leave it as a multi-line quote in this case as it's probably neater.

tony-griffin added 14 commits January 15, 2025 13:32

add buildspec-deploy.yml file for codebase module

2fbcbc1

add install platform -helper

d74a458

Merge branch 'DBTP-1503-manual-pipeline' into DBTP-1507-setup-codepip…

31f6ee1

…eline-notifications

Add codebuild env vars for deploy buildspec

065f434

Add local env vars for use in deploy buildspec

7c43eb5

Add platform-helper notify slack messages to codebase deploy buildspec

8f0e3a4

Add slack channel module variable

ad16c0f

Add slack channel variable parameter store type

1a1f80c

Update tests to reflect new codebase codebuild env vars

0b84d7c

Add platform-helper notify command with emojis

04667f7

Add IAM permissions for ECR & SSM access for deploy buildspec

6479d1f

Build env vars for pipeline execution ID URL

5f1a862

Add default codepipeline execution ID env var at pipeline level

cd25d02

Update terraform tests to reflect adding env vars to codebuild actions

5c7cddf

tony-griffin added 4 commits January 18, 2025 15:44

Add extra codebuild env vars to manual release codebase pipeline

069b091

FIX: environment variable for manual release pipeline

3e61b5d

Update tests for manual release pipeline env vars

26d8ebf

Remove unnecessary local vars for region & account

2631bef

JohnStainsby changed the base branch from main to DBTP-1503-manual-pipeline January 20, 2025 09:16

JohnStainsby changed the base branch from DBTP-1503-manual-pipeline to main January 20, 2025 09:20

tony-griffin marked this pull request as ready for review January 20, 2025 11:49

tony-griffin requested a review from a team as a code owner January 20, 2025 11:49

tony-griffin and others added 3 commits January 20, 2025 12:36

Update message output

9e0bec7

Merge branch 'DBTP-1503-manual-pipeline' into DBTP-1507-setup-codepip…

58767b9

…eline-notifications

Merge branch 'DBTP-1507-setup-codepipeline-notifications' of github.c…

6aad173

…om:uktrade/terraform-platform-modules into DBTP-1507-setup-codepipeline-notifications

JohnStainsby changed the base branch from main to DBTP-1503-manual-pipeline January 21, 2025 10:42

antroy-madetech reviewed Jan 21, 2025

View reviewed changes

tony-griffin and others added 2 commits January 21, 2025 14:24

Update codebase-pipelines/buildspec-deploy.yml

8e6a57b

Co-authored-by: Anthony Roy <[email protected]>

Update codebase-pipelines/buildspec-deploy.yml

3503dec

Co-authored-by: Anthony Roy <[email protected]>

JohnStainsby reviewed Jan 21, 2025

View reviewed changes

codebase-pipelines/iam.tf Outdated Show resolved Hide resolved

JohnStainsby reviewed Jan 21, 2025

View reviewed changes

codebase-pipelines/iam.tf Outdated Show resolved Hide resolved

tony-griffin added 5 commits January 21, 2025 14:36

Update bash syntax in buildspec file

28ce523

Update name of ssm deploy access policy and data policy

e877faa

Update name of ssm deploy data policy test override

740442a

Update role policy name

af54f02

Add missing tests for data policy and policy for deploy ssm access

2f40fb0

JohnStainsby reviewed Jan 21, 2025

View reviewed changes

codebase-pipelines/tests/unit.tftest.hcl Show resolved Hide resolved

tony-griffin and others added 2 commits January 21, 2025 15:28

Update codebase-pipelines/tests/unit.tftest.hcl

6323922

Co-authored-by: John Stainsby <[email protected]>

Fix test indentation

5b28002

antroy-madetech approved these changes Jan 22, 2025

View reviewed changes

tony-griffin merged commit 5b28002 into DBTP-1503-manual-pipeline Jan 23, 2025
48 checks passed

tony-griffin deleted the DBTP-1507-setup-codepipeline-notifications branch January 23, 2025 13:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: DBTP-1507 Setup codepipeline notifications #314

feat: DBTP-1507 Setup codepipeline notifications #314

tony-griffin commented Jan 16, 2025 •

edited

Loading

codecov-commenter commented Jan 17, 2025 •

edited

Loading

antroy-madetech left a comment

antroy-madetech Jan 21, 2025

antroy-madetech Jan 21, 2025

antroy-madetech Jan 21, 2025

antroy-madetech Jan 21, 2025

antroy-madetech Jan 21, 2025

antroy-madetech Jan 21, 2025

tony-griffin Jan 21, 2025

antroy-madetech Jan 22, 2025

antroy-madetech Jan 21, 2025

antroy-madetech Jan 21, 2025

antroy-madetech left a comment

antroy-madetech Jan 22, 2025

antroy-madetech Jan 22, 2025


		# Construct the pipeline execution URL
		- PIPELINE_EXECUTION_URL="https://${AWS_REGION}.console.aws.amazon.com/codesuite/codepipeline/pipelines/${PIPELINE_NAME}/executions/${PIPELINE_EXECUTION_ID}"

feat: DBTP-1507 Setup codepipeline notifications #314

feat: DBTP-1507 Setup codepipeline notifications #314

Conversation

tony-griffin commented Jan 16, 2025 • edited Loading

Checklist:

Title:

Description:

Tasks:

codecov-commenter commented Jan 17, 2025 • edited Loading

Codecov Report

antroy-madetech left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antroy-madetech left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tony-griffin commented Jan 16, 2025 •

edited

Loading

codecov-commenter commented Jan 17, 2025 •

edited

Loading