Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(build): publish lib as a Lambda Layer #884

Merged
merged 35 commits into from
Aug 9, 2022
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
dcf724d
Add a cdk app capable of building and publishing Lambda Layer
May 19, 2022
18ed8c9
add github action for e2e
May 20, 2022
6c8939e
add local layer construct npm package
May 20, 2022
76fa1d8
disable fail fast
May 20, 2022
e063843
fix RUNTIME naming in matrix
May 20, 2022
5a884ee
fix tests
May 20, 2022
bc0a318
fix outputs export name
May 20, 2022
b057127
Add capability to pass package version to layer builder/publisher
May 20, 2022
583a12f
Add initial doc
May 20, 2022
e12dc75
Add github workflows
Jul 21, 2022
022c517
add gitignore
Jul 21, 2022
e24df14
fix install deps
Jul 22, 2022
0f50311
pass layer name
Jul 22, 2022
01b91a8
make layer public and store details in ssm
Jul 22, 2022
e286e8d
fix e2e tests
Jul 22, 2022
cdf4e1f
fix context
Jul 22, 2022
b03413e
remvove groups for tests
Jul 22, 2022
e7a254d
publish layer on release
Jul 22, 2022
44f0728
fix account in doc
Jul 22, 2022
8401da8
fix test deps
Jul 22, 2022
fb99afc
deploy to all region
Jul 22, 2022
297907e
fix account number for layer
Jul 22, 2022
2ebe192
fix unit tests
Jul 22, 2022
5d8ad97
add install for layer deps in pr workflow
Jul 22, 2022
e70e5c1
run unit test of layer publisher for supported node versions only
Jul 22, 2022
f509c6a
fix node version in doc
Jul 22, 2022
a2d1be5
add node 16 to layer e2e
Jul 22, 2022
1e58383
fix comments
Jul 22, 2022
f8fee27
fix doc
Jul 22, 2022
810e0dc
rename layer to TypeScript
Aug 4, 2022
d07ccf9
take care of comments
Aug 4, 2022
c8465e2
remove layer from doc for now
Aug 4, 2022
c2b0f1b
remove pasted commenbt
Aug 4, 2022
4a1c436
chore(build): broke up pr workflow & measure package size (#1031)
dreamorosi Aug 9, 2022
fa2f59b
Merge branch 'feat/layerPublisher' of github.com:awslabs/aws-lambda-p…
dreamorosi Aug 9, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/on-merge-to-main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ jobs:
# the dependencies in a separate step
working-directory: ./examples/cdk
run: npm ci
- name: Install Layer publisher app
working-directory: ./layer-publisher
run: npm ci
- name: "Setup SAM"
# We use an ad-hoc action so we can specify the SAM CLI version
uses: aws-actions/setup-sam@v2
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/pr_lint_and_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@ jobs:
# the dependencies in a separate step
working-directory: ./examples/cdk
run: npm ci
- name: Install Layer publisher app
working-directory: ./layer-publisher
run: npm ci
- name: "Setup SAM"
# We use an ad-hoc action so we can specify the SAM CLI version
uses: aws-actions/setup-sam@v2
Expand Down
81 changes: 81 additions & 0 deletions .github/workflows/publish_layer.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
name: Deploy layer to all regions

permissions:
id-token: write
contents: read

on:
# Manual trigger
workflow_dispatch:
inputs:
latest_published_version:
description: "Latest npm published version to rebuild corresponding layer for, e.g. v1.0.2"
default: "v1.0.2"
required: true
# Automatic trigger after release
workflow_run:
workflows: ["release"]
types:
- completed

jobs:
# Build layer by running cdk synth in layer-publisher directory and uploading cdk.out for deployment
build-layer:
runs-on: ubuntu-latest
if: ${{ (github.event.workflow_run.conclusion == 'success') || (github.event_name == 'workflow_dispatch') }}
ijemmy marked this conversation as resolved.
Show resolved Hide resolved
defaults:
run:
working-directory: ./layer-publisher
steps:
- name: checkout
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: "16.12"
- name: Set release notes tag
run: |
RELEASE_INPUT=${{ inputs.latest_published_version }}
LATEST_TAG=$(git describe --tag --abbrev=0)
RELEASE_TAG_VERSION=${RELEASE_INPUT:-$LATEST_TAG}
echo "RELEASE_TAG_VERSION=${RELEASE_TAG_VERSION:1}" >> $GITHUB_ENV
ijemmy marked this conversation as resolved.
Show resolved Hide resolved
- name: install cdk and deps
run: |
npm install -g [email protected]
cdk --version
- name: install deps
run: |
npm ci
- name: CDK build
run: cdk synth --context PowerToolsPackageVersion=$RELEASE_TAG_VERSION -o cdk.out
- name: zip output
run: zip -r cdk.out.zip cdk.out
- name: Archive CDK artifacts
uses: actions/upload-artifact@v3
with:
name: cdk-layer-artefact
path: layer-publisher/cdk.out.zip

# Deploy layer to all regions in beta account
deploy-beta:
needs:
- build-layer
uses: ./.github/workflows/reusable_deploy_layer_stack.yml
with:
stage: "BETA"
artefact-name: "cdk-layer-artefact"
secrets:
target-account-role: ${{ secrets.AWS_LAYERS_BETA_ROLE_ARN }}

# Deploy layer to all regions in prod account
deploy-prod:
needs:
- deploy-beta
uses: ./.github/workflows/reusable_deploy_layer_stack.yml
with:
stage: "PROD"
artefact-name: "cdk-layer-artefact"
secrets:
target-account-role: ${{ secrets.AWS_LAYERS_PROD_ROLE_ARN }}
81 changes: 81 additions & 0 deletions .github/workflows/reusable_deploy_layer_stack.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
name: Deploy cdk stack

permissions:
id-token: write
contents: read

on:
workflow_call:
inputs:
stage:
required: true
type: string
artefact-name:
required: true
type: string
secrets:
target-account-role:
required: true

jobs:
deploy-cdk-stack:
runs-on: ubuntu-latest
defaults:
run:
working-directory: ./layer-publisher
strategy:
fail-fast: false
matrix:
region:
[
"af-south-1",
"eu-central-1",
"us-east-1",
"us-east-2",
"us-west-1",
"us-west-2",
"ap-east-1",
"ap-south-1",
"ap-northeast-1",
"ap-northeast-2",
"ap-southeast-1",
"ap-southeast-2",
"ca-central-1",
"eu-west-1",
"eu-west-2",
"eu-west-3",
"eu-south-1",
"eu-north-1",
"sa-east-1",
"ap-southeast-3",
"ap-northeast-3",
"me-south-1",
]
steps:
- name: checkout
uses: actions/checkout@v3
- name: aws credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-region: ${{ matrix.region }}
role-to-assume: ${{ secrets.target-account-role }}
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: "16.12"
- name: install cdk and deps
run: |
npm install -g [email protected]
cdk --version
- name: install deps
run: |
npm ci
- name: Download artifact
uses: actions/download-artifact@v3
with:
name: ${{ inputs.artefact-name }}
path: layer-publisher
- name: unzip artefact
run: unzip cdk.out.zip
- name: CDK Deploy Layer
run: cdk deploy --app cdk.out --context region=${{ matrix.region }} 'LayerPublisherStack' --require-approval never --verbose
ijemmy marked this conversation as resolved.
Show resolved Hide resolved
34 changes: 32 additions & 2 deletions .github/workflows/run-e2e-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ jobs:
cd examples/cdk
npm install ../../packages/**/dist/aws-lambda-powertools-*
npm run test
e2e-tests:
package-e2e-tests:
runs-on: ubuntu-latest
permissions:
id-token: write # needed to interact with GitHub's OIDC Token endpoint.
Expand Down Expand Up @@ -68,6 +68,36 @@ jobs:
with:
role-to-assume: ${{ secrets.AWS_ROLE_ARN_TO_ASSUME }}
aws-region: eu-west-1
- name: "Run integration tests"
- name: "Run packages integration tests"
run: |
RUNTIME=nodejs${{ matrix.version }}x npm run test:e2e -w packages/${{ matrix.package }}
layer-e2e-tests:
runs-on: ubuntu-latest
permissions:
id-token: write # needed to interact with GitHub's OIDC Token endpoint.
contents: read
strategy:
fail-fast: false
matrix:
version: [12, 14, 16]
steps:
- name: "Checkout"
uses: actions/checkout@v3
- name: "Use NodeJS 14"
uses: actions/setup-node@v3
with:
# Always use version 16 as we use TypeScript target es2020
node-version: 16
- name: "Install [email protected]"
run: npm i -g npm@next-8
- name: "Configure AWS credentials"
uses: aws-actions/[email protected]
with:
role-to-assume: ${{ secrets.AWS_ROLE_ARN_TO_ASSUME }}
aws-region: eu-west-1
- name: "Run layer integration tests"
run: |
npm ci --foreground-scripts
RUNTIME=nodejs${{ matrix.version }}.x npm run test:e2e
working-directory: layer-publisher

12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,18 @@ Find the complete project's [documentation here](https://awslabs.github.io/aws-l

### Installation

You have 2 ways of consuming those utilities:
* NPM modules
* Lambda Layer

#### Lambda layers

The AWS Lambda Powertools for TypeScript utilities is packaged as a single [AWS Lambda Layer](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-concepts.html#gettingstarted-concepts-layer)

👉 [Installation guide for the **AWS Lambda Powertools for TypeScript** layer](https://awslabs.github.io/aws-lambda-powertools-typescript/latest/#lambda-layer)

#### NPM modules

The AWS Lambda Powertools for TypeScript utilities follow a modular approach, similar to the official [AWS SDK v3 for JavaScript](https://github.com/aws/aws-sdk-js-v3).
Each TypeScript utility is installed as standalone NPM package.

Expand Down
8 changes: 8 additions & 0 deletions layer-publisher/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
*.js
!jest.config.js
*.d.ts
node_modules

# CDK asset staging directory
.cdk.staging
cdk.out
6 changes: 6 additions & 0 deletions layer-publisher/.npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
*.ts
!*.d.ts

# CDK asset staging directory
.cdk.staging
cdk.out
38 changes: 38 additions & 0 deletions layer-publisher/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Lambda Powertools for TypeScript Layer Publisher

This CDK app is meant to be used to publish Powertools for TypeScript Lambda Layer. It is composed of a single stack deploying the Layer into the target account.

# Usage

```sh
npm ci
npm run cdk deploy
```

By default it will package the layer with the latest version publicly available but you can force the public version to use with `PowerToolsPackageVersion` context variable:
```sh
npm run cdk deploy -- --context PowerToolsPackageVersion='0.9.0'
```

# Tests

## Units

```sh
npm run test
```

## E2E

This will deploy and destroy several stacks in your AWS Account

```sh
npm run test:e2e
```

PS: You can force
* the lambda runtime to test with the RUNTIME env variable
* the Powertools version with VERSION env variable
```sh
RUNTIME=node12.x VERSION=0.9.0 npm run test:e2e
```
13 changes: 13 additions & 0 deletions layer-publisher/bin/layer-publisher.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { LayerPublisherStack } from '../src/layer-publisher-stack';

const SSM_PARAM_LAYER_ARN = '/layers/powertools-layer-arn';

const app = new cdk.App();
new LayerPublisherStack(app, 'LayerPublisherStack', {
powerToolsPackageVersion: app.node.tryGetContext('PowerToolsPackageVersion'),
layerName: 'AWSLambdaPowertoolsTypeScript',
ssmParameterLayerArn: SSM_PARAM_LAYER_ARN,
});
Binary file added layer-publisher/[email protected]
Binary file not shown.
32 changes: 32 additions & 0 deletions layer-publisher/cdk.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Putting this comment here because the file above called is a binary and doesn't allow comments:

Do we need this layer-publisher/[email protected]???

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for now yes since my PR there is not merged yet : aws-samples/cdk-lambda-powertools-python-layer#52

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha, thanks for the explanation

"app": "npx ts-node --prefer-ts-exts bin/layer-publisher.ts",
"watch": {
"include": [
"**"
],
"exclude": [
"README.md",
"cdk*.json",
"**/*.d.ts",
"**/*.js",
"tsconfig.json",
"package*.json",
"yarn.lock",
"node_modules",
"test"
]
},
"context": {
"@aws-cdk/aws-apigateway:usagePlanKeyOrderInsensitiveId": true,
"@aws-cdk/core:stackRelativeExports": true,
"@aws-cdk/aws-rds:lowercaseDbIdentifier": true,
"@aws-cdk/aws-lambda:recognizeVersionProps": true,
"@aws-cdk/aws-cloudfront:defaultSecurityPolicyTLSv1.2_2021": true,
"@aws-cdk-containers/ecs-service-extensions:enableDefaultLogDriver": true,
"@aws-cdk/aws-ec2:uniqueImdsv2TemplateName": true,
"@aws-cdk/core:target-partitions": [
"aws",
"aws-cn"
]
}
}
8 changes: 8 additions & 0 deletions layer-publisher/jest.config.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
module.exports = {
testEnvironment: 'node',
roots: ['<rootDir>/tests'],
testMatch: ['**/*.test.ts'],
transform: {
'^.+\\.tsx?$': 'ts-jest'
}
};
Loading