Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): migrate E2E tests to CDK CLI and off Docker #1501

Merged
merged 33 commits into from
Sep 20, 2022
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
610688c
chore(ci): add cdk environment; failing still
heitorlessa Aug 31, 2022
c1d103f
chore: remove vpc as from_attrs isnt reliable
heitorlessa Sep 1, 2022
cfde54b
chore: ignore cdk.out
heitorlessa Sep 1, 2022
ea7fe48
chore: migrate to CDK CLI stage 1
heitorlessa Sep 1, 2022
72b2237
chore: migrate to CDK CLI stage 2 - Layers work
heitorlessa Sep 2, 2022
91b3bab
chore: migrate to CDK CLI stage 3 - off Docker
heitorlessa Sep 3, 2022
c0ed5e5
chore: migrate to CDK CLI stage 4 - Parallel deployment works
heitorlessa Sep 4, 2022
8aef75f
chore: migrate to CDK CLI stage 5 - Abstract Local Powertools Layer
heitorlessa Sep 4, 2022
055e2f5
chore: migrate to CDK CLI stage 6 - Build Layer if source has changed
heitorlessa Sep 4, 2022
5f28a53
chore: cleanup deploy, guard for lack of infra
heitorlessa Sep 4, 2022
a322be7
refactor: remove need for feature_name and super in stacks
heitorlessa Sep 4, 2022
36e92e6
refactor(tracer): allow service to be changed at runtime for e2e testing
heitorlessa Sep 4, 2022
fa3e47c
docs(maintainers): update workflow diagram
heitorlessa Sep 6, 2022
b48d6cf
docs(maintainers): update structure
heitorlessa Sep 6, 2022
7e52789
chore: cleanup cdk.out generated file names
heitorlessa Sep 6, 2022
62e92bb
chore: add cdk context file
heitorlessa Sep 6, 2022
a6a4da0
chore: add more info on sync stack name
heitorlessa Sep 6, 2022
b6fcc01
chore: add CDK CLI as a project dependency
heitorlessa Sep 6, 2022
dbf7fa1
chore: micro-optimization to not regenerate cdk app if already present
heitorlessa Sep 6, 2022
06e1ca5
docs(maintainers): cleanup structure and parallelization
heitorlessa Sep 6, 2022
6976e28
docs(maintainers): explain framework mechanics
heitorlessa Sep 9, 2022
ddf4a80
docs(maintainers): explain CDK CLI parallelization; reorder TOC
heitorlessa Sep 9, 2022
184d743
docs(maintainers): fix internal link from E2E structure
heitorlessa Sep 9, 2022
ab612ed
docs(maintainers): fix typos
heitorlessa Sep 9, 2022
c07e315
docs(maintainers): clean up structure section
heitorlessa Sep 9, 2022
a800f81
docs(maintainers): typos
heitorlessa Sep 9, 2022
b4d7b94
docs(maintainers): line editing, additional visuals to make it clearer
heitorlessa Sep 9, 2022
83e03cc
docs(maintainers): add note on VSCode subprocess debug
heitorlessa Sep 9, 2022
e845eb1
chore: sync pyproject with pypi
heitorlessa Sep 20, 2022
e834314
chore(deps): old mike 0.6 was yanked (vuln.), move to 1.x
heitorlessa Sep 9, 2022
c5de855
fix(e2e): fix tests
rubenfonseca Sep 20, 2022
be6dc2e
fix(tests): fix tracer e2e tests
rubenfonseca Sep 20, 2022
7f62b29
chore(gitignore): remove cdk context json file
heitorlessa Sep 20, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions .github/workflows/run-e2e-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ jobs:
strategy:
matrix:
# Maintenance: disabled until we discover concurrency lock issue with multiple versions and tmp
# version: ["3.7", "3.8", "3.9"]
version: ["3.7"]
version: ["3.7", "3.8", "3.9"]
# version: ["3.7"]
steps:
- name: "Checkout"
uses: actions/checkout@v3
Expand All @@ -41,6 +41,14 @@ jobs:
python-version: ${{ matrix.version }}
architecture: "x64"
cache: "poetry"
- name: Setup Node.js
uses: actions/setup-node@v3
with:
node-version: "16.12"
- name: Install CDK CLI
run: |
npm install
cdk --version
- name: Install dependencies
run: make dev
- name: Configure AWS credentials
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -310,3 +310,5 @@ site/

!.github/workflows/lib
examples/**/sam/.aws-sam

cdk.out
393 changes: 326 additions & 67 deletions MAINTAINERS.md

Large diffs are not rendered by default.

25 changes: 13 additions & 12 deletions aws_lambda_powertools/tracing/tracer.py
Original file line number Diff line number Diff line change
Expand Up @@ -300,16 +300,6 @@ def handler(event, context):
@functools.wraps(lambda_handler)
def decorate(event, context, **kwargs):
with self.provider.in_subsegment(name=f"## {lambda_handler_name}") as subsegment:
global is_cold_start
logger.debug("Annotating cold start")
subsegment.put_annotation(key="ColdStart", value=is_cold_start)

if is_cold_start:
is_cold_start = False

if self.service:
subsegment.put_annotation(key="Service", value=self.service)

try:
logger.debug("Calling lambda handler")
response = lambda_handler(event, context, **kwargs)
Expand All @@ -325,7 +315,18 @@ def decorate(event, context, **kwargs):
self._add_full_exception_as_metadata(
method_name=lambda_handler_name, error=err, subsegment=subsegment, capture_error=capture_error
)

raise
finally:
global is_cold_start
logger.debug("Annotating cold start")
subsegment.put_annotation(key="ColdStart", value=is_cold_start)

if is_cold_start:
is_cold_start = False

if self.service:
subsegment.put_annotation(key="Service", value=self.service)

return response

Expand Down Expand Up @@ -672,7 +673,7 @@ def _add_response_as_metadata(
if data is None or not capture_response or subsegment is None:
return

subsegment.put_metadata(key=f"{method_name} response", value=data, namespace=self._config["service"])
subsegment.put_metadata(key=f"{method_name} response", value=data, namespace=self.service)

def _add_full_exception_as_metadata(
self,
Expand All @@ -697,7 +698,7 @@ def _add_full_exception_as_metadata(
if not capture_error:
return

subsegment.put_metadata(key=f"{method_name} error", value=error, namespace=self._config["service"])
subsegment.put_metadata(key=f"{method_name} error", value=error, namespace=self.service)

@staticmethod
def _disable_tracer_provider():
Expand Down
41 changes: 41 additions & 0 deletions cdk.context.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
{
"acknowledged-issue-numbers": [
19836
],
"vpc-provider:account=231436140809:filter.isDefault=true:region=eu-west-1:returnAsymmetricSubnets=true": {
"vpcId": "vpc-4d79432b",
"vpcCidrBlock": "172.31.0.0/16",
"availabilityZones": [],
"subnetGroups": [
{
"name": "Public",
"type": "Public",
"subnets": [
{
"subnetId": "subnet-59e15003",
"cidr": "172.31.32.0/20",
"availabilityZone": "eu-west-1a",
"routeTableId": "rtb-cc0a27b5"
},
{
"subnetId": "subnet-c99febaf",
"cidr": "172.31.0.0/20",
"availabilityZone": "eu-west-1b",
"routeTableId": "rtb-cc0a27b5"
},
{
"subnetId": "subnet-d543049d",
"cidr": "172.31.16.0/20",
"availabilityZone": "eu-west-1c",
"routeTableId": "rtb-cc0a27b5"
}
]
}
]
},
"availability-zones:account=231436140809:region=eu-west-1": [
"eu-west-1a",
"eu-west-1b",
"eu-west-1c"
]
}
heitorlessa marked this conversation as resolved.
Show resolved Hide resolved
58 changes: 58 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 7 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
{
"name": "aws-lambda-powertools-python-e2e",
"version": "1.0.0",
"devDependencies": {
"aws-cdk": "2.40.0"
}
}
1 change: 0 additions & 1 deletion parallel_run_e2e.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ def main():
workers = len(list(features)) - 1

command = f"poetry run pytest -n {workers} --dist loadfile -o log_cli=true tests/e2e"
print(f"Running E2E tests with: {command}")
subprocess.run(command.split(), shell=False)


Expand Down
Loading