-
Notifications
You must be signed in to change notification settings - Fork 403
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Maintenance: Create a Powertools Lambda layer for each Python version #3859
Comments
One small note to also increase our canary testing across all Python versions |
Assigning to @sthulb! |
This is the main blocker preventing us from releasing Powertools v3, so I'm taking over this to release v3 as soon as possible. I'm working on this since last Friday and I've made some progress. I can now build the Powertools layer using the current five supported Python versions, but I still have some issues we need to work together to get them passed: 1 - We are currently building the Powertools layer using Python 3.8 and need make this flexible to accept any version of python. We need to pay attention to Python 3.12 because it is using Amazon Linux 2023 instead of AL2, which may require us to create a specific Dockerfile for this. Maybe using arguments, maybe using multistage build, idk, need to test. Can you help with this item @am29d? 2 - We are currently using Buildx + QEMU to emulate arch arm64 on x86 servers, but it looks like we need to improve things here to make it work properly. If we look at this line we are uploading the file twice with the same name, which forces a replacement of the file and is ALWAYS using the arm64. Fortunately, our current layer does not have any pre-compiled libraries and therefore works correctly. Am I wrong @heitorlessa? 3 - I need help thinking about how to show all these new ARNs on our documentation. We also need to think about GovCloud layers. I think with some help on a few items we will unblock this item soon and we will move towards the V3 release. Thanks |
which is two tabs one for x86 and one for ARM, with each one containing the ARNs for each region. In my opinion, Gov Cloud could just be another entry in that list, just like any other region. Regarding the others, before making a suggestion - it'd be helpful for me to have an example of the ARNs we'll have after making this change. |
Python 3.12 is failing when using AWS based images due to using AL2023. I know how to solve it, but I'll leave it until the end Dockerfile:17
--------------------
path /tmp/jsii-kernel-Rx5134/node_modules/cdk-aws-lambda-powertools-layer/layer/Python
15 | # PACKAGE_SUFFIX = ''
16 |
17 | >>> RUN yum update -y && yum install -y zip unzip wget tar gzip binutils
18 |
19 | # Install build essentials
--------------------
ERROR: failed to solve: process "/bin/sh -c yum update -y && yum install -y zip unzip wget tar gzip binutils" did not complete successfully: exit code: 127
jsii.errors.JavaScriptError:
Error: docker exited with status 1
--> Command: docker build -t cdk-ff22dfe9b0c5c12f271427156b61df72c8c53b32355ffe9b809b982ecc7b23ec --platform "linux/amd64" --build-arg "PACKAGE_SUFFIX=[all]==1.0.0" --build-arg "PYTHON_VERSION=3.12" "/tmp/jsii-kernel-Rx5134/node_modules/cdk-aws-lambda-powertools-layer/layer/Python"
at dockerExec (/tmp/jsii-kernel-Rx5134/node_modules/aws-cdk-lib/core/lib/private/asset-staging.js:2:237)
at DockerImage.fromBuild (/tmp/jsii-kernel-Rx5134/node_modules/aws-cdk-lib/core/lib/bundling.js:1:4761)
at Code.fromDockerBuild (/tmp/jsii-kernel-Rx5134/node_modules/aws-cdk-lib/aws-lambda/lib/code.js:5:531)
at new LambdaPowertoolsLayer (/tmp/jsii-kernel-Rx5134/node_modules/cdk-aws-lambda-powertools-layer/lib/lambda-powertools-layer.js:59:49)
at Kernel._Kernel_create (/tmp/tmpxfbwozhn/lib/program.js:10119:25)
at Kernel.create (/tmp/tmpxfbwozhn/lib/program.js:9790:93)
at KernelHost.processRequest (/tmp/tmpxfbwozhn/lib/program.js:11707:36)
at KernelHost.run (/tmp/tmpxfbwozhn/lib/program.js:11667:22)
at Immediate._onImmediate (/tmp/tmpxfbwozhn/lib/program.js:11668:46)
at process.processImmediate (node:internal/timers:476:21) |
Work in progress! |
Closed via #4826 |
|
This is now released under 2.43.0 version! |
Why is this needed?
Powertools is growing very fast and we are adding more features day by day. Some of these features require compiled libraries and those libraries must be compiled for the specific version of Python. An example of this is aws-encryption-sdk which brings cryptography as a dependency. We had an issue (#3852) recently and we need to implement it to avoid more problems in the future.
Which area does this relate to?
Other
Solution
Build the Powertools Lambda layer for the specific version of Python.
Acknowledgment
The text was updated successfully, but these errors were encountered: