‼️ (aws-lambda-python): SAM - CDK integration is broken in CDK V2.4.0

[moelasmar]

What is the problem?

There is a new change in CDK Version 2.4.0 that makes the bundled assets to be created under the folder structure as following:

├── <asset root>
│   └── asset-input
│       ├── asset files

The issue is SAM expects to get the asset location in metadata property aws:asset:path, so SAM can read the Lambda Function source code, so the customers can do local testing, but in the new Version, the value of the added metadata is the path till the Asset root directory (till the ), although it should be (/asset-input)

For an application that contains a PythonFunction, This is the directory structure after running cdk synth using version 2.4.0

147dda696921:cdk_app melasmar$ cdk --version
2.4.0 (build 993f14d)
147dda696921:cdk_app melasmar$ cd cdk.out
147dda696921:cdk.out melasmar$ tree
.
├── CDKV2SupportDemoStack.assets.json
├── CDKV2SupportDemoStack.template.json
├── asset.993168cfa75b295eeea55bf603340284b9be46ebc079e4965f2c16f5470efda5
│   └── asset-input
│       ├── __init__.py
│       └── app.py
├── cdk.out
├── manifest.json
└── tree.json

and this is the synthesized template:

"MyFunction3BAA72D1": {
      "Type": "AWS::Lambda::Function",
      "Properties": {
              ............   
        },
      "DependsOn": [
        "MyFunctionServiceRole3C357FF2"
      ],
      "Metadata": {
        "aws:cdk:path": "CDKV2SupportDemoStack/MyFunction/Resource",
        "aws:asset:path": "asset.993168cfa75b295eeea55bf603340284b9be46ebc079e4965f2c16f5470efda5",
        "aws:asset:is-bundled": true,
        "aws:asset:property": "Code"
      }

and this is the output in case of using CDK version 2.3.0

147dda696921:cdk_app melasmar$ cdk --version
2.3.0 (build beaa5b2)
147dda696921:cdk_app melasmar$ cd cdk.out
147dda696921:cdk.out melasmar$ tree
.
├── CDKV2SupportDemoStack.assets.json
├── CDKV2SupportDemoStack.template.json
├── asset.8f47b59bc8d67146c1b3fdd46c077e64cf2e0021298beb44f424619eeaaad8d0
│   ├── __init__.py
│   └── app.py
├── cdk.out
├── manifest.json
└── tree.json

and this is the synthesized template:

"MyFunction3BAA72D1": {
      "Type": "AWS::Lambda::Function",
      "Properties": {
         .........
      },
      "DependsOn": [
        "MyFunctionServiceRole3C357FF2"
      ],
      "Metadata": {
        "aws:cdk:path": "CDKV2SupportDemoStack/MyFunction/Resource",
        "aws:asset:path": "asset.8f47b59bc8d67146c1b3fdd46c077e64cf2e0021298beb44f424619eeaaad8d0",
        "aws:asset:is-bundled": true,
        "aws:asset:property": "Code"
      }
    },

Reproduction Steps

• create a simple CDK stack that contains a PythonFunction.
• run cdk synth
• check the synthesized template, and the bundled assets directory

What did you expect to happen?

The value of the added aws:asset:path metadata property should be the location of the source code of the bundled assets.

What actually happened?

The value of the added aws:asset:path metadata property is the root asset directory.

CDK CLI Version

2.4.0

Framework Version

No response

Node.js Version

v14.15.4

OS

mac os

Language

Typescript

Language Version

No response

Other information

No response

[moelasmar]

This issue also exist in CDK versions 1.137.0 and 1.138.0

[moelasmar]

I believe the issue here

aws-cdk/packages/@aws-cdk/aws-lambda-python/lib/bundling.ts

Line 90 in 272b6b1

bundlingCommands.push(`cp -R ${options.inputDir} ${options.outputDir}`);

cp -R src dist will copy the src directory to dist directory instead of copying the src content to dist. It should be cp -R src/ dist

[njlynch]

Thanks for reporting this. We have reproduced and are working on a patch, and a release to fix the issue.

For v2 users, you can remain on 2.3.0-alpha.0 version of @aws-cdk/aws-lambda-python-alpha, even while upgrading to 2.4.0 of aws-cdk-lib.

[moelasmar]

Thanks @njlynch .. This issue also exist in versions 1.137.0, and 1.138.0. Are you going to release a fix for these versions as well?

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[otaviomacedo]

Pending release for v2.

[jrasky]

I might be doing something wrong, but I get the synth failures due to the issue above on 1.138.0 and 1.138.1 (python cdk). 1.137.0 works correctly.

Bundling asset StockUpdater/Beta/StockUpdater/list_symbols/Code/Stage...
cp: cannot copy a directory, ‘/asset-input/’, into itself, ‘asset-output’
jsii.errors.JavaScriptError:
  Error: Failed to bundle asset StockUpdater/Beta/StockUpdater/list_symbols/Code/Stage, bundle output is located at C:\Users\Jerome\Documents\scripts\python\Stock-Updater\cdk.out\asset.7c69e747efb65e72ce77df91aa2dd02dff645ce37b10efeac7f4b55d94d28f09-error: Error: docker exited with status 1

[njlynch]

See also this comment from @chrispykim: #18082 (comment)

{
  "errorMessage": "Unable to import module 'hello_lambda': No module named 'hello_lambda'",
  "errorType": "Runtime.ImportModuleError",
  "stackTrace": []
}

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[raincoastchris]

Patch released: https://github.com/aws/aws-cdk/releases/tag/v1.138.1

@otaviomacedo can you confirm this is fix causes python lambdas to work as expected in cdk>=1.138.1? I have a project in which Python code is still bundled at <asset-root>/asset-input/ instead of at <asset-root>/ as expected, when using both 1.138.1 and 1.138.2. For now I will pin CDK to 1.136.0, but I don't think the issue in CDK 1.x is resolved.

[otaviomacedo]

@raincoastchris thanks for calling this out mergify closed the issue automatically. I'm reopening, as the actual fix will be available in the next release.

[setu4993]

This is resolved for me in https://github.com/aws/aws-cdk/releases/tag/v1.139.0.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[Z11]

This is still an issue on CDK Python since it's still creating the asset directory and lambda is unable to find the handler. The workaround was to stay on 2.3.0-alpha.0 as stated by @njlynch :

aws-cdk.aws-lambda-python-alpha==2.3.0a0

[setu4993]

@Z11 : Interesting. I just used v1.140.0 with TypeScript and it worked fine, so this might be a CDK Python-specific issue. Makes me wonder if there's something with the JSII interpretation / transpilation.

[ghost]

Currently working with the following:
aws-cdk-lib==2.5.0
aws-cdk.aws-lambda-python-alpha==2.3.0a0

[gshpychka]

@corymhall can you reopen this? It's still an issue.

[lcastroemello]

@corymhall I am having this issue currently, with aws-cdk-lib: 2.40.0 and aws-cdk/aws-lambda-python-alpha:2.40.0a0 (typescript stack with a lambdaPython).