Cannot find asset with NodejsFunction created asset

[mbergkvist]

I trying to deploy a NodejsFunction with a stackset but I can't get the file asset to work. I get the following error

Error: Cannot find asset at /Users/markus/src/awsCore-tagScheduler/infrastructure/cdk.out/assembly-awsCoreTagScheduler-awsCoreTagScheduler-frankfurt-dev/Users/markus/src/awsCore-tagScheduler/infrastructure

It looks like it's adding the asset directory as source to the bucketDeployment instead of the asset filename. Can't figure out why though.

How to reproduce

A simplified example of what I'm trying to accomplish, more or less following the example in the project README.

export class TagSchedulerStage extends Stage {
  constructor(
    scope: Construct,
    id: string,
    props: StageProps,
  ) {
    super(scope, id, props);

    const bucketStack = new Stack(this, 'BucketStack');
    const bucket = new s3.Bucket(bucketStack, 'Assets', {
      bucketName: 'cdkstacket-asset-bucket-xyz',
    });
    bucket.addToResourcePolicy(
      new iam.PolicyStatement({
        actions: ['s3:Get*', 's3:List*'],
        resources: [bucket.arnForObjects('*'), bucket.bucketArn],
        principals: [new iam.OrganizationPrincipal('ou-ab12')],
      }),
    );

    const stack = new Stack(this, 'TopLevelStack');

    const stackSetStack = new StackSetStack(stack, 'MyStackSetStack', {
      assetBucket: bucket,
    });

    new lambdaNodejs.NodejsFunction(
      stackSetStack,
      'MyLambdaHandler',
      {
        handler: 'handler',
        entry: path.join(__dirname, '../../src/function.ts'),
      },
    );

    new StackSet(stack, 'MyStackSet', {
      capabilities: [Capability.IAM],
      deploymentType: DeploymentType.serviceManaged(),
      target: StackSetTarget.fromAccounts({
        regions: ['eu-central-1'],
        accounts: ['123456789012'],
      }),
      template: StackSetTemplate.fromStackSetStack(stackSetStack),
    });
  }
}

Any advice?

[mbergkvist]

I have put up an example application here, https://github.com/mbergkvist/stacksetsandassets

The command I execute is

npm run cdk list

and the error is

Error: Cannot find asset at /Users/markus/tmp/stacksetsandassets/cdk.out/Users/markus/tmp/stacksetsandassets

[mbergkvist]

Some updates:

• npm run cdk synth -- --all works fine
• I get the exact same error with RustFunction
• I do not get any error with PythonFunction, neither with cdk synth nor cdk list
• PythonFunction have another (related?) issue related to bundling and uploading the lambda asset, where the S3Key for the asset in the lambda stack is not the same as the one uploaded by the S3BucketDeployment.

The example in https://github.com/mbergkvist/stacksetsandassets is updated to include PythonFunction.

[josh-demuth]

thanks for reporting this. i tested a nodejs function with the recent updates in #325 and it seems to work OK. there is an issue with python lambda with the s3 asset file name that gets produced that we will look into.

[mbergkvist]

thanks for reporting this. i tested a nodejs function with the recent updates in #325 and it seems to work OK. there is an issue with python lambda with the s3 asset file name that gets produced that we will look into.

I'll take #325 for a spin on my example repo, testing it with both nodejs function and Rust function.

[mbergkvist]

@josh-demuth Tested #325 with the project in mbergkvist/stacksetsandassets#1 but I still get the same error. It looks like it's adding the asset directory as source to the bucketDeployment instead of the asset filename.

[josh-demuth]

yes was testing more and sometimes it would work and sometimes not. the s3 bucket deployment process is getting a different hash. the lambda asset process creates a directory and not a zip at this stage of the process. are you seeing a .zip file be created in your cdk.out?

[mbergkvist]

There is a zip-file asset.3322b7049fb0ed2b7cbb644a2ada8d1116ff80c32dca89e6ada846b5de26f961.zip. The content of the file is the awscli, not the lambda-code. There is also a folder asset.e976a796f036a5efbf44b99e44cfb5a961df08d8dbf7cd37e60bf216fb982a00 with a Python script that looks like a Custom Resource handler, I assume it comes from the CDK.

[mbergkvist]

Updated to cdk-stackset-0.0.150 (release of #325) in mbergkvist/stacksetsandassets#1. Unfortunately getting the same error.

[atali]

Hello team, I am experimenting the same issue when running cdk ls

Error: Cannot find asset at /Users/myfolder/cdk.out/Users/myfolder
    at new AssetStaging (/Users/myfolder/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:2119)
    at new Asset (/Users/myfolder/node_modules/aws-cdk-lib/aws-s3-assets/lib/asset.js:1:1141)
    at Object.bind (/Users/myfolder/node_modules/aws-cdk-lib/aws-s3-deployment/lib/source.js:1:1460)
    at BucketDeployment.addSource (/Users/myfolder/node_modules/aws-cdk-lib/aws-s3-deployment/lib/bucket-deployment.js:1:6709)
    at StackSetStackSynthesizer.addFileAsset (/Users/myfolder/node_modules/cdk-stacksets/src/stackset-stack.ts:96:42)
    at new Asset (/Users/myfolder/node_modules/aws-cdk-lib/aws-s3-assets/lib/asset.js:1:1623)
    at AssetCode.bind (/Users/myfolder/node_modules/aws-cdk-lib/aws-lambda/lib/code.js:5:3487)
    at new Function (/Users/myfolder/node_modules/aws-cdk-lib/aws-lambda/lib/function.js:1:10003)
    at new NodejsFunction (/Users/myfolder/node_modules/aws-cdk-lib/aws-lambda-nodejs/lib/function.js:1:2111)
    at new ScheduledReport (/Users/myfolder/src/constructs/scheduled-reports/scheduled-report.ts:46:24)
    ```

[mgroff2]

I found this thread with high hopes until I clicked and saw it was still open. I am too experienceing this issue.

Error: Cannot find asset at /Users/mypath/cdk.out/Users/mypath
    at new AssetStaging (/Users/mypath/node_modules/aws-cdk-lib/core/lib/asset-staging.js:1:2119)
    at new Asset (/Users/mypath/node_modules/aws-cdk-lib/aws-s3-assets/lib/asset.js:1:1141)
    at Object.bind (/Users/mypath/node_modules/aws-cdk-lib/aws-s3-deployment/lib/source.js:1:1460)
    at BucketDeployment.addSource (/Users/mypath/node_modules/aws-cdk-lib/aws-s3-deployment/lib/bucket-deployment.js:1:6709)
    at StackSetStackSynthesizer.addFileAsset (/Users/mypath/node_modules/cdk-stacksets/src/stackset-stack.ts:96:42)
    at new Asset (/Users/mypath/node_modules/aws-cdk-lib/aws-s3-assets/lib/asset.js:1:1623)
    at AssetCode.bind (/Users/mypath/node_modules/aws-cdk-lib/aws-lambda/lib/code.js:5:3487)
    at new Function (/Users/mypath/node_modules/aws-cdk-lib/aws-lambda/lib/function.js:1:10003)
    at new NodejsFunction (/Users/mypath/node_modules/aws-cdk-lib/aws-lambda-nodejs/lib/function.js:1:2111)
    at new ScheduledReport (/Users/mypath/src/constructs/scheduled-reports/scheduled-report.ts:45:24)

[jtirrell-amazon]

thanks for reporting this. i tested a nodejs function with the recent updates in #325 and it seems to work OK. there is an issue with python lambda with the s3 asset file name that gets produced that we will look into.

This is an issue that I am seeing, if I create the lambda function using the bundler the deployment is unable to find the file in the s3 bucket.
ie:

    const functionHandler = new cdk.aws_lambda.Function(this, "server", {
      code: cdk.aws_lambda.Code.fromAsset(`${lambdaSrcDir}/api/`,{
        bundling: {
        image: Runtime.PYTHON_3_12.bundlingImage,
        command: [
          'bash', '-c',
          'pip install -r requirements.txt -t /asset-output && cp -au . /asset-output'
        ],
      },
      }),
      handler: "server.lambda_handler",
      runtime: cdk.aws_lambda.Runtime.PYTHON_3_12,
      timeout: cdk.Duration.minutes(1),
      environment: {
        CLEAN_DATA_BUCKET: secureSightCleanBucket.bucketName,
        ORG_NAME: secureSightConfig.organizationName,
      },
      role: roleSecureSightServer,
    });

If I manually package the file and add the zip file as an asset, everything works fine.

[achernev]

I hit the same issue and discovered that if the bundling options are omitted, then the asset archive filename (the SHA256 one) will be identical to the one in the staging bucket. This is a workaround for the problem, if you can omit the bundling options or bundle the function code externally.

Example:

# This produces error because the asset hash is different from the staging asset hash:
on_event_handler_function = lambda_.Function(
    self,
    id="OnEventFunction",
    function_name=on_event_handler_function_name,
    runtime=lambda_.Runtime.PYTHON_3_13,
    role=on_event_handler_role,
    code=lambda_.Code.from_asset(
        os.path.join(os.path.dirname(__file__), "lambda", "dependency_signal"),
        bundling=BundlingOptions(
            image=lambda_.Runtime.PYTHON_3_13.bundling_image,
            command=[
                "bash",
                "-c",
                "pip install --no-cache-dir -r requirements.txt -t /asset-output && cp -au . /asset-output",
            ],
        ),
    ),
    timeout=Duration.minutes(5),
    log_retention=logs.RetentionDays.ONE_WEEK,
    handler="index.on_event",
)

# This DOES NOT produce error because the asset hash is identical to the staging asset hash:
on_event_handler_function = lambda_.Function(
    self,
    id="OnEventFunction",
    function_name=on_event_handler_function_name,
    runtime=lambda_.Runtime.PYTHON_3_13,
    role=on_event_handler_role,
    code=lambda_.Code.from_asset(os.path.join(os.path.dirname(__file__), "lambda", "dependency_signal")),
    timeout=Duration.minutes(5),
    log_retention=logs.RetentionDays.ONE_WEEK,
    handler="index.on_event",
)

Apologies if I am not using the correct terminology, I am fairly new to this.