Use same image for multiple lambda functions

[tuliocasagrande]

Description:

I'd like to build a single image and deploy to different lambda functions by overriding the CMD.

More precisely, I'm copying multiple modules to my container:

my_lambda_image/
├── Dockerfile
├── requirements.txt
└── src
    ├── first_lambda.py
    └── second_lambda.py

And would like to set up accordingly:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Resources:
  rFirstLambdaFunction:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      ImageConfig:
        Command:
          - first_lambda.lambda_handler
    Metadata:
      Dockerfile: Dockerfile
      DockerContext: ./my_lambda_image
      DockerTag: latest

  rSecondLambdaFunction:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      ImageConfig:
        Command:
          - second_lambda.lambda_handler
    Metadata:
      Dockerfile: Dockerfile
      DockerContext: ./my_lambda_image
      DockerTag: latest

However, sam build is adding ImageUri for the first lambda, but not the second. Resulting in the following error for sam deploy:

samcli.commands.package.exceptions.ImageNotFoundError: Image not found for ImageUri parameter of rSecondLambdaFunction resource.

If I manually add ImageUri to the second lambda, sam deploy runs smoothly:

# .aws-sam/build/template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
  rFirstLambdaFunction:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      ImageConfig:
        Command:
        - first_lambda.lambda_handler
      ImageUri: rfirstlambdafunction:latest
    Metadata:
      Dockerfile: Dockerfile
      DockerContext: ./my_lambda_image
      DockerTag: latest
  rSecondLambdaFunction:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      ImageConfig:
        Command:
        - second_lambda.lambda_handler
      ImageUri: rfirstlambdafunction:latest # ADDED MANUALLY
    Metadata:
      Dockerfile: Dockerfile
      DockerContext: ./my_lambda_image
      DockerTag: latest

Steps to reproduce:

Use the minimal template above, run sam build and sam deploy.

Observed result:

$ sam build --debug
2021-01-31 15:29:57,266 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2021-01-31 15:29:57,458 | 'build' command is called
2021-01-31 15:29:57,462 | No Parameters detected in the template
2021-01-31 15:29:57,488 | 2 resources found in the template
2021-01-31 15:29:57,489 | Found Serverless function with name='rFirstLambdaFunction' and ImageUri='None'
2021-01-31 15:29:57,489 | Found Serverless function with name='rSecondLambdaFunction' and ImageUri='None'
2021-01-31 15:29:57,489 | No Parameters detected in the template
2021-01-31 15:29:57,513 | Instantiating build definitions
2021-01-31 15:29:57,516 | Unique function build definition found, adding as new (Function Build Definition: BuildDefinition(None, ., Image, , eef8fdd3-af72-4d77-a304-125f1536e727, {'Dockerfile': 'Dockerfile', 'DockerContext': './my_lambda_image', 'DockerTag': 'latest'}, []), Function: Function(name='rFirstLambdaFunction', functionname='rFirstLambdaFunction', runtime=None, memory=None, timeout=None, handler=None, imageuri=None, packagetype='Image', imageconfig={'Command': ['first_lambda.lambda_handler']}, codeuri='.', environment=None, rolearn=None, layers=[], events=None, metadata={'Dockerfile': 'Dockerfile', 'DockerContext': './my_lambda_image', 'DockerTag': 'latest'}, codesign_config_arn=None))
2021-01-31 15:29:57,516 | Same function build definition found, adding function (Previous: BuildDefinition(None, ., Image, , eef8fdd3-af72-4d77-a304-125f1536e727, {'Dockerfile': 'Dockerfile', 'DockerContext': './my_lambda_image', 'DockerTag': 'latest'}, ['rFirstLambdaFunction']), Current: BuildDefinition(None, ., Image, , bff76818-3018-4a12-97d3-745b1062000c, {'Dockerfile': 'Dockerfile', 'DockerContext': './my_lambda_image', 'DockerTag': 'latest'}, []), Function: Function(name='rSecondLambdaFunction', functionname='rSecondLambdaFunction', runtime=None, memory=None, timeout=None, handler=None, imageuri=None, packagetype='Image', imageconfig={'Command': ['second_lambda.lambda_handler']}, codeuri='.', environment=None, rolearn=None, layers=[], events=None, metadata={'Dockerfile': 'Dockerfile', 'DockerContext': './my_lambda_image', 'DockerTag': 'latest'}, codesign_config_arn=None))
2021-01-31 15:29:57,518 | Building codeuri: . runtime: None metadata: {'Dockerfile': 'Dockerfile', 'DockerContext': './my_lambda_image', 'DockerTag': 'latest'} functions: ['rFirstLambdaFunction', 'rSecondLambdaFunction']
2021-01-31 15:29:57,518 | Building to following folder <REDACTED>/.aws-sam/build/rFirstLambdaFunction
2021-01-31 15:29:57,518 | Building image for rFirstLambdaFunction function
2021-01-31 15:29:57,524 | Setting DockerBuildArgs: {} for rFirstLambdaFunction function
Step 1/5 : FROM public.ecr.aws/lambda/python:3.8
 ---> 2b21444c216d
Step 2/5 : COPY requirements.txt .
 ---> Using cache
 ---> b6ec3121c96a
Step 3/5 : RUN pip install     --no-cache-dir     -r requirements.txt
 ---> Using cache
 ---> 657c2758866f
Step 4/5 : COPY src/ .
 ---> Using cache
 ---> 5748b9100327
Step 5/5 : CMD [ "first_lambda.lambda_handler" ]
 ---> Using cache
 ---> d9efb0a96fed
Successfully built d9efb0a96fed
Successfully tagged rfirstlambdafunction:latest

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Invoke Function: sam local invoke
[*] Deploy: sam deploy --guided

2021-01-31 15:29:57,787 | Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 519, 'exitReason': 'success', 'exitCode': 0, 'requestId': '8ba8e18b-a853-44ca-9ef8-df9da5b469bb', 'installationId': 'c0b677ea-e7aa-4fba-97c8-8f05049a7fdd', 'sessionId': '861f76c4-36af-4db3-9a3e-2fdc6a6f2c1e', 'executionEnvironment': 'CLI', 'pyversion': '3.8.6', 'samcliVersion': '1.15.0'}}]}
2021-01-31 15:29:58,543 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)

sam deploy --debug \
                                --stack-name my-stack-name \
                                --image-repository <REDACTED>/my-lambda-image \
                                --capabilities CAPABILITY_IAM CAPABILITY_NAMED_IAM CAPABILITY_AUTO_EXPAND
2021-01-31 15:30:49,316 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
The push refers to repository [<REDACTED>/my-lambda-image]
70b820efc6c2: Layer already exists
c63e8ac96937: Layer already exists
e4de1f5e339b: Layer already exists
fd69d1d5e19a: Layer already exists
ce82d8247b71: Layer already exists
d6fa53d6caa6: Layer already exists
9c62fd6c35d0: Layer already exists
0db0d030b6f0: Layer already exists
a59cee00f3f7: Layer already exists
rfirstlambdafunction-d9efb0a96fed-latest: digest: sha256:78e7ea385eabecf83b56303dac2bccf858649d4c51070f3a99de9d5ae272e149 size: 2207

2021-01-31 15:30:55,240 | Unable to export
Traceback (most recent call last):
  File "/usr/local/Cellar/aws-sam-cli/1.15.0/libexec/lib/python3.8/site-packages/samcli/lib/package/packageable_resources.py", line 220, in export
    self.do_export(resource_id, resource_dict, parent_dir)
  File "/usr/local/Cellar/aws-sam-cli/1.15.0/libexec/lib/python3.8/site-packages/samcli/lib/package/packageable_resources.py", line 233, in do_export
    uploaded_url = upload_local_image_artifacts(
  File "/usr/local/Cellar/aws-sam-cli/1.15.0/libexec/lib/python3.8/site-packages/samcli/lib/package/utils.py", line 99, in upload_local_image_artifacts
    raise ImageNotFoundError(property_name=property_name, resource_id=resource_id)
samcli.commands.package.exceptions.ImageNotFoundError: Image not found for ImageUri parameter of rSecondLambdaFunction resource.

2021-01-31 15:30:55,243 | Sending Telemetry: {'metrics': [{'commandRun': {'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam deploy', 'duration': 5925, 'exitReason': 'ExportFailedError', 'exitCode': 1, 'requestId': 'c5690171-6494-485a-9c82-856357bc7a55', 'installationId': 'c0b677ea-e7aa-4fba-97c8-8f05049a7fdd', 'sessionId': 'ce7b0b4e-0f6e-40f6-9683-60e2872c8457', 'executionEnvironment': 'CLI', 'pyversion': '3.8.6', 'samcliVersion': '1.15.0'}}]}
2021-01-31 15:30:55,987 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
Error: Unable to upload artifact None referenced by ImageUri parameter of rSecondLambdaFunction resource.
Image not found for ImageUri parameter of rSecondLambdaFunction resource.

Expected result:

I expected to have a single image published to ECR, but with different CMDs.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: macOS Catalina 10.15.7
2. sam --version: SAM CLI, version 1.15.0

[c2tarun]

Hi @tuliocasagrande,

Thanks for bringing this to our attention. This is a bug on our side. During build optimization project we are treating Metadata as a way to decide if a function should be built or not. Since Metadata for both of your function is same it is skipping building second function.

We are working on fixing this issue, however, as a workaround you can add a dummy field with different value to each Function and that'll work.

Sample:

# .aws-sam/build/template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
  rFirstLambdaFunction:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      ImageConfig:
        Command:
        - first_lambda.lambda_handler
      ImageUri: rfirstlambdafunction:latest
    Metadata:
      Dockerfile: Dockerfile
      DockerContext: ./my_lambda_image
      DockerTag: latest
      DummyField: 1
  rSecondLambdaFunction:
    Type: AWS::Serverless::Function
    Properties:
      PackageType: Image
      ImageConfig:
        Command:
        - second_lambda.lambda_handler
      ImageUri: rfirstlambdafunction:latest # ADDED MANUALLY
    Metadata:
      Dockerfile: Dockerfile
      DockerContext: ./my_lambda_image
      DockerTag: latest
      DummyField: 2

[sriram-mv]

#2624 should address this issue.

[mgrandis]

Fixed in https://github.com/aws/aws-sam-cli/releases/tag/v1.18.2