Problem using layers with multiple files

[jernestosuarez]

Ruby Lambda functions with additional dependencies add lot of files into the layer. Seems that the zip file is being accessed before is available.

I've executed the same tf with and without the dependencie following the doc example

code

module "lambda_function" {
  source = "terraform-aws-modules/lambda/aws"

  function_name = "my-ssm-nofier"
  description   = "SSM notifier"
  handler       = "main.rb"
  runtime       = "ruby2.5"
  publish       = true

  source_path = "./src/lambda-notifier"

  store_on_s3 = true
  s3_bucket   = aws_s3_bucket.lambda-layer.id

  layers = [
    module.lambda_layer_s3.this_lambda_layer_arn,
  ]

  environment_variables = {
    sms_user = "xxx"
    sms_pass = "xxxx"
  }

  tags = {
    Module = "lambda-with-layer"
  }
}

module "lambda_layer_s3" {
  source = "terraform-aws-modules/lambda/aws"

  create_layer = true

  layer_name          = "lambda-layer-s3"
  description         = "My amazing lambda layer (deployed from S3)"
  compatible_runtimes = ["ruby2.5"]

  source_path = "./src/lambda-notifier"

  store_on_s3 = true
  s3_bucket   = aws_s3_bucket.lambda-layer.id
}

Error

module.lambda_function.null_resource.archive[0] (local-exec): cp vendor/bundle/ruby/2.5.0/specifications/zeitwerk-2.3.0.gemspec /var/folders/b5/lb4wb2ms3fg3t4zyzh88bnvw0000gn/T/terraform-aws-lambda-gdt0tztv/vendor/bundle/ruby/2.5.0/specifications/zeitwerk-2.3.0.gemspec
module.lambda_function.null_resource.archive[0] (local-exec): Created: builds/faf3b83c4fdbdf988f6356c44e94ca60f9aa4a063444ddef8b16c1a1eb725755.zip
module.lambda_layer_s3.null_resource.archive[0]: Creation complete after 8s [id=796992783701503752]
module.lambda_function.null_resource.archive[0]: Creation complete after 8s [id=86816938617462868]

Error: Error opening S3 bucket object source (builds/faf3b83c4fdbdf988f6356c44e94ca60f9aa4a063444ddef8b16c1a1eb725755.zip): open builds/faf3b83c4fdbdf988f6356c44e94ca60f9aa4a063444ddef8b16c1a1eb725755.zip: no such file or directory

  on .terraform/modules/lambda_function/terraform-aws-lambda-1.3.0/main.tf line 89, in resource "aws_s3_bucket_object" "lambda_package":
  89: resource "aws_s3_bucket_object" "lambda_package" {



Error: Error opening S3 bucket object source (builds/4c6d5d6ccac8a6971b9f225282462b1d701896f14f7d07a1b5528f3bb97d1b87.zip): open builds/4c6d5d6ccac8a6971b9f225282462b1d701896f14f7d07a1b5528f3bb97d1b87.zip: no such file or directory

  on .terraform/modules/lambda_layer_s3/terraform-aws-lambda-1.3.0/main.tf line 89, in resource "aws_s3_bucket_object" "lambda_package":
  89: resource "aws_s3_bucket_object" "lambda_package" {`

After execution, the zip file is available in the cache

[antonbabenko]

Thanks, @jernestosuarez for opening this issue!

v1.5.0 has been just released with this fix.

Please let us know if the problem persists.

[jernestosuarez]

Hi @antonbabenko , after update the problem persists but seems that the dependency must be added in some other place, cause now is showing the error just one time instead two like before.

module.lambda_function.null_resource.archive[0]: Creation complete after 11s [id=8034055426066007337]
module.lambda_function.aws_s3_bucket_object.lambda_package[0]: Creating...
module.lambda_function.aws_s3_bucket_object.lambda_package[0]: Still creating... [10s elapsed]
module.lambda_function.aws_s3_bucket_object.lambda_package[0]: Creation complete after 13s [id=builds/2d54bf36f039b75b9283512843729eea4544124a4b673a638f25ba09f2563c1c.zip]

Error: Error opening S3 bucket object source (builds/fe7b8e928e2b3ac80e744dd53fdb680fa15fa9585b8c9ff103dd07c7b25588d5.zip): open builds/fe7b8e928e2b3ac80e744dd53fdb680fa15fa9585b8c9ff103dd07c7b25588d5.zip: no such file or directory

  on .terraform/modules/lambda_layer_s3/terraform-aws-lambda-1.3.0/main.tf line 89, in resource "aws_s3_bucket_object" "lambda_package":
  89: resource "aws_s3_bucket_object" "lambda_package" {


Releasing state lock. This may take a few moments...
[terragrunt] 2020/06/10 18:48:28 Hit multiple errors:
exit status 1

If any change is made on the filesystem and the .tf is executed a second time works well.

[antonbabenko]

I can see that it is still pointing to an old version of the module (.terraform/modules/lambda_layer_s3/terraform-aws-lambda-1.3.0/).

Please make sure to remove .terraform and run terraform init.

[jernestosuarez]

My apologies, you're right, now works like a charm : )

Amazing work with this modules, congrats!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.