Terraform aws provider invalid arn for s3 lambda permission

[kalpaitch]

Packaging the chalice app for terraform is giving me an invalid source_arn error for the aws_lambda_permission s3event "source_arn": "arn:*:s3:::lambda-function-name

Error: "source_arn" (arn:*:s3:::lambda-function-name) is an invalid ARN:
invalid partition value (expecting to match regular expression: ^aws(-[a-z]+)*$)

I can't determine whether this is an issue with the validation of the provider or with how chalice is packaging this up?

chalice/chalice/package.py

Line 853 in ba93124

'source_arn': 'arn:*:s3:::%s' % resource.bucket

chalice 1.18.1 & 1.19.0, python 3.8.0, terraform aws provider 3.4.0

[jamesls]

I believe this is related to the changes from #1417, which add suport for non-aws partitions.

The validation from terraform looks overly restrictive. This is a valid resource ARN. From the IAM docs:

You can use wildcards as part of the resource ARN. You can use wildcard characters (* and ?) within any ARN segment (the parts separated by colons). An asterisk (*) represents any combination of characters and a question mark (?) represents any single character.

That being said, I think we can get around this if we change the pattern to be arn:aws*:s3:::. Let me confirm that works real quick before making that change.

EDIT: I confirmed that arn:*:s3:::... is a valid ARN but when I try to use arn:aws*:s3::: it fails with this message:

Partition "aws*" is not valid for resource "arn:aws*:s3:::fdgksjfdkjh".

So unfortunately my suggested fix won't work. The wildcard was chosen so that the packaging configuration/region didn't have to match the configuration/region that was doing the actual deployment (e.g. terraform apply). Preferably we'd get terraform to update their validation but I'm not sure if there's a reason for their existing validation.

Related terraform issue: hashicorp/terraform-provider-aws#9972

[kalpaitch]

Thanks for this update and for identifying that issue.

On a semi separate note is arn:*:s3:::lambda-function-name specific enough, does it not give permission to another lambda function with the same name in a different region. Is there a way that external resources such as s3 buckets, sqs queues etc could be identified in the chalice config in a similar way that the api_gateway_custom_domain can??

[jamesls]

Yes we should be able to do that. At this point, I think that's our best option, fill in the ARN to be as specific as possible.

[kapilt]

a previous version in #1417 used a terraform data partition to construct arns, that should be used again to fix these issues, it was removed for reasons of inconsistencies on dns suffix, but it should be retained for arn construction.

[jamesls]

@kapilt Should be fixed in #1550. I switched it over to ${data.aws_partition.chalice.partition}.