SNS to SQS incompetency and incomplete documentation

[jianhuawuchn]

following the URL of
https://www.terraform.io/docs/providers/aws/r/sns_topic_subscription.html

does not get the job "sending message from SNS to SQS done". As up to this stage, there is no permission set aside in SQS queue which allows to receive message from SNS.

see this link for reference http://docs.aws.amazon.com/sns/latest/dg/SendMessageToSQS.html#SendMessageToSQS.arn

we could use "policy" argument in ws_sqs_queue to achieve our intention, however, since we could not use variable reference in JSON format policy, after setting up SNS and SQS via terraform tool, we had to login to AWS SQS console, and do permission changes on the permission tab.

[jianhuawuchn]

the terraform documentation has to be changed, as it doesn't support email protocol in SNS, so we are bound to use SQS. or we need to hard code ARN of working SNS and SQS in to terraform variable file.

[stack72]

Hi @jianhuawuchn

Thanks for reporting the issue here. Apologies that the documentation isn't up to par in this area. Do you know what the changes to the docs should be? I haven't worked with these resources much and would love to be able to get them fixed up. We would love for you to open a PR with the suggested changes to help make it better

Thanks

Paul

[jianhuawuchn]

the most easy fix is to ask users to login to their AWS SQS console, and make a subscribe from within SQS console page (this is different from the subscribe on AWS SNS console), after running "terraform apply" command.

If there is an SNS topic applied prior defining SQS queue, you can change

resource "aws_sqs_queue" "user_updates_queue" {
    name = "user-updates-queue"
}

to

resource "aws_sqs_queue" "user_updates_queue" {
    name = "user-updates-queue"
    policy = {
  "Version": "2012-10-17",
  "Id": "ID_of_SQS",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": "*",
      "Action": "SQS:SendMessage",
      "Resource": "arn_of_sqs,
      "Condition": {
        "ArnEquals": {
          "aws:SourceArn": "arn_of_sns_topic"
        }
      }

  ]
}
}
}

however, in reality, most of time, we just use SNS with email protocol. we SQS with other applications like ELK.

Terraform doesn't support SNS with email protocol directly, but we can use aws_cloudformation_stack resource to set up SNS via AWS cloudformation.

[jedi4ever]

maybe related to #3549

[dekz]

Seems related to #4157

[AndHei]

@jianhuawuchn

I commented in #6909 with an example that shows how to create SNS topics and SQS queues that are subscribed to each other without requiring any manual interaction in the AWS Console or the API.

Key for success are the two "aws_iam_policy_document" data items which make sure that the SNS topic can post messages to the SQS queue as well as the SQS queue can subscribe to the SNS topic.

If you don't require cross region / cross account SNS to SQS subscriptions than you can simplify the providers section to a single provider.

@stack72

I agree with @jianhuawuchn that the documentation requires improvement. Do you think it would make sense to put my whole example from #6909 into the documentation?

@jedi4ever @dekz

#5973 seems unrelated to #3549 / #4157

[AndHei]

I added #9838 to update documentation with notes and examples about cross account / region SNS topic to SQS queue subscription.

[stack72]

Hi Friends, does the PR added by @AndHei close this issue?

[mitchellh]

Hello! I'm going through waiting-response labeled issues and closing issues that haven't been responded to with a meaningful response in at least 2+ weeks. If this is still an issue, please open a new issue so we can start anew. Feel free to reference this existing issue to make a link between the two. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.