Fix #34 & Fix #32 & Expose Lambda Environment Variables

CHANGELOG.md

@@ -1,5 +1,10 @@
 # Change Log for Terraform AWS Kinesis Firehose Splunk
 
+## v8.2.0
+ * Fix [#34](https://github.com/disney/terraform-aws-kinesis-firehose-splunk/issues/34) - Add documentation note in README.md for Splunk Cloud customers. Thanks[@out-of-mana](https://github.com/out-of-mana)
+ * Fix [#32](https://github.com/disney/terraform-aws-kinesis-firehose-splunk/pull/32) - Enable Cloudwatch Logs Access From Multiple Regions. `var.region` is now Deprecated. Thanks [@bogdannazarenko](https://github.com/bogdannazarenko)
+ * Expose Lambda environment variables. Thanks [@tlopo](https://github.com/tlopo). 
+
 ## v8.1.0
  * Change `var.name_cloudwatch_logs_to_ship` to be non-mandatory. It will now default to `null` and the subscription filter will not be created if it is `null`. See `var.cloudwatch_log_group_names_to_ship` to create subscription filters to multiple log groups.
  * Fix [#27](https://github.com/disney/terraform-aws-kinesis-firehose-splunk/issues/27) - Add `var.cloudwatch_log_group_names_to_ship` to allow creating subscription filters to multiple log groups.

README.md

@@ -14,7 +14,9 @@ Once you have received the token, you can proceed forward in creating a `module`
 ```hcl
 module "kinesis_firehose" {
   source                             = "disney/kinesis-firehose-splunk/aws"
+  version                            = "<version>"
   region                             = "us-east-1"
+  cloudwatch_log_regions             = ["us-east-1", "us-west-2"]
   name_cloudwatch_logs_to_ship       = "/test/test01"
   cloudwatch_log_group_names_to_ship = ["/aws/svc/loggroup1", "log-group-2", "/aws/svc2/loggroup"]
   hec_url                            = "<Splunk_Kinesis_ingest_URL>"
@@ -27,6 +29,11 @@ module "kinesis_firehose" {
 ```
 Please see the [S3 Life Cycle Rule example](examples/s3_bucket_lifecycle_rule.md) if you wish to configure them.
 
+## Splunk Cloud Customers
+If you are a Splunk Cloud customer, once you have successfully deployed all the resources, you will need to ensure that your Splunk Cloud instance has the Kinesis Data Firehose egress CIDRs allow listed under `Server Settings > IP Allow List Management > HEC access for ingestion`.
+
+For more details on the relevant CIDRs please reference this [article](https://docs.aws.amazon.com/firehose/latest/dev/controlling-access.html#using-iam-splunk-vpc).
+
 ### Upgrading from v6.0.0 to v7.0.0
 
 If you choose to change the way you pass in your HEC token (see section below) when upgrading from v6.0.0 to v7.0.0, when you run `terraform apply`, you _might_ run into Terraform reporting that it is going to make changes to resources such as IAM policies when nothing has changed with them. Others have experienced this issue as well, please see this [issue](https://github.com/hashicorp/terraform/issues/32849).
@@ -89,6 +96,7 @@ As of v7.0.0, there are two additional options available to pass in the HEC toke
 | [archive_file.lambda_function](https://registry.terraform.io/providers/hashicorp/archive/latest/docs/data-sources/file) | data source |
 | [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
 | [aws_iam_policy_document.cloudwatch_to_fh_access_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
+| [aws_iam_policy_document.cloudwatch_to_firehose_trust_assume_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
 | [aws_iam_policy_document.kinesis_firehose_policy_document](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
 | [aws_iam_policy_document.lambda_policy_doc](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
 | [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
@@ -98,14 +106,13 @@ As of v7.0.0, there are two additional options available to pass in the HEC toke
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
 | <a name="input_hec_url"></a> [hec\_url](#input\_hec\_url) | Splunk Kinesis URL for submitting CloudWatch logs to splunk | `string` | n/a | yes |
-| <a name="input_region"></a> [region](#input\_region) | The region of AWS you want to work in, such as us-west-2 or us-east-1 | `string` | n/a | no |
-| <a name="input_cloudwatch_log_regions"></a> [region](#input\_cloudwatch_log_regions) | List of regions to allow CloudWatch logs to be shipped from. Set in Kinesis Firehose role's trust polucy | `list(string)` | n/a | no |
 | <a name="input_s3_bucket_name"></a> [s3\_bucket\_name](#input\_s3\_bucket\_name) | Name of the s3 bucket Kinesis Firehose uses for backups | `string` | n/a | yes |
 | <a name="input_arn_cloudwatch_logs_to_ship"></a> [arn\_cloudwatch\_logs\_to\_ship](#input\_arn\_cloudwatch\_logs\_to\_ship) | arn of the CloudWatch Log Group that you want to ship to Splunk. | `string` | `null` | no |
 | <a name="input_aws_s3_bucket_versioning"></a> [aws\_s3\_bucket\_versioning](#input\_aws\_s3\_bucket\_versioning) | Versioning state of the bucket. Valid values: Enabled, Suspended, or Disabled. Disabled should only be used when creating or importing resources that correspond to unversioned S3 buckets. | `string` | `null` | no |
 | <a name="input_cloudwach_log_group_kms_key_id"></a> [cloudwach\_log\_group\_kms\_key\_id](#input\_cloudwach\_log\_group\_kms\_key\_id) | KMS key ID of the key to use to encrypt the Cloudwatch log group | `string` | `null` | no |
 | <a name="input_cloudwatch_log_filter_name"></a> [cloudwatch\_log\_filter\_name](#input\_cloudwatch\_log\_filter\_name) | Name of Log Filter for CloudWatch Log subscription to Kinesis Firehose | `string` | `"KinesisSubscriptionFilter"` | no |
 | <a name="input_cloudwatch_log_group_names_to_ship"></a> [cloudwatch\_log\_group\_names\_to\_ship](#input\_cloudwatch\_log\_group\_names\_to\_ship) | List of CloudWatch Log Group names that you want to ship to Splunk. | `list(string)` | `null` | no |
+| <a name="input_cloudwatch_log_regions"></a> [cloudwatch\_log\_regions](#input\_cloudwatch\_log\_regions) | List of regions to allow CloudWatch logs to be shipped from. Set in Kinesis Firehose role's trust polucy | `list(string)` | `[]` | no |
 | <a name="input_cloudwatch_log_retention"></a> [cloudwatch\_log\_retention](#input\_cloudwatch\_log\_retention) | Length in days to keep CloudWatch logs of Kinesis Firehose | `number` | `30` | no |
 | <a name="input_cloudwatch_to_fh_access_policy_name"></a> [cloudwatch\_to\_fh\_access\_policy\_name](#input\_cloudwatch\_to\_fh\_access\_policy\_name) | Name of IAM policy attached to the IAM role for CloudWatch to Kinesis Firehose subscription | `string` | `"KinesisCloudWatchToFirehosePolicy"` | no |
 | <a name="input_cloudwatch_to_firehose_trust_iam_role_name"></a> [cloudwatch\_to\_firehose\_trust\_iam\_role\_name](#input\_cloudwatch\_to\_firehose\_trust\_iam\_role\_name) | IAM Role name for CloudWatch to Kinesis Firehose subscription | `string` | `"CloudWatchToSplunkFirehoseTrust"` | no |
@@ -125,6 +132,7 @@ As of v7.0.0, there are two additional options available to pass in the HEC toke
 | <a name="input_kinesis_firehose_iam_policy_name"></a> [kinesis\_firehose\_iam\_policy\_name](#input\_kinesis\_firehose\_iam\_policy\_name) | Name of the IAM Policy attached to IAM Role for the Kinesis Firehose | `string` | `"KinesisFirehose-Policy"` | no |
 | <a name="input_kinesis_firehose_lambda_role_name"></a> [kinesis\_firehose\_lambda\_role\_name](#input\_kinesis\_firehose\_lambda\_role\_name) | Name of IAM Role for Lambda function that transforms CloudWatch data for Kinesis Firehose into Splunk compatible format | `string` | `"KinesisFirehoseToLambaRole"` | no |
 | <a name="input_kinesis_firehose_role_name"></a> [kinesis\_firehose\_role\_name](#input\_kinesis\_firehose\_role\_name) | Name of IAM Role for the Kinesis Firehose | `string` | `"KinesisFirehoseRole"` | no |
+| <a name="input_lambda_function_environment_variables"></a> [lambda\_function\_environment\_variables](#input\_lambda\_function\_environment\_variables) | Environment variables for the lambda function | `map(string)` | `{}` | no |
 | <a name="input_lambda_function_name"></a> [lambda\_function\_name](#input\_lambda\_function\_name) | Name of the Lambda function that transforms CloudWatch data for Kinesis Firehose into Splunk compatible format | `string` | `"kinesis-firehose-transform"` | no |
 | <a name="input_lambda_function_timeout"></a> [lambda\_function\_timeout](#input\_lambda\_function\_timeout) | The function execution time at which Lambda should terminate the function. | `number` | `180` | no |
 | <a name="input_lambda_iam_policy_name"></a> [lambda\_iam\_policy\_name](#input\_lambda\_iam\_policy\_name) | Name of the IAM policy that is attached to the IAM Role for the lambda transform function | `string` | `"Kinesis-Firehose-to-Splunk-Policy"` | no |
@@ -142,6 +150,7 @@ As of v7.0.0, there are two additional options available to pass in the HEC toke
 | <a name="input_object_lock_configuration_mode"></a> [object\_lock\_configuration\_mode](#input\_object\_lock\_configuration\_mode) | Default Object Lock retention mode you want to apply to new objects placed in the specified bucket. Valid values: COMPLIANCE, GOVERNANCE | `string` | `null` | no |
 | <a name="input_object_lock_configuration_token"></a> [object\_lock\_configuration\_token](#input\_object\_lock\_configuration\_token) | S3 bucket object lock configuration token | `string` | `null` | no |
 | <a name="input_object_lock_configuration_years"></a> [object\_lock\_configuration\_years](#input\_object\_lock\_configuration\_years) | Required if days is not specified. Number of years that you want to specify for the default retention period | `number` | `null` | no |
+| <a name="input_region"></a> [region](#input\_region) | DEPRECATED. The region of AWS you want to work in, such as us-west-2 or us-east-1 (deprecated: use `var.cloudwatch_log_regions` instead) | `string` | `null` | no |
 | <a name="input_s3_backup_mode"></a> [s3\_backup\_mode](#input\_s3\_backup\_mode) | Defines how documents should be delivered to Amazon S3. Valid values are FailedEventsOnly and AllEvents. | `string` | `"FailedEventsOnly"` | no |
 | <a name="input_s3_bucket_block_public_access_enabled"></a> [s3\_bucket\_block\_public\_access\_enabled](#input\_s3\_bucket\_block\_public\_access\_enabled) | Set to 1 if you would like to add block public access settings for the s3 bucket Kinesis Firehose uses for backups | `number` | `0` | no |
 | <a name="input_s3_bucket_key_enabled"></a> [s3\_bucket\_key\_enabled](#input\_s3\_bucket\_key\_enabled) | Whether or not to use Amazon S3 Bucket Keys for SSE-KMS. | `bool` | `null` | no |

main.tf

@@ -1,7 +1,7 @@
 locals {
   lambda_function_source_file = var.local_lambda_file != null ? var.local_lambda_file : "${path.module}/files/kinesis-firehose-cloudwatch-logs-processor.js"
   lambda_function_handler     = var.local_lambda_file_handler != null ? var.local_lambda_file_handler : "kinesis-firehose-cloudwatch-logs-processor.handler"
-  cloudwatch_log_regions = var.region == null ? var.cloudwatch_log_regions : [var.region]
+  cloudwatch_log_regions      = var.region == null ? var.cloudwatch_log_regions : [var.region]
 }
 
 # Kenisis firehose stream
@@ -309,9 +309,9 @@ resource "aws_lambda_function" "firehose_lambda_transform" {
   timeout                        = var.lambda_function_timeout
   reserved_concurrent_executions = var.lambda_reserved_concurrent_executions
 
-  environment  {
+  environment {
     variables = var.lambda_function_environment_variables
-  }  
+  }
 
   dynamic "tracing_config" {
     for_each = var.lambda_tracing_config == null ? [] : [1]
@@ -411,7 +411,7 @@ resource "aws_iam_role_policy_attachment" "kinesis_fh_role_attachment" {
 data "aws_iam_policy_document" "cloudwatch_to_firehose_trust_assume_policy" {
   statement {
     actions = ["sts:AssumeRole"]
-    effect = "Allow"
+    effect  = "Allow"
     principals {
       type        = "Service"
       identifiers = [for region in local.cloudwatch_log_regions : "logs.${region}.amazonaws.com"]
@@ -422,8 +422,8 @@ data "aws_iam_policy_document" "cloudwatch_to_firehose_trust_assume_policy" {
 resource "aws_iam_role" "cloudwatch_to_firehose_trust" {
   name        = var.cloudwatch_to_firehose_trust_iam_role_name
   description = "Role for CloudWatch Log Group subscriptions"
-  
-  assume_role_policy = "${data.aws_iam_policy_document.cloudwatch_to_firehose_trust_assume_policy.json}"
+
+  assume_role_policy = data.aws_iam_policy_document.cloudwatch_to_firehose_trust_assume_policy.json
 }
 
 data "aws_iam_policy_document" "cloudwatch_to_fh_access_policy" {

variables.tf

@@ -1,13 +1,13 @@
 variable "region" {
-  description = "The region of AWS you want to work in, such as us-west-2 or us-east-1 (deprecated: use cloudwatch_log_regions instead)"
+  description = "DEPRECATED. The region of AWS you want to work in, such as us-west-2 or us-east-1 (deprecated: use `var.cloudwatch_log_regions` instead)"
   type        = string
-  default = null
+  default     = null
 }
 
 variable "cloudwatch_log_regions" {
   description = "List of regions to allow CloudWatch logs to be shipped from. Set in Kinesis Firehose role's trust polucy"
   type        = list(string)
-  default = [ ]
+  default     = []
 }
 
 variable "hec_url" {
@@ -153,8 +153,9 @@ variable "lambda_function_timeout" {
 }
 
 variable "lambda_function_environment_variables" {
-    description = "Environment variables for the lambda function"
-    default = { }
+  description = "Environment variables for the lambda function"
+  default     = {}
+  type        = map(string)
 }
 
 variable "lambda_iam_policy_name" {