Terraform Module: RDS Snapshots Export To S3

Brief

Terraform module that deploys Lambda functions that take care of triggering and monitoring exports of RDS snapshots to S3.

Design

A Lambda function takes care of triggering the RDS Start Export Task for the given database name. The snapshots will be exported to the given S3 bucket.

Another Lambda function is only interested in RDS Export Task events that match a given database name. Whenever a match is detected, a message will be published in the given SNS topic which you can use to trigger other components. E.g. a Lambda function that sends notifications to Slack.

A single CloudWatch Event Rule takes care of listening for RDS Snapshots Events in order to call the aforementioned Lambda functions.

Important considerations

• Please note, that only customer managed keys (CMK) are allowed.
• Either customer_kms_key_arn provided key is used for exported snapshots encryption or new CMK created with create_customer_kms_key enabled
• Since the module (optionally) creates its own KMS CMK, keep that in mind regarding KMS pricing; not only regarding the pricing of a single key, but also things like key rotations/versions and KMS API requests.
• The module requires you to provide the S3 bucket that will be used for storing the exported snapshots. The good thing about this is that you are able to configure the bucket in any way you need. E.g. replication, lifecycle, locking, and so on.
• The module can create an export monitor SNS notification topic, also existing SNS topics are supported via notifications_topic_arn variable.
• Multi-region support via terraform providers.
• If triggering from manual snapshots, the snapshot must be named as rds-<database-name>-<timestamp> with timestamp of format eg '2023-08-09-18-07'.

Requirements

Name	Version
aws	>= 3.19

Providers

Name	Version
aws	>= 3.19

Modules

Name	Source	Version
monitor_export_task_lambda	github.com/terraform-aws-modules/terraform-aws-lambda	v2.23.0
start_export_task_lambda	github.com/terraform-aws-modules/terraform-aws-lambda	v2.23.0

Resources

Name	Type
aws_cloudwatch_event_rule.rdsSnapshotCreation	resource
aws_cloudwatch_event_target.rdsSnapshotCreationTopic	resource
aws_iam_policy.rdsMonitorExportTaskLambda	resource
aws_iam_policy.rdsStartExportTaskLambda	resource
aws_iam_role.rdsSnapshotExportTask	resource
aws_iam_role_policy.rdsSnapshotExportToS3	resource
aws_kms_alias.snapshotExportEncryptionKey	resource
aws_kms_key.snapshotExportEncryptionKey	resource
aws_lambda_permission.snsCanTriggerMonitorExportTask	resource
aws_lambda_permission.snsCanTriggerStartExportTask	resource
aws_sns_topic.exportMonitorNotifications	resource
aws_sns_topic.rdsSnapshotsEvents	resource
aws_sns_topic_policy.default	resource
aws_sns_topic_subscription.lambdaRdsSnapshotToS3Exporter	resource
aws_sns_topic_subscription.lambdaRdsSnapshotToS3Monitor	resource
aws_caller_identity.current	data source
aws_region.current	data source

Inputs

Name	Description	Type	Default	Required
create_customer_kms_key	Create customer managed KMS key which is used for encrypting the exported snapshots on S3. If set to 'false', then 'customer_kms_key_arn' is used.	bool	false	no
create_notifications_topic	Create new SNS notifications topic which will be used for publishing notifications messages.	bool	true	no
customer_kms_key_arn	The ARN of customer managed key used for RDS export encryption. Mandatory if 'create_customer_kms_key' is set to false. arn:aws:kms:::key/	string	null	no
database_names	The names of the databases whose snapshots we want to export to S3. Comma-separated values), ex: 'db-cluster1, db-cluster2'	string	null	yes
log_level	The log level of the Lambda function.	string	"INFO"	no
notifications_topic_arn	The ARN of an SNS Topic which will be used for publishing notifications messages. Required if 'create_notifications_topic' is set to 'false'.	string	null	no
postfix	Postfix that will be used for naming resources. 'resouce-name-'.	string	null	no
prefix	Prefix that will be used for naming resources. 'resouce-name'.	string	null	no
rds_event_ids	RDS (CloudWatch) Event ID that will trigger the calling of RDS Start Export Task API: - Automated snapshots of Aurora RDS: RDS-EVENT-0169 - Manual snapshots of Aurora RDS: RDS-EVENT-0075 - Automated snapshots of non-Aurora RDS: RDS-EVENT-0091 - Manual snapshots of non-Aurora RDS: RDS-EVENT-0042 Automated and/or manual backups of either RDS Aurora and RDS non-Aurora are supported. Ref: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_Events.Messages.html#USER_Events.Messages.snapshot Ref: https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/USER_Events.Messages.html#USER_Events.Messages.cluster-snapshot	string	"RDS-EVENT-0091, RDS-EVENT-0169"	no
snapshots_bucket_name	The name of the bucket where the RDS snapshots will be exported to.	string	null	yes
snapshots_bucket_prefix	The Amazon S3 bucket prefix to use as the file name and path of the exported snapshot. For example, use the prefix exports/2019/	string	null	yes
tags	(Optional) A mapping of tags to assign to the bucket.	map(string)	{}	no

Outputs

Name	Description
monitor_export_task_lambda_function_arn	Start Export Task Monitor Lambda Function ARN
monitor_export_task_lambda_role_arn	Start Export Task Monitor Lambda Role ARN
snapshots_events_export_monitor_sns_topics_arn	RDS Snapshots Export Monitor Events SNS Topics ARN
snapshots_events_sns_topics_arn	RDS Snapshots Events SNS Topics ARN
snapshots_export_encryption_key_arn	Snapshots Export Encryption Key ARN
start_export_task_lambda_function_arn	Start Export Task Lambda Function ARN
start_export_task_lambda_role_arn	Start Export Task Lambda Role ARN