Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lambda): sns topic dlq #4763

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 39 additions & 7 deletions packages/@aws-cdk/aws-lambda/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,8 +100,13 @@ See the documentation for the __@aws-cdk/aws-lambda-event-sources__ module for m

### Lambda with DLQ

A dead-letter queue can be automatically created for a Lambda function by
setting the `deadLetterQueueEnabled: true` configuration.
See [the AWS documentation](https://docs.aws.amazon.com/lambda/latest/dg/dlq.html)
to learn more about AWS Lambdas and DLQs.

#### SQS

An SQS dead-letter queue can be automatically created for a Lambda function by
using the `DeadLetterQueue.fromSqsQueue()` method.

```ts
import lambda = require('@aws-cdk/aws-lambda');
Expand All @@ -110,7 +115,7 @@ const fn = new lambda.Function(this, 'MyFunction', {
runtime: lambda.Runtime.NODEJS_8_10,
handler: 'index.handler',
code: lambda.Code.fromInline('exports.handler = function(event, ctx, cb) { return cb(null, "hi"); }'),
deadLetterQueueEnabled: true
dlq: lambda.DeadLetterQueue.fromSqsQueue(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that in this case the from prefix doesn't feel right because we allow no parameters and so "from what?". I'd argue that lambda.DeadLetterQueue.sqsQueue() reads better in this case and cal also accomnodate DeadLetterQueue.sqsQueue(x) (@rix0rrr, thoughts?).

Alternatively we can do: fromNewQueue() and fromQueue(x).

});
```

Expand All @@ -120,17 +125,44 @@ It is also possible to provide a dead-letter queue instead of getting a new queu
import lambda = require('@aws-cdk/aws-lambda');
import sqs = require('@aws-cdk/aws-sqs');

const dlq = new sqs.Queue(this, 'DLQ');
const queue = new sqs.Queue(this, 'DLQ');
const fn = new lambda.Function(this, 'MyFunction', {
runtime: lambda.Runtime.NODEJS_8_10,
handler: 'index.handler',
code: lambda.Code.fromInline('exports.handler = function(event, ctx, cb) { return cb(null, "hi"); }'),
deadLetterQueue: dlq
dlq: lambda.DeadLetterQueue.fromSqsQueue(queue),
});
```

See [the AWS documentation](https://docs.aws.amazon.com/lambda/latest/dg/dlq.html)
to learn more about AWS Lambdas and DLQs.
#### SNS

It is also possible create SNS notifications from dead Lambda events:

```ts
import lambda = require('@aws-cdk/aws-lambda');

const fn = new lambda.Function(this, 'MyFunction', {
runtime: lambda.Runtime.NODEJS_8_10,
handler: 'index.handler',
code: lambda.Code.fromInline('exports.handler = function(event, ctx, cb) { return cb(null, "hi"); }'),
dlq: lambda.DeadLetterQueue.fromSnsTopic(),
});
```

Likewise, you can also provide an existing topic instead of creating one for your Lmabda function:

```ts
import lambda = require('@aws-cdk/aws-lambda');
import sns = require('@aws-cdk/aws-sns');

const topic = new sns.Topic(this, 'DeadLetterTopic');
const fn = new lambda.Function(this, 'MyFunction', {
runtime: lambda.Runtime.NODEJS_8_10,
handler: 'index.handler',
code: lambda.Code.fromInline('exports.handler = function(event, ctx, cb) { return cb(null, "hi"); }'),
dlq: lambda.DeadLetterQueue.fromSnsTopic(topic),
});
```

### Lambda with X-Ray Tracing

Expand Down
143 changes: 130 additions & 13 deletions packages/@aws-cdk/aws-lambda/lib/function.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import cloudwatch = require('@aws-cdk/aws-cloudwatch');
import ec2 = require('@aws-cdk/aws-ec2');
import iam = require('@aws-cdk/aws-iam');
import logs = require('@aws-cdk/aws-logs');
import sns = require('@aws-cdk/aws-sns');
import sqs = require('@aws-cdk/aws-sqs');
import { Construct, Duration, Fn, Lazy, Stack, Token } from '@aws-cdk/core';
import { Code, CodeConfig } from './code';
Expand Down Expand Up @@ -33,6 +34,110 @@ export enum Tracing {
DISABLED = "Disabled"
}

/**
* Describes the type of resource to use as a Lambda dead-letter queue
*
* @internal
*/
export enum DeadLetterQueueType {
/**
* SQS DLQ
*/
SQS = 'SQS',
/**
* SNS DLQ
*/
SNS = 'SNS',
}

/**
* Resource to be used as a dead-letter queue
*
* @internal
*/
export interface DeadLetterQueueOptions {
/**
* SQS queue
*
* @default - no queue
*/
readonly queue?: sqs.IQueue;
/**
* SNS topic
*
* @default - no topic
*/
readonly topic?: sns.ITopic;
}

/**
* Dead-letter queue settings
*/
export class DeadLetterQueue {
/**
* Use a SQS queue as dead-letter queue
*
* @param queue The SQS queue to associate.
* If not provided, a new SQS queue will be created with a 14 day retention period
*/
public static fromSqsQueue(queue?: sqs.IQueue): DeadLetterQueue {
return new DeadLetterQueue(DeadLetterQueueType.SQS, { queue });
}

/**
* Use a SNS topic as dead-letter queue
*
* @param topic The SNS topic to associate.
* If not provided, a new SNS topic will be created
*/
public static fromSnsTopic(topic?: sns.ITopic): DeadLetterQueue {
return new DeadLetterQueue(DeadLetterQueueType.SNS, { topic });
}

/**
* Retrieve the ARN identifying the dead-letter queue resource
*
* @returns ARN of the DLQ
*/
public get targetArn(): string {
if (!this.queue && !this.topic) {
throw new Error('never - bind(function) method should have been invoked first');
}

return this.queue ? this.queue.queueArn : this.topic!.topicArn;
}

private queue?: sqs.IQueue;
private topic?: sns.ITopic;
private constructor(public readonly type: DeadLetterQueueType, options: DeadLetterQueueOptions = {}) {
this.queue = options.queue;
this.topic = options.topic;
}

/**
* Creates the appropriate IAM policy required for the Lambda function to send its dead events.
*
* This method also generates the default resource (queue or topic) if it was not provided on construction
*
* @param fn The Lambda function to bind
*/
public bind(fn: Function) {
switch (this.type) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of this, just make bind abstract and implement it in-place for each fromXxx method:

export abstract class DeadLetterQueue {
  public static fromSqsQueue(queue?: sqs.IQueue): DeadLetterQueue {
    return {
      bind: fn => { /* code for SQS queue */ }
    };
  }

  public static fromSnsTopic(topic?: sns.ITopic): DeadLetterQueue {
    return {
      bind: fn => { /* code for SNS */ }
  }

  public abstract bind(fn: lambda.Function): void;
}

case DeadLetterQueueType.SQS:
this.queue = this.queue || new sqs.Queue(fn, 'DeadLetterQueue', { retentionPeriod: Duration.days(14) });
break;
case DeadLetterQueueType.SNS:
this.topic = this.topic || new sns.Topic(fn, 'DeadLetterTopic', {});
break;
}

fn.addToRolePolicy(new iam.PolicyStatement({
actions: [this.queue ? 'sqs:SendMessage' : 'sns:Publish'],
resources: [this.targetArn]
}));
}
}

export interface FunctionProps {
/**
* The source code of your Lambda function. You can point to a file in an
Expand Down Expand Up @@ -164,18 +269,29 @@ export interface FunctionProps {
*/
readonly allowAllOutbound?: boolean;

/**
* Dead-letter queue behavior
*
* @default - no dead-letter queue will be used or created
*/
readonly dlq?: DeadLetterQueue;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a fan of dlq as a property name. The "queue" part isn't even true anymore. How about deadLetter? Same for the class, DeadLetter?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use deadLetterTarget and DeadLetterTarget; consistent with the name Lambda has given it - https://docs.aws.amazon.com/lambda/latest/dg/API_DeadLetterConfig.html


/**
* Enabled DLQ. If `deadLetterQueue` is undefined,
* an SQS queue with default options will be defined for your Function.
*
* @default - false unless `deadLetterQueue` is set, which implies DLQ is enabled.
*
* @deprecated see {@link FunctionProps.dlq} with {@link DeadLetterQueue.fromSqsQueue}
*/
readonly deadLetterQueueEnabled?: boolean;

/**
* The SQS queue to use if DLQ is enabled.
*
* @default - SQS queue with 14 day retention period if `deadLetterQueueEnabled` is `true`
*
* @deprecated see {@link FunctionProps.dlq} with {@link DeadLetterQueue.fromSqsQueue}
*/
readonly deadLetterQueue?: sqs.IQueue;

Expand Down Expand Up @@ -598,25 +714,26 @@ export class Function extends FunctionBase {
}

private buildDeadLetterConfig(props: FunctionProps) {
if (props.deadLetterQueue && props.deadLetterQueueEnabled === false) {
throw Error('deadLetterQueue defined but deadLetterQueueEnabled explicitly set to false');
if (props.dlq && (props.deadLetterQueue || props.deadLetterQueueEnabled)) {
throw Error('dlq cannot be used with deadLetterQueue or deadLetterQueueEnabled. Please only use dlq');
}

if (!props.deadLetterQueue && !props.deadLetterQueueEnabled) {
return undefined;
}
let dlq = props.dlq;
if (props.deadLetterQueue || props.deadLetterQueueEnabled) {
if (props.deadLetterQueueEnabled === false) {
throw Error('deadLetterQueue defined but deadLetterQueueEnabled explicitly set to false');
}

const deadLetterQueue = props.deadLetterQueue || new sqs.Queue(this, 'DeadLetterQueue', {
retentionPeriod: Duration.days(14)
});
dlq = DeadLetterQueue.fromSqsQueue(props.deadLetterQueue);
}

this.addToRolePolicy(new iam.PolicyStatement({
actions: ['sqs:SendMessage'],
resources: [deadLetterQueue.queueArn]
}));
if (!dlq) {
return undefined;
}

dlq.bind(this);
return {
targetArn: deadLetterQueue.queueArn
targetArn: dlq.targetArn,
};
}

Expand Down
2 changes: 2 additions & 0 deletions packages/@aws-cdk/aws-lambda/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@
"@aws-cdk/aws-logs": "1.15.0",
"@aws-cdk/aws-s3": "1.15.0",
"@aws-cdk/aws-s3-assets": "1.15.0",
"@aws-cdk/aws-sns": "1.15.0",
"@aws-cdk/aws-sqs": "1.15.0",
"@aws-cdk/core": "1.15.0",
"@aws-cdk/cx-api": "1.15.0"
Expand All @@ -102,6 +103,7 @@
"@aws-cdk/aws-logs": "1.15.0",
"@aws-cdk/aws-s3": "1.15.0",
"@aws-cdk/aws-s3-assets": "1.15.0",
"@aws-cdk/aws-sns": "1.15.0",
"@aws-cdk/aws-sqs": "1.15.0",
"@aws-cdk/core": "1.15.0",
"@aws-cdk/cx-api": "1.15.0"
Expand Down
Loading