Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lambda-event-sources): failure handling for stream event sources #5929

Merged
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
e5ab49e
feat(lambda-event-sources): add failure handling parameters to lambda…
xerofun Jan 22, 2020
26dcde6
Merge branch 'master' into xerofun/lambda-event-source-mapping-parame…
xerofun Jan 23, 2020
87b3477
Merge branch 'master' into xerofun/lambda-event-source-mapping-parame…
xerofun Jan 23, 2020
5627c7c
Merge branch 'master' into xerofun/lambda-event-source-mapping-parame…
xerofun Jan 24, 2020
c271545
Standardize default documentation on maximumRetryAttempts
xerofun Jan 27, 2020
ebf5404
Refactor maximumRecordAgeInSeconds to be a duration
xerofun Jan 27, 2020
3690df1
Refactor away from using lambda.generated for destination config
xerofun Jan 28, 2020
4697831
Remove unnecessary linter exceptions
xerofun Jan 28, 2020
d8bdf54
Merge branch 'master' into xerofun/lambda-event-source-mapping-parame…
xerofun Jan 28, 2020
95eb54b
Simplified comments, variable names, and conditionals
xerofun Jan 30, 2020
5d387c3
Merge branch 'xerofun/lambda-event-source-mapping-parameters-5236' of…
xerofun Jan 30, 2020
7662d58
Update README with new parameters
xerofun Jan 30, 2020
dbafdcd
Update packages/@aws-cdk/aws-lambda-event-sources/README.md
xerofun Feb 5, 2020
9a51afd
Update packages/@aws-cdk/aws-lambda-event-sources/README.md
xerofun Feb 5, 2020
daff204
Update packages/@aws-cdk/aws-lambda-event-sources/test/test.dynamo.ts
xerofun Feb 5, 2020
e13f488
Update packages/@aws-cdk/aws-lambda-event-sources/test/test.dynamo.ts
xerofun Feb 5, 2020
a4fccec
Update packages/@aws-cdk/aws-lambda-event-sources/test/test.dynamo.ts
xerofun Feb 5, 2020
8223345
Update README with links to lambda event links
xerofun Feb 5, 2020
3cae74c
Use better variable names
xerofun Feb 5, 2020
61918bb
Add integration test and refactor away from IDestination
xerofun Feb 13, 2020
a68f2d5
Merge branch 'master' into xerofun/lambda-event-source-mapping-parame…
xerofun Feb 13, 2020
f3b3e22
Pull up DLQ libraries to remedy circular dependency
xerofun Feb 17, 2020
0f994f8
Merge branch 'master' into xerofun/lambda-event-source-mapping-parame…
xerofun Feb 17, 2020
2c50c32
Merge branch 'master' into xerofun/lambda-event-source-mapping-parame…
xerofun Feb 17, 2020
8d7e516
Clarify documentation on dead letter queue objects
xerofun Feb 17, 2020
7e34a57
Remove package-lock.json
xerofun Mar 2, 2020
7263cc5
Standardize case on new classes
xerofun Mar 2, 2020
17eee38
Clarify documentation
xerofun Mar 2, 2020
22b05c8
Merge pull request #1 from aws/master
xerofun Mar 4, 2020
3a9a9dc
Merge remote-tracking branch 'origin/master' into xerofun/lambda-even…
xerofun Mar 4, 2020
c44be13
Move DLQ implementations to event sources package
xerofun Mar 5, 2020
a4f05fe
Extract test lambda from integration test into external file
xerofun Mar 6, 2020
a8b2fed
fixup readme
Mar 9, 2020
a4e59b2
2 space indent
Mar 9, 2020
01f5ec3
Add unit test for event source mapping validations
xerofun Mar 9, 2020
faca307
Add permissions to send messages to DLQs to lambda
xerofun Mar 9, 2020
068be7a
fix integ test and stack verification steps
Mar 9, 2020
b4f51a1
Update packages/@aws-cdk/aws-lambda/lib/event-source-mapping.ts
xerofun Mar 9, 2020
fb6957b
Clean up tests
xerofun Mar 9, 2020
43cd63b
Add event source mapping to DLQ bind signature
xerofun Mar 9, 2020
303aabb
Fix integration test expected
xerofun Mar 9, 2020
c3f4dfe
added IEventSourceMapping
Mar 10, 2020
2d2e1d7
Refactor to IEventSourceMapping
xerofun Mar 10, 2020
9d0d2f0
Merge branch 'master' into xerofun/lambda-event-source-mapping-parame…
mergify[bot] Mar 11, 2020
e2dc206
Merge branch 'master' into xerofun/lambda-event-source-mapping-parame…
Mar 13, 2020
b6e842f
Merge branch 'master' into xerofun/lambda-event-source-mapping-parame…
Mar 16, 2020
f149ceb
Merge branch 'master' into xerofun/lambda-event-source-mapping-parame…
Mar 16, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 47 additions & 1 deletion packages/@aws-cdk/aws-lambda-event-sources/lib/stream.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import * as lambda from '@aws-cdk/aws-lambda';
import {Duration} from '@aws-cdk/core';
import { Duration, IResolvable } from '@aws-cdk/core';

/**
* The set of properties for event sources that follow the streaming model,
Expand All @@ -21,6 +21,47 @@ export interface StreamEventSourceProps {
*/
readonly batchSize?: number;

/**
* If the function returns an error, split the batch in two and retry.
*
* @default false
*/
readonly bisectBatchOnFunctionError?: boolean;
xerofun marked this conversation as resolved.
Show resolved Hide resolved

/**
* An Amazon SQS queue or Amazon SNS topic destination for discarded records.
*
* @default discarded records are ignored
*/
readonly destinationConfig?: lambda.CfnEventSourceMapping.DestinationConfigProperty | IResolvable | undefined;
nija-at marked this conversation as resolved.
Show resolved Hide resolved

/**
* The maximum age of a record that Lambda sends to a function for processing.
* Valid Range:
* * Minimum value of 60
* * Maximum value of 604800
*
* @default 604800
xerofun marked this conversation as resolved.
Show resolved Hide resolved
xerofun marked this conversation as resolved.
Show resolved Hide resolved
*/
readonly maximumRecordAgeInSeconds?: number;
xerofun marked this conversation as resolved.
Show resolved Hide resolved

/**
* Maximum number of retry attempts
xerofun marked this conversation as resolved.
Show resolved Hide resolved
*
* @default todoNumber
xerofun marked this conversation as resolved.
Show resolved Hide resolved
*/
readonly maximumRetryAttempts?: number;
xerofun marked this conversation as resolved.
Show resolved Hide resolved

/**
* The number of batches to process from each shard concurrently.
* Valid Range:
* * Minimum value of 1
* * Maximum value of 10
*
* @default 1
xerofun marked this conversation as resolved.
Show resolved Hide resolved
*/
readonly parallelizationFactor?: number;

/**
* Where to begin consuming the stream.
*/
Expand Down Expand Up @@ -48,8 +89,13 @@ export abstract class StreamEventSource implements lambda.IEventSource {
return {
...options,
batchSize: this.props.batchSize || 100,
bisectBatchOnFunctionError: this.props.bisectBatchOnFunctionError,
startingPosition: this.props.startingPosition,
maxBatchingWindow: this.props.maxBatchingWindow,
maximumRecordAgeInSeconds: this.props.maximumRecordAgeInSeconds,
maximumRetryAttempts: this.props.maximumRetryAttempts,
parallelizationFactor: this.props.parallelizationFactor,
destinationConfig: this.props.destinationConfig
};
}
}
10 changes: 8 additions & 2 deletions packages/@aws-cdk/aws-lambda-event-sources/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,13 @@
"docs-public-apis:@aws-cdk/aws-lambda-event-sources.KinesisEventSourceProps",
"docs-public-apis:@aws-cdk/aws-lambda-event-sources.S3EventSourceProps",
"props-default-doc:@aws-cdk/aws-lambda-event-sources.S3EventSourceProps.filters",
"docs-public-apis:@aws-cdk/aws-lambda-event-sources.SqsEventSourceProps"
"docs-public-apis:@aws-cdk/aws-lambda-event-sources.SqsEventSourceProps",
"duration-prop-type:@aws-cdk/aws-lambda-event-sources.DynamoEventSourceProps.maximumRecordAgeInSeconds",
"duration-prop-type:@aws-cdk/aws-lambda-event-sources.KinesisEventSourceProps.maximumRecordAgeInSeconds",
"duration-prop-name:@aws-cdk/aws-lambda-event-sources.KinesisEventSourceProps.maximumRecordAgeInSeconds",
"duration-prop-type:@aws-cdk/aws-lambda-event-sources.StreamEventSourceProps.maximumRecordAgeInSeconds",
"duration-prop-name:@aws-cdk/aws-lambda-event-sources.StreamEventSourceProps.maximumRecordAgeInSeconds",
"duration-prop-name:@aws-cdk/aws-lambda-event-sources.DynamoEventSourceProps.maximumRecordAgeInSeconds"
xerofun marked this conversation as resolved.
Show resolved Hide resolved
]
}
}
}
Loading