Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(logs): delete associated log group when stack is deleted #21113

Merged
merged 34 commits into from
Aug 8, 2022
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
720b9a6
add delete policy to manage logs created by logretention
YichenQian09 Jul 12, 2022
5a7dbe5
move the enum variable LogDeletionPolicy to log-retention.ts, solve t…
YichenQian09 Jul 12, 2022
fcf6988
Merge branch 'main' into delete-logs
pahud Jul 18, 2022
9d93294
Merge branch 'main' into delete-logs
YichenQian09 Jul 19, 2022
21bad7e
add a integration test and resolve some conversations
YichenQian09 Jul 21, 2022
b84c87e
Merge branch 'main' into delete-logs
YichenQian09 Jul 21, 2022
ec31cc8
run integration test
YichenQian09 Jul 22, 2022
df3d211
add IntegTest
YichenQian09 Jul 22, 2022
1a81cf7
Merge branch 'delete-logs' of https://github.com/YichenQian09/aws-cdk…
YichenQian09 Jul 22, 2022
8e23c53
integ.log-retention passed
YichenQian09 Jul 23, 2022
ee29525
Merge branch 'main' into delete-logs
YichenQian09 Jul 23, 2022
e38d647
update readme.md
YichenQian09 Jul 23, 2022
24c478a
Merge branch 'delete-logs' of https://github.com/YichenQian09/aws-cdk…
YichenQian09 Jul 23, 2022
3421be2
updated the unit test of log-retention
YichenQian09 Jul 23, 2022
a9dab41
add new unit test for delete log group in log-retention-provider.test
YichenQian09 Jul 27, 2022
8b5bcdc
Merge branch 'main' into delete-logs
YichenQian09 Jul 27, 2022
23a09e6
fix the error of template
YichenQian09 Jul 27, 2022
9833f4d
Merge branch 'delete-logs' of https://github.com/YichenQian09/aws-cdk…
YichenQian09 Jul 27, 2022
fe481ec
scope the role down to only allow delete specific log group
YichenQian09 Jul 30, 2022
6247400
Merge branch 'main' into delete-logs
YichenQian09 Jul 30, 2022
59c60b0
Merge branch 'delete-logs' of https://github.com/YichenQian09/aws-cdk…
YichenQian09 Jul 30, 2022
263ad44
use the original enum removalpolicy
YichenQian09 Aug 4, 2022
af3e626
Merge branch 'main' into delete-logs
YichenQian09 Aug 5, 2022
96250f5
Merge branch 'main' into delete-logs
mergify[bot] Aug 5, 2022
2b9bb65
Merge branch 'delete-logs' of https://github.com/YichenQian09/aws-cdk…
YichenQian09 Aug 5, 2022
e2e35bf
changed the iam
YichenQian09 Aug 5, 2022
df84129
add additional unit tests
YichenQian09 Aug 5, 2022
52d5b8d
update snapshot
YichenQian09 Aug 5, 2022
6682f3c
only add removalPolicy if it is set to destroy; update unit tests
YichenQian09 Aug 5, 2022
989433a
update readme.md
YichenQian09 Aug 5, 2022
89a9286
udpate logstream arn and readme
YichenQian09 Aug 8, 2022
209fb69
pass the value directly and format
YichenQian09 Aug 8, 2022
9343ae4
remove empty line
YichenQian09 Aug 8, 2022
aa30341
Merge branch 'main' into delete-logs
mergify[bot] Aug 8, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions packages/@aws-cdk/aws-logs/lib/log-retention-provider/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,35 @@ async function createLogGroupSafe(logGroupName: string, region?: string, options
} while (true); // exit happens on retry count check
}

//delete a log group
async function deleteLogGroup(logGroupName: string, region?: string, options?: SdkRetryOptions) {
YichenQian09 marked this conversation as resolved.
Show resolved Hide resolved
let retryCount = options?.maxRetries == undefined ? 10 : options.maxRetries;
const delay = options?.retryOptions?.base == undefined ? 10 : options.retryOptions.base;
do {
try {
const cloudwatchlogs = new AWS.CloudWatchLogs({ apiVersion: '2014-03-28', region, ...options });
await cloudwatchlogs.deleteLogGroup({ logGroupName }).promise();
return;
} catch (error) {
if (error.code === 'ResourceNotFoundException') {
YichenQian09 marked this conversation as resolved.
Show resolved Hide resolved
// The log group doesn't exist
return;
}
if (error.code === 'OperationAbortedException') {
YichenQian09 marked this conversation as resolved.
Show resolved Hide resolved
if (retryCount > 0) {
retryCount--;
await new Promise(resolve => setTimeout(resolve, delay));
continue;
} else {
// The log group is still being deleted by another execution but we are out of retries
YichenQian09 marked this conversation as resolved.
Show resolved Hide resolved
throw new Error('Out of attempts to delete a logGroup');
}
}
throw error;
}
} while (true); // exit happens on retry count check
}

/**
* Puts or deletes a retention policy on a log group.
*
Expand Down Expand Up @@ -105,6 +134,9 @@ export async function handler(event: AWSLambda.CloudFormationCustomResourceEvent
// Parse to AWS SDK retry options
const retryOptions = parseRetryOptions(event.ResourceProperties.SdkRetry);

//The deletion policy of log after the stack is deleted
const logDeletionPolicy = event.ResourceProperties.LogDeletionPolicy;
YichenQian09 marked this conversation as resolved.
Show resolved Hide resolved

if (event.RequestType === 'Create' || event.RequestType === 'Update') {
// Act on the target log group
await createLogGroupSafe(logGroupName, logGroupRegion, retryOptions);
Expand All @@ -124,6 +156,15 @@ export async function handler(event: AWSLambda.CloudFormationCustomResourceEvent
}
}

//if the requestType is delete, then delete the log group created by the logRetention
if (event.RequestType === 'Delete') {
//if the removal policy is delete, then delete the log group
if (logDeletionPolicy === 'destroy') {
await deleteLogGroup(logGroupName, logGroupRegion, retryOptions);
}
//else retain the log group
}

await respond('SUCCESS', 'OK', logGroupName);
} catch (e) {
console.log(e);
Expand Down
24 changes: 23 additions & 1 deletion packages/@aws-cdk/aws-logs/lib/log-retention.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,27 @@ export interface LogRetentionProps {
* @default - AWS SDK default retry options
*/
readonly logRetentionRetryOptions?: LogRetentionRetryOptions;

/**
* The deletionPolicy the LogRetention uses on log
* @default LogDeletionPolicy.RETAINLOG
*/
readonly logDeletionPolicy?: LogDeletionPolicy;
YichenQian09 marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* LogDeletionPolicy for the log group when the stack is deleted
*/
export enum LogDeletionPolicy {
YichenQian09 marked this conversation as resolved.
Show resolved Hide resolved
/**
* This is the default log deletion policy. It means that when the resource is
* removed from the app, the log will retain.
*/
RETAIN = 'retain',
/**
* This uses the 'destroyLog' DeletionPolicy, which will destroy the log when the stack is deleted.
*/
DESTROY = 'destroy',
}

/**
Expand Down Expand Up @@ -93,6 +114,7 @@ export class LogRetention extends Construct {
base: retryOptions.base?.toMilliseconds(),
} : undefined,
RetentionInDays: props.retention === RetentionDays.INFINITE ? undefined : props.retention,
LogDeletionPolicy: props.logDeletionPolicy ? props.logDeletionPolicy : LogDeletionPolicy.RETAIN,
},
});

Expand Down Expand Up @@ -145,7 +167,7 @@ class LogRetentionFunction extends Construct implements cdk.ITaggable {
});
// Duplicate statements will be deduplicated by `PolicyDocument`
role.addToPrincipalPolicy(new iam.PolicyStatement({
actions: ['logs:PutRetentionPolicy', 'logs:DeleteRetentionPolicy'],
actions: ['logs:PutRetentionPolicy', 'logs:DeleteRetentionPolicy', 'logs:DeleteLogGroup'],
YichenQian09 marked this conversation as resolved.
Show resolved Hide resolved
// We need '*' here because we will also put a retention policy on
// the log group of the provider function. Referencing its name
// creates a CF circular dependency.
Expand Down
20 changes: 20 additions & 0 deletions packages/@aws-cdk/aws-logs/test/integ.log-retention.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import { App, Stack, StackProps } from '@aws-cdk/core';
//import { IntegTest } from '@aws-cdk/integ-tests';
import { LogRetention, RetentionDays, LogDeletionPolicy } from '../lib';

class LogRetentionIntegStack extends Stack {
constructor(scope: App, id: string, props?: StackProps) {
super(scope, id, props);

new LogRetention(this, 'MyLambda', {
logGroupName: 'logRetentionLogGroup',
retention: RetentionDays.ONE_DAY,
logDeletionPolicy: LogDeletionPolicy.DESTROY,
});
}
}

const app = new App();
new LogRetentionIntegStack(app, 'aws-cdk-log-retention-integ');
app.synth();
//new IntegTest(app, 'LogRetentionInteg', { testCases: [stack] });
YichenQian09 marked this conversation as resolved.
Show resolved Hide resolved