Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds pipeline for aws-nuke #399

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions tests/pipelines/cleanup/aws/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Cleanup old aws resources

Here is how users can clean up old aws resources periodically.

The general method is to use a CronJob to trigger a Task that deletes old aws resources that are not protected by aws-nuke config.

## Prerequisites

* A Kubernetes cluster with Tekton Pipelines installed
* Several old aws resources you wish to delete

## Scheduling the cleanup job

You'll need to install all the files in this directory to run the cleanup task.


* [cleanup-template.yaml](cleanup-template.yaml): this creates the TriggerTemplate that spawns the TaskRun that does the deleting. It uses the `aws-nuke` CLI to do the deleting.

* [binding.yaml](binding.yaml): this creates the TriggerBinding that is used to pass parameters to the TaskRun. There are two parameters that are passed by this.
- `aws-nuke-s3-config-path`: this holds the aws-nuke config s3 path. The config holds the resources that needs to be retained by the sweeper job. For instructions on building a aws-nuke config, refer to this https://github.com/rebuy-de/aws-nuke
- `aws-account-alias`: aws-nuke requires account alias for confirmation before deleting. Here is how the account alias can be setup. https://docs.aws.amazon.com/accounts/latest/reference/manage-acct-alias.html

* [eventlistener.yaml](eventlistener.yaml): this creates the sink that receives the incoming event that triggers the creation of the cleanup job.

* [cronjob.yaml](cronjob.yaml): this is used to run the cleanup job on a schedule. The schedule for the job running can be set in the `.spec.schedule` field using [crontab format](https://crontab.guru/)
12 changes: 12 additions & 0 deletions tests/pipelines/cleanup/aws/binding.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerBinding
metadata:
name: aws-account-cleanup
namespace: tekton-pipelines
spec:
params:
- name: aws-nuke-s3-config-path
value: <Replace this with s3 path of config>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's move this to internal ?
Because Flux will automatically apply this file and when it can't file this value, the job will start failing.

- name: aws-account-alias
value: <Replace this with account alias>
44 changes: 44 additions & 0 deletions tests/pipelines/cleanup/aws/cleanup-template.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerTemplate
metadata:
name: aws-account-cleanup
namespace: tekton-pipelines
spec:
params:
- name: aws-nuke-s3-config-path
description: S3 path for the aws nuke's config file.
- name: aws-account-alias
description: aws account alias for the account to be sweeped.
resourcetemplates:
- apiVersion: tekton.dev/v1beta1
kind: TaskRun
metadata:
name: aws-account-cleanup-runs-$(uid)
spec:
serviceAccountName: tekton-pipelines-executor
taskSpec:
description: |
Sweeps down an aws account for leftover resources, based on the config provided
workspaces:
- name: config
params:
- name: aws-nuke-s3-config-path
description: S3 path for the aws nuke's config file. i.e s3://dev-eks-rnshis/aws-nuke.yaml
- name: aws-account-alias
description: aws account alias for the account to be sweeped.
steps:
- name: download-config
image: amazon/aws-cli
script: |
aws s3 cp $(params.aws-nuke-s3-config-path) $(workspaces.config.path)/config.yaml
- name: sweep-aws-account
image: quay.io/rebuy/aws-nuke:v2.22.1
script: |
# TODO: Add --no-dry-run to start deleting the resources
echo "$(params.aws-account-alias)" | aws-nuke -c $(workspaces.config.path)/config.yaml
params:
- name: aws-nuke-s3-config-path
value: $(tt.params.aws-nuke-s3-config-path)
- name: aws-account-alias
value: $(tt.params.aws-account-alias)
17 changes: 17 additions & 0 deletions tests/pipelines/cleanup/aws/cronjob.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: aws-account-cleanup
namespace: tekton-pipelines
spec:
schedule: "0 0 */30 * *" # Triggers in every 30 days
jobTemplate:
spec:
template:
spec:
containers:
- name: curl
image: curlimages/curl
args: ["curl", "-X", "POST", "--data", "{}", "el-aws-account-cleanup.tekton-pipelines.svc.cluster.local:8080"]
restartPolicy: Never
14 changes: 14 additions & 0 deletions tests/pipelines/cleanup/aws/eventlistener.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
apiVersion: triggers.tekton.dev/v1alpha1
kind: EventListener
metadata:
name: aws-account-cleanup
namespace: tekton-pipelines
spec:
serviceAccountName: tekton-triggers
triggers:
- name: cron
bindings:
- ref: aws-account-cleanup
template:
ref: aws-account-cleanup