Automatic pruning of old TaskRuns / PipelineRuns

[jace-ys]

Expected Behavior

The tekton-pipelines-controller should automatically prune TaskRuns / PipelineRuns past a given age. This age is optional and can be configured via a ConfigMap value.

Actual Behavior

TaskRuns and PipelineRuns accumulate over the use of Tekton, even though old resources might no longer be valuable (their logs can't be retrieved past 24 hours if I'm not wrong). It makes little sense to keep them around, and they also cause performance issues in the Dashboard as it tries to load all of them. See the related issue here: tektoncd/dashboard#842

Steps to Reproduce the Problem

1. Generate huge amounts of TaskRuns and PipelineRuns without cleaning anything up
2. Over time, the number of resources grow
3. Dashboard gets slower as it tries to load everything

Additional Info

• Kubernetes version:

  Output of kubectl version:

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-18T02:59:13Z", GoVersion:"go1.14.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.8-gke.15", GitCommit:"9cabee15e0922c3b36724de4866a98f6c2da5e6a", GitTreeState:"clean", BuildDate:"2020-05-01T21:47:04Z", GoVersion:"go1.13.8b4", Compiler:"gc", Platform:"linux/amd64"}

• Tekton Pipeline version:

  Output of tkn version or kubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'

Client version: 0.8.0
Pipeline version: v0.12.0

Even if pagination is implemented in the Dashboard, it might still be a good feature to allow automatic pruning of resources if they serve little purpose.

We currently have our own internal script to automate the pruning of TaskRuns / PipelineRuns older than 3 days, via a cron job. It will be great if we could ditch this hacky solution if this became a feature that we can configure as part of the pipelines controller.

[ghost]

One option is to use a CronJob to clean up old resources. We have some work in flight to implement that as part of our own plumbing: tektoncd/plumbing#442 and some more conversation around this feature idea in the experimental repo: tektoncd/experimental#479

[jace-ys]

One option is to use a CronJob to clean up old resources. We have some work in flight to implement that as part of our own plumbing: tektoncd/plumbing#442 and some more conversation around this feature idea in the experimental repo: tektoncd/experimental#479

Thanks for pointing out those - glad to see we're not the only ones! 😄 And like you said, we are using a CronJob running our script right now to do our own clean up, but seems like a nice feature to have out of the box with Tekton.

[wlynch]

Yup, definitely not alone. We've had similar FRs requesting this, so I'm going to close this issue as a duplicate in favor of #1334. Feel free to add additional comments/context to that issue. Thanks!

/triage duplicate
/close

[tekton-robot]

@wlynch: Closing this issue.

In response to this:

Yup, definitely not alone. We've had similar FRs requesting this, so I'm going to close this issue as a duplicate in favor of #1334. Feel free to add additional comments/context to that issue. Thanks!

/triage duplicate
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[wlynch]

/kind feature