Kubetest2 - Cleanup leaked resources from previous clusters #11250
Conversation
If --up is specified then we always delete any previous cluster with the same name since it would conflict otherwise and ensures leaked resources are cleaned up. We don't do this if --terraform is specified because Down() will run `terraform destroy` but each kubetest2 invocation uses a random temp directory for its terraform state, so it wouldn't know of any resources to destroy
k8s-ci-robot
added
cncf-cla: yes
|
/hold this was intended to fix the leaked resources identified in #11249 but each job invocation in one PR will have different cluster names because the cluster name is dependent on the JOB_NAME (which remains constant) and BUILD_ID (which varies every invocation), so there will never be a cluster with the same name that is running at the start of an invocation. One option is to have presubmit jobs replace BUILD_ID with the PR number in their cluster name. |
k8s-ci-robot
added
the
do-not-merge/hold
This should force the same e2e job in one PR's presubmits to use the same cluster name on every job invocation. Mimics the kubetest1 logic here: https://github.com/kubernetes/test-infra/blob/f7e21a3c18f4f4bbc7ee170675ed53e4544a0632/scenarios/kubernetes_e2e.py#L242-L251
k8s-ci-robot
added
size/S
|
/hold cancel |
k8s-ci-robot
removed
the
do-not-merge/hold
k8s-ci-robot
added
the
lgtm
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: hakman The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
k8s-ci-robot
added
the
approved
|
/retest Its possible that all of the regions are "full" of leaked VPCs from other PRs. It seems that aws-janitor has been failing: https://testgrid.k8s.io/sig-testing-maintenance#ci-aws-janitor so we may need to get that working first :/ |
|
/retest |
2 similar comments
|
/retest |
|
/retest |
If --up is specified then we always delete any previous cluster with the same name since it would conflict otherwise and ensures leaked resources are cleaned up.
We don't do this if --terraform is specified because Down() will run
terraform destroybut each kubetest2 invocation uses a random temp directory for its terraform state, so it wouldn't know of any resources to destroy