Does flux delete kustomization delete the resources?

[theFong]

Hello,
When I delete a kustomization, the resources appear to not be delete. Is this expected? In the docs it says the resources should be deleted but when I check my cluster the namespaces still exist. https://fluxcd.io/docs/cmd/flux_delete_kustomization/#examples

[kingdonb]

The behavior as described there depends on having spec.prune: true set in the Flux Kustomization.

It would be better if the docs called this out.

I opened:

• Add detail to delete docs #2624

to address this. Hope this clarifies things!

[craigbryson]

I'm seeing something similar where - if I delete a Kustomization from my repo the kustomization object is deleted but the resources created by it remain - I have prune: true

I see something like this in my kustomize controller log and I was wondering if I'm perhaps seeing the error because I deleted the Kustomization object as well as the manifests that built it at the same time.

{"level":"info","ts":"2022-04-07T12:09:12.299Z","logger":"controller.kustomization","msg":"Reconciliation finished in 298.74335ms, next run in 2m0s","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"rxxxx","namespace":"flux-system","revision":"main/8bae21879b14e32055eb0f2532069dacc9415229"}
{"level":"error","ts":"2022-04-07T12:09:50.640Z","logger":"controller.kustomization","msg":"Reconciliation failed after 101.932265ms, next try in 2m0s","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"xxxx","namespace":"flux-system","revision":"main/85324aba8f447d52ea009223d3c6ee9fec48ab22","error":"kustomization path not found: stat /tmp/xxxx/3048725746/apps/xxxx: no such file or directory"}
{"level":"info","ts":"2022-04-07T12:09:52.972Z","logger":"controller.kustomization","msg":"garbage collection completed: Kustomization/flux-system/xxxx deleted","reconciler group":"kustomize.toolkit.fluxcd.io","reconciler kind":"Kustomization","name":"fluxcd2-sync","namespace":"flux-system"}

It would be good to understand better how pruning is supposed to work, does the controller expect the manifest files to still exist in order to compute what should be deleted, or is some other method used like annotations on resources?

[craigbryson]

Hmm according to kubectl get kustomizations -A the kustomization is indeed deleted.

The flow was basically that we are decommissioning an app and a developer merged a PR which deleted the Kustomization and all of the resource manifests. That led to the log above.

I have noticed whilst investigating that a lot of our kustomizations are using v1beta1 so I wondered if that comes into play?

[kingdonb]

This might be a bug. I think if you can reproduce this behavior, it should be reported as an issue on kustomize-controller.

A Flux Kustomization with prune enabled, that is not suspended, should not be deleted without cleaning up its resources. If you can get this issue to reproduce, please provide detailed steps to repeat it. I don't know the code well enough to say what it does exactly, you can read it and find out if you read Go, or if you have an edge case that breaks the garbage collection, we can definitely follow it up in the issue report.

[stefanprodan]

Flux should be able to delete the resources even if they are removed from Git in the same commit. Please make sure all your Kustomizations are on v1beta2, if the issue persists with the latest version of Flux, then this is a bug.

[craigbryson]

No worries- I have started to see if I can repro this with podinfo on v1beta1. I'll repeat on v1beta2 to see if anything changes and if I can reproduce successfully I'll file an issue :). It will probably be tomorrow

[craigbryson]

Raised an issue at #2627 - but it seems resolved by v1beta2 although there are still some errors reported in logs