VPA - Implementing automatic detection of injected sidecars

[krzysied]

There will be new feature added to VPA dedicated to detecting injected sidecars and omitting them during making pod eviction decision.
This feature assumes that containers handled by Admission Controller will be listed in additional pod annotation. Containers not listed in this annotation will not be taken into account by the Updater.

[krzysied]

/assign
/cc @bskiba
/cc @kgolab

[krzysied]

Implementation plan:

• Adding annotation module. (Adding vpa blessing annotation #2727)
• Adding annotation creation to Admission Controller. (Vpa - Observed containers admission controller annotation #2731)
• Adding omitting not listed containers by Updater. (VPA - Vpa observed containers annotation handling in updater #2736)

Tests:

• Adding e2e test (VPA - Injected sidecar e2e test #2795)

[bskiba]

Please add an e2e test as a way of verifying the feature works.

[krzysied]

I agree that an e2e test is something nice to have.
Assuming that we need a custom mutating webhook to test this change, I need to do some research about whether to implement a testing webbook or to use an available opensource one...
I will update the issue when I come up with the solution.

[krzysied]

@aleksandra-malinowska suggest brilliant idea that webhooks should already be tested by k8s e2e tests.
https://github.com/kubernetes/kubernetes/blob/master/test/e2e/apimachinery/webhook.go
It looks like there is a pod mutation webhook test. I will try to re-use some of its components/code.

[krzysied]

The Agnhost webhook (used by k8s e2e tests) adds initContainer instead of standard container. -_-'
I will try add sidecar injection functionality to Agnhost. kubernetes/kubernetes#87383

[krzysied]

The e2e test pr was merged. I will wait for two green runs of https://testgrid.k8s.io/sig-autoscaling-vpa#autoscaling-vpa-actuation and then I will close the issue.

[krzysied]

The test is passing. Nothing more to do here.
I'm closing the issue.