feat: declarative sidecar injection

[mumoshu]

Sidecar proxies are important components that powers service meshes. Injecting sidecar proxies to your application is achieved usually by:

1. install a mutating webhook onto your cluster or
2. modifying a pod spec to include a sidecar container or
3. use a mesh-provided CLI app to modify the pod spec to include a sidecar definition in it, like linkerd inject and istioctl kube-inject

In my opinion, the option 1 is easy for on-boarding, but terrible for long-term maintenance. That's because you don't have what's being injected and when. You even need to occasionally detect which pods has sidecars installed, and rolling-update them after the mutating webhook server has been updated with a new sidecar definition.

The option 2 is the safest option as you can managed all the k8s manifests including sidecar specs declaratively. But it's painful to continuously watch and replicate how the upstream tool like istioctl kube-inject produces the sidecar container definitions.

The option 3 has potential to control what's being injected by version-controlling the CLI tool rather than the sidecar container definition itself. That's good.

But how can you inject sidecars with a tool like istioctl kube-inject, if you manage your environment with helmfile?

Shameless plug: Adding helm-x integration to helmfile would allow you to inject sidecars to your apps while keeping all the benefits of using helmfile and helm.

Under the hood, it doesn't just run helm template | istioctl kube-inject -f -. That would break many things helm provides, as it won't create helm release objects that are required in order to helm list, helm status, helm diff, helm get values, helm test, and so on.

Instead it creates a temporary local chart by running helm template, and then updating all the manifests with sidecar injections, and finally running helm upgrade --install against the temp local chart.

With the helmfile integration, one would just add injector to your release in helmfile.yaml:

releases:
- name: myapp
   chart: repo/anychart
   injector: "istioctl kube-inject f=FILE,injectConfigFile=inject-config.yaml"

Now version-control istioctl within your container image and you have full-control of which release gets sidecars injected with which version of istioctl.