Implement imagePullSecrets

[jandubois]

Is your feature request related to a problem? Please describe.

It should be possible to pull kubecf from a private registry requiring authentication. Or just a subset of images, like rootfs and buildpacks for a custom stack.

Describe the solution you'd like

We should support a global list of image pull secret names, say kube.image_pull_secrets. This would be a list of names, not of imagePullSecret objects (which are just maps with a single name element). This would allow the user to deploy with:

helm install ... --set kube.image_pull_secrets={foo,bar}

The secrets themselves will have to be created by the user before installing kubecf. We don't want to create the secrets as part of the kubecf helm chart because we don't want to make any assumption about how the secrets need to be specified.

These secrets will be added to the default service account. Since the namespace and service account already exist, it will be necessary to patch it using a helm pre-install hook. See for example helm patch default service account.

This also needs to be done to the service account of the Eirini namespace because the staging image will be pulled using that account.

The new kube.image_pull_secrets property needs to be added to the values.schema.yaml file as an "array of strings".

Describe alternatives you've considered

I originally considered defining imagePullSecrets for each release (and each non-bosh container image), and then setting them using the cf-operator facilities. However, imagePullSecrets are declared on the pod and not the container, so we would need to loop through all the jobs in an instance group to determine the list or required secrets. And for non cf-deployment based releases we only have the job name, but not the release name.

So this all gets pretty complex very quickly, but doesn't seem to provide any benefits over adding all the secrets to the service account, so I decided we should pursue the simpler alternative.

Additional context

cc @jbuns

[jbuns]

Tying it to the Issue I created a few months back with:
#1003

[jandubois]

Implemented by #1482