Ability to add sidecar containers to static pod manifests

[dsexton]

Is this a BUG REPORT or FEATURE REQUEST?

FEATURE REQUEST

It would be helpful if kubeadm had the ability add additional containers to the generated static pod manifests. For example, adding the aws-encryption-provider to the apiserver pod.

Example:

apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
apiServer:
  extraArgs:
    cloud-provider: "aws"
  extraContainers:
    - name: aws-encryption-provider
      image: k8s.gcr.io/aws-encryption-provider:v0.1.0
      resources:
        limits:
          memory: "200Mi"
        requests:
          memory: "100Mi"

You could have a fixed one-off just for encryption provider support, however having this be generic could provide a solution to other use cases down the road.

Misc References

• PR for adding aws-encryption-provider to kops
• Using a KMS provider for data encryption

/cc @chuckha @sethpollack

[neolit123]

• the configuration format is already locked for 1.15 (v1beta2) feature wise.
• it's already possible to do this using kubeadm phases:

# write your own CP component manifest files in /etc/kubernetes
kubeadm init --skip-phases=control-plane

• such a cluster with side cars to the CP components might be unsupported in upgrades.

@kubernetes/sig-cluster-lifecycle

[fabriziopandini]

IMO this issue should be merged with discussion on #1379
/release Next

[timothysc]

Closing and deferring to the other related issue.