Add ARM64 support for masters

[hakman]

This PR adds build and side-loading support for ARM64 masters:

• build multi-arch images (the AMD64 image is pushed twice, with and without the -amd64 repo suffix.
• add support for side-loading the multi-arch images
• disable the Protokube checks in mock

The Protokube check has to be disabled in mock to allow changing the image names change to multi-arch from:

dns-controller.tar.gz
kops-controller.tar.gz
kube-apiserver-healthcheck.tar.gz
protokube.tar.gz

to:

dns-controller-amd64.tar.gz
dns-controller-arm64.tar.gz
kops-controller-amd64.tar.gz
kops-controller-arm64.tar.gz
kube-apiserver-healthcheck-amd64.tar.gz
kube-apiserver-healthcheck-arm64.tar.gz
protokube-amd64.tar.gz
protokube-arm64.tar.gz

TODO (in future PRs):

• re-enable the Protokube checks in mock after the next release
• decide on how to handle multi-arch manifests for the kops images
• remove the -plain bazel image targets
• build etcd-manager multi-arch images Support for ARM64 kopeio/etcd-manager#334

[johngmyers]

This supports a mixed-arch control plane, which seems unlikely. Perhaps we should have a cluster spec field specifying the arch of the control plane, allowing apply_cluster to save a bunch of asset management.

[hakman]

/test pull-kops-e2e-cni-cilium

[hakman]

This supports a mixed-arch control plane, which seems unlikely. Perhaps we should have a cluster spec field specifying the arch of the control plane, allowing apply_cluster to save a bunch of asset management.

@johngmyers I thought about that also, but I decided on not complicating the spec even more. Current implementation allows the node to autodetect whatever OS arch the node has and keeps the logic consistent in the asset management code.

I am in four of extending this to a cluster spec option though:
#9535

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hakman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [hakman]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hakman]

/test pull-kops-e2e-cni-cilium

[hakman]

/hold for feedback from reviewers

[olemarkus]

/lgtm

This gives me a valid cluster bar the autoscaler pods that comes in #9744

[johngmyers]

/lgtm

[hakman]

/hold cancel

[johngmyers]

/lgtm

[hakman]

/test all

[hakman]

/test pull-kops-e2e-cni-kuberouter