vpa: add multiarch builds

[povilasv]

Fixes: #3074

Had to change a test function's parameter from int to int64 as it was overflowing in 32bit version of arm -> https://github.com/kubernetes/autoscaler/compare/master...povilasv:multiarch?expand=1#diff-246f689f90fe5dd9dec48e0c8af223bcR126

Images

Example images:
https://quay.io/repository/povilasv/vpa-admission-controller?tab=tags
https://quay.io/repository/povilasv/vpa-updater?tab=tags
https://quay.io/repository/povilasv/vpa-recommender?tab=tags

For example:
https://quay.io/repository/povilasv/vpa-recommender
is made from:
https://quay.io/repository/povilasv/vpa-recommender-amd64
https://quay.io/repository/povilasv/vpa-recommender-arm64
https://quay.io/repository/povilasv/vpa-recommender-arm
https://quay.io/repository/povilasv/vpa-recommender-ppc64le
https://quay.io/repository/povilasv/vpa-recommender-s390x

Tests

I have tested on arm64 & arm 32bit and it works great!

Example arm64:

kubectl get po -o wide
vpa-admission-controller-c4797dfd5-bzs8f   1/1     Running   0          11h    10.200.1.142    rock64-1         <none>           <none>
vpa-recommender-796d6648b4-qgsmv           1/1     Running   0          61s    10.200.0.236    rock64-0         <none>           <none>
vpa-updater-768587c8dd-jc8z8               1/1     Running   0          107m   10.200.1.146    rock64-1         <none>           <none>

k get node rock64-1 --show-labels
NAME       STATUS   ROLES    AGE   VERSION           LABELS
rock64-1   Ready    <none>   50d   v1.19.0-alpha.0   beta.kubernetes.io/arch=arm64,beta.kubernetes.io/os=linux,kubernetes.io/arch=arm64,kubernetes.io/hostname=rock64-1,kubernetes.io/os=linux

k describe vpa ..

  Recommendation:
    Container Recommendations:
      Container Name:  admission-controller
      Lower Bound:
        Cpu:     25m
        Memory:  262144k
      Target:
        Cpu:     25m
        Memory:  262144k
      Uncapped Target:
        Cpu:     25m
        Memory:  262144k
      Upper Bound:
        Cpu:     1
        Memory:  500Mi

Makefile

Makefile has been tested with:

make release REGISTRY=quay.io/povilasv

Here is the output:

Logs

``` ➜ recommender git:(multiarch) ✗ make release REGISTRY=quay.io/povilasv rm -f recommender-amd64 rm -f recommender-arm rm -f recommender-arm64 rm -f recommender-ppc64le rm -f recommender-s390x docker build -t vpa-autoscaling-builder ../../builder Sending build context to Docker daemon 3.584kB Step 1/10 : FROM golang:1.12.10 ---> f945ea07f224 Step 2/10 : LABEL maintainer="Beata Skiba " ---> Using cache ---> 12b153e9d688 Step 3/10 : ENV GOPATH /gopath/ ---> Using cache ---> 3dc1e435d011 Step 4/10 : ENV PATH $GOPATH/bin:$PATH ---> Using cache ---> fe89a2194054 Step 5/10 : ARG GOARCH ---> Using cache ---> 8c45cdfe09d3 Step 6/10 : ARG LDFLAGS ---> Using cache ---> 652778f4f536 Step 7/10 : RUN go version ---> Using cache ---> afcb9efa90e5 Step 8/10 : RUN go get github.com/tools/godep ---> Using cache ---> 5b3e2ebf08c4 Step 9/10 : RUN godep version ---> Using cache ---> f59e97b8545b Step 10/10 : CMD ["/bin/bash"] ---> Using cache ---> 977562161c9f Successfully built 977562161c9f Successfully tagged vpa-autoscaling-builder:latest docker run -v `pwd`/../..:/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler vpa-autoscaling-builder:latest bash -c 'cd /gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler && make build-binary-with-vendor-amd64 -C pkg/recommender' make: Entering directory '/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler/pkg/recommender' CGO_ENABLED=0 LD_FLAGS=-s GO111MODULE=on GOARCH=amd64 GOOS=linux go build -mod vendor -o recommender-amd64 make: Leaving directory '/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler/pkg/recommender' docker run -v `pwd`/../..:/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler vpa-autoscaling-builder:latest bash -c 'cd /gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler && make build-binary-with-vendor-arm -C pkg/recommender' make: Entering directory '/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler/pkg/recommender' CGO_ENABLED=0 LD_FLAGS=-s GO111MODULE=on GOARCH=arm GOOS=linux go build -mod vendor -o recommender-arm make: Leaving directory '/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler/pkg/recommender' docker run -v `pwd`/../..:/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler vpa-autoscaling-builder:latest bash -c 'cd /gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler && make build-binary-with-vendor-arm64 -C pkg/recommender' make: Entering directory '/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler/pkg/recommender' CGO_ENABLED=0 LD_FLAGS=-s GO111MODULE=on GOARCH=arm64 GOOS=linux go build -mod vendor -o recommender-arm64 make: Leaving directory '/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler/pkg/recommender' docker run -v `pwd`/../..:/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler vpa-autoscaling-builder:latest bash -c 'cd /gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler && make build-binary-with-vendor-ppc64le -C pkg/recommender' make: Entering directory '/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler/pkg/recommender' CGO_ENABLED=0 LD_FLAGS=-s GO111MODULE=on GOARCH=ppc64le GOOS=linux go build -mod vendor -o recommender-ppc64le make: Leaving directory '/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler/pkg/recommender' docker run -v `pwd`/../..:/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler vpa-autoscaling-builder:latest bash -c 'cd /gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler && make build-binary-with-vendor-s390x -C pkg/recommender' make: Entering directory '/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler/pkg/recommender' CGO_ENABLED=0 LD_FLAGS=-s GO111MODULE=on GOARCH=s390x GOOS=linux go build -mod vendor -o recommender-s390x make: Leaving directory '/gopath/src/k8s.io/autoscaler/vertical-pod-autoscaler/pkg/recommender' docker build --pull -t quay.io/povilasv/vpa-recommender-amd64:dev --build-arg ARCH=amd64 . Sending build context to Docker daemon 263.5MB Step 1/6 : FROM gcr.io/distroless/static:latest latest: Pulling from distroless/static Digest: sha256:c6d5981545ce1406d33e61434c61e9452dad93ecd8397c41e89036ef977a88f4 Status: Image is up to date for gcr.io/distroless/static:latest ---> e4d2a899b1bf Step 2/6 : MAINTAINER Krzysztof Grygiel "[email protected]" ---> Using cache ---> cbb79823569d Step 3/6 : ARG ARCH ---> Running in 9b78e72cc0bc Removing intermediate container 9b78e72cc0bc ---> d3c879051ce8 Step 4/6 : COPY recommender-$ARCH /recommender ---> c89cdf1baca1 Step 5/6 : ENTRYPOINT ["/recommender"] ---> Running in 4fcab8a654e3 Removing intermediate container 4fcab8a654e3 ---> b931cd559c27 Step 6/6 : CMD ["--v=4", "--stderrthreshold=info", "--prometheus-address=http://prometheus.monitoring.svc"] ---> Running in 51b68f95cc42 Removing intermediate container 51b68f95cc42 ---> 5693774455f5 Successfully built 5693774455f5 Successfully tagged quay.io/povilasv/vpa-recommender-amd64:dev docker build --pull -t quay.io/povilasv/vpa-recommender-arm:dev --build-arg ARCH=arm . Sending build context to Docker daemon 263.5MB Step 1/6 : FROM gcr.io/distroless/static:latest latest: Pulling from distroless/static Digest: sha256:c6d5981545ce1406d33e61434c61e9452dad93ecd8397c41e89036ef977a88f4 Status: Image is up to date for gcr.io/distroless/static:latest ---> e4d2a899b1bf Step 2/6 : MAINTAINER Krzysztof Grygiel "[email protected]" ---> Using cache ---> cbb79823569d Step 3/6 : ARG ARCH ---> Using cache ---> d3c879051ce8 Step 4/6 : COPY recommender-$ARCH /recommender ---> 37d5b0d5eea6 Step 5/6 : ENTRYPOINT ["/recommender"] ---> Running in 6def8cbd5a8e Removing intermediate container 6def8cbd5a8e ---> 9dae5f89386c Step 6/6 : CMD ["--v=4", "--stderrthreshold=info", "--prometheus-address=http://prometheus.monitoring.svc"] ---> Running in 35de396de210 Removing intermediate container 35de396de210 ---> b291cfeb9ace Successfully built b291cfeb9ace Successfully tagged quay.io/povilasv/vpa-recommender-arm:dev docker build --pull -t quay.io/povilasv/vpa-recommender-arm64:dev --build-arg ARCH=arm64 . Sending build context to Docker daemon 263.5MB Step 1/6 : FROM gcr.io/distroless/static:latest latest: Pulling from distroless/static Digest: sha256:c6d5981545ce1406d33e61434c61e9452dad93ecd8397c41e89036ef977a88f4 Status: Image is up to date for gcr.io/distroless/static:latest ---> e4d2a899b1bf Step 2/6 : MAINTAINER Krzysztof Grygiel "[email protected]" ---> Using cache ---> cbb79823569d Step 3/6 : ARG ARCH ---> Using cache ---> d3c879051ce8 Step 4/6 : COPY recommender-$ARCH /recommender ---> c037b1f463e2 Step 5/6 : ENTRYPOINT ["/recommender"] ---> Running in aedc557d1c46 Removing intermediate container aedc557d1c46 ---> 9124c3bd0e70 Step 6/6 : CMD ["--v=4", "--stderrthreshold=info", "--prometheus-address=http://prometheus.monitoring.svc"] ---> Running in 421d0926f280 Removing intermediate container 421d0926f280 ---> 04a730546423 Successfully built 04a730546423 Successfully tagged quay.io/povilasv/vpa-recommender-arm64:dev docker build --pull -t quay.io/povilasv/vpa-recommender-ppc64le:dev --build-arg ARCH=ppc64le . Sending build context to Docker daemon 263.5MB Step 1/6 : FROM gcr.io/distroless/static:latest latest: Pulling from distroless/static Digest: sha256:c6d5981545ce1406d33e61434c61e9452dad93ecd8397c41e89036ef977a88f4 Status: Image is up to date for gcr.io/distroless/static:latest ---> e4d2a899b1bf Step 2/6 : MAINTAINER Krzysztof Grygiel "[email protected]" ---> Using cache ---> cbb79823569d Step 3/6 : ARG ARCH ---> Using cache ---> d3c879051ce8 Step 4/6 : COPY recommender-$ARCH /recommender ---> cc9dccce9098 Step 5/6 : ENTRYPOINT ["/recommender"] ---> Running in 5d355046d5e8 Removing intermediate container 5d355046d5e8 ---> b138f4836ec6 Step 6/6 : CMD ["--v=4", "--stderrthreshold=info", "--prometheus-address=http://prometheus.monitoring.svc"] ---> Running in 1eff3f23026d Removing intermediate container 1eff3f23026d ---> 64a9ae8ee289 Successfully built 64a9ae8ee289 Successfully tagged quay.io/povilasv/vpa-recommender-ppc64le:dev docker build --pull -t quay.io/povilasv/vpa-recommender-s390x:dev --build-arg ARCH=s390x . Sending build context to Docker daemon 263.5MB Step 1/6 : FROM gcr.io/distroless/static:latest latest: Pulling from distroless/static Digest: sha256:c6d5981545ce1406d33e61434c61e9452dad93ecd8397c41e89036ef977a88f4 Status: Image is up to date for gcr.io/distroless/static:latest ---> e4d2a899b1bf Step 2/6 : MAINTAINER Krzysztof Grygiel "[email protected]" ---> Using cache ---> cbb79823569d Step 3/6 : ARG ARCH ---> Using cache ---> d3c879051ce8 Step 4/6 : COPY recommender-$ARCH /recommender ---> f516bd9a70a9 Step 5/6 : ENTRYPOINT ["/recommender"] ---> Running in 616be715163d Removing intermediate container 616be715163d ---> 02cb15642189 Step 6/6 : CMD ["--v=4", "--stderrthreshold=info", "--prometheus-address=http://prometheus.monitoring.svc"] ---> Running in acbef0d6a3b6 Removing intermediate container acbef0d6a3b6 ---> a662b4d4caec Successfully built a662b4d4caec Successfully tagged quay.io/povilasv/vpa-recommender-s390x:dev docker push quay.io/povilasv/vpa-recommender-amd64:dev The push refers to repository [quay.io/povilasv/vpa-recommender-amd64] 6ce628544964: Pushed 0d1435bd79e4: Layer already exists dev: digest: sha256:8149b74fd72bda67e5bee1577d7ad5552e2853f8dc9d416f78eb854024c351d2 size: 739 docker push quay.io/povilasv/vpa-recommender-arm:dev The push refers to repository [quay.io/povilasv/vpa-recommender-arm] 3057986d58f9: Pushed 0d1435bd79e4: Layer already exists dev: digest: sha256:d1150f11656c009a767aa39240b21ca1a159bd74396c22c95e6de7673bed9319 size: 739 docker push quay.io/povilasv/vpa-recommender-arm64:dev The push refers to repository [quay.io/povilasv/vpa-recommender-arm64] 7317c3441eb8: Pushed 0d1435bd79e4: Layer already exists dev: digest: sha256:e63cc040840138d85ec6cb1fa42b54814a813c3a93017594b5c095d128188a38 size: 739 docker push quay.io/povilasv/vpa-recommender-ppc64le:dev The push refers to repository [quay.io/povilasv/vpa-recommender-ppc64le] 756e22937a1c: Pushed 0d1435bd79e4: Layer already exists dev: digest: sha256:b29cd6e78d5bee93c84e80f8373a8d45ca8b7aea2af8e57968512dd8e3e6b5af size: 739 docker push quay.io/povilasv/vpa-recommender-s390x:dev The push refers to repository [quay.io/povilasv/vpa-recommender-s390x] 256dbd75b7dd: Pushed 0d1435bd79e4: Layer already exists dev: digest: sha256:2291252e404ac7493b51326d0d9f2b61afc16f882b3829d69469bb9e275d8880 size: 739 docker manifest create --amend quay.io/povilasv/vpa-recommender:dev quay.io/povilasv/vpa-recommender-amd64:dev quay.io/povilasv/vpa-recommender-arm:dev quay.io/povilasv/vpa-recommender-arm64:dev quay.io/povilasv/vpa-recommender-ppc64le:dev quay.io/povilasv/vpa-recommender-s390x:dev Created manifest list quay.io/povilasv/vpa-recommender:dev docker manifest push --purge quay.io/povilasv/vpa-recommender:dev Pushed ref quay.io/povilasv/vpa-recommender@sha256:8149b74fd72bda67e5bee1577d7ad5552e2853f8dc9d416f78eb854024c351d2 with digest: sha256:8149b74fd72bda67e5bee1577d7ad5552e2853f8dc9d416f78eb854024c351d2 Pushed ref quay.io/povilasv/vpa-recommender@sha256:d1150f11656c009a767aa39240b21ca1a159bd74396c22c95e6de7673bed9319 with digest: sha256:d1150f11656c009a767aa39240b21ca1a159bd74396c22c95e6de7673bed9319 Pushed ref quay.io/povilasv/vpa-recommender@sha256:e63cc040840138d85ec6cb1fa42b54814a813c3a93017594b5c095d128188a38 with digest: sha256:e63cc040840138d85ec6cb1fa42b54814a813c3a93017594b5c095d128188a38 Pushed ref quay.io/povilasv/vpa-recommender@sha256:b29cd6e78d5bee93c84e80f8373a8d45ca8b7aea2af8e57968512dd8e3e6b5af with digest: sha256:b29cd6e78d5bee93c84e80f8373a8d45ca8b7aea2af8e57968512dd8e3e6b5af Pushed ref quay.io/povilasv/vpa-recommender@sha256:2291252e404ac7493b51326d0d9f2b61afc16f882b3829d69469bb9e275d8880 with digest: sha256:2291252e404ac7493b51326d0d9f2b61afc16f882b3829d69469bb9e275d8880 sha256:096de8ea08e087eb7884f2c41801e6561ea1b2637e3b7658b793965c752628e2 Full in-docker release vpa-recommender:dev completed ```

[povilasv]

FYI hamster demo doesn't work as it's not a multiarch image, maybe we can switch from k8s.gcr.io/ubuntu-slim:0.1 to ubuntu:20.04, new ubuntu images are also slim :)

[povilasv]

maybe @bskiba you could take a look?

[bskiba]

/lgtm
/approve
Thanks a lot for taking the time to clean up the makefiles as well and sorry to keep you waiting so long!

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bskiba

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• vertical-pod-autoscaler/OWNERS [bskiba]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[povilasv]

@bskiba no worries, thanks for reviewing and let me know if anything breaks :)