releng: update pull-release-image-kube-cross to set the cluster where it should run

[cpanato]

follow up of #20689

the default cluster does not have the service account that is needed for this job, so adding the cluster that contains the setup

The error when the job tried to run

Pod can not be created: pods "63ae2683-6532-11eb-82fa-d61530c4f03a" ... count test-pods/gcb-builder: serviceaccount "gcb-builder" not found

/assign @saschagrunert @hasheddan @xmudrii @puerco
cc: @kubernetes/release-engineering

[cpanato]

jobs_test.go:351: pull-release-image-kube-cross: presubmits may not run in cluster: k8s-infra-prow-build-trusted

/hold to see how to fix that

[cpanato]

@spiffxp @saschagrunert @hasheddan

We need to trigger a GCB build for PRs to test the images, but looks like the gcb-builder service account is not available in pre-submits, I did not find another service account, does that exist? If not how can we add that?

[hasheddan]

@cpanato I am guessing this is by design as typically we wouldn't be using this SA in a presubmit. However, in this specific case we have set up a testing repo so I would think the optimal scenario here would be to put a scoped token in the untrusted cluster if we want to maintain the idea of not running presubmits in trusted. Would defer to @spiffxp to weigh in though.

[cpanato]

@hasheddan agree with you on this, lets wait for Aaron to see other options

[spiffxp]

The reason the clusters are called "trusted" is that we don't want them running "untrusted" code, e.g. presubmits. We treat the act of merging as the line of trust... code has passed tests, and the relevant humans have reviewed it. So postubmits/periodics on trusted clusters are fine, but not presubmits.

@hasheddan's suggestion of scoped SA in untrusted cluster SGTM, PR to kubernetes/k8s.io would look something like:

• update scripts to make the GCP service account, bind via workload identity to an account of the same name on k8s-infra-prow-build (something like https://github.com/kubernetes/k8s.io/blob/master/infra/gcp/ensure-main-project.sh#L144-L155)
• give it privs to write to gcr for the project you care about (something like https://github.com/kubernetes/k8s.io/blob/35c51085f1096c01516021a8bcb34b9cfda656bd/infra/gcp/ensure-staging-storage.sh#L285-L295)
• add the service account resource to https://github.com/kubernetes/k8s.io/blob/master/infra/gcp/clusters/projects/k8s-infra-prow-build/prow-build/resources/build-serviceaccounts.yaml (eg: https://github.com/kubernetes/k8s.io/blob/master/infra/gcp/clusters/projects/k8s-infra-prow-build-trusted/prow-build-trusted/resources/build-serviceaccounts.yaml)

Now might be a good time to start adding some enforcement around which jobs are allowed to use which service accounts via tests in config/tests/jobs (ssuggested this in kubernetes/k8s.io#1393 (comment))

This approach would also be baby steps toward "we should probably be using a service account per staging project" which might help address gcb quota issues #20652

[spiffxp]

change this to k8s-infra-prow-build

change L93 to serviceAccountName: gcb-builder-releng-test (ref: kubernetes/k8s.io#1626)

[cpanato]

thanks @spiffxp. updated

[cpanato]

the job will run on GCB so I don't think we need many resources

[spiffxp]

/lgtm
/hold cancel

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cpanato, spiffxp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• config/jobs/kubernetes/release/OWNERS [cpanato,spiffxp]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

@cpanato: Updated the job-config configmap in namespace default at cluster test-infra-trusted using the following files:

• key release-config.yaml using file config/jobs/kubernetes/release/release-config.yaml

In response to this:

follow up of #20689

the default cluster does not have the service account that is needed for this job, so adding the cluster that contains the setup

The error when the job tried to run

Pod can not be created: pods "63ae2683-6532-11eb-82fa-d61530c4f03a" ... count test-pods/gcb-builder: serviceaccount "gcb-builder" not found

/assign @saschagrunert @hasheddan @xmudrii @puerco
cc: @kubernetes/release-engineering

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.