Dependency update - Golang 1.22.0

[MadhavJivrajani]

Tracking info

Link to any previous tracking issue: #3076

Golang mailing list announcement: for go1.22 https://groups.google.com/g/golang-announce/c/TpowDYVBMoY/m/3YwLkOoOAAAJ

SIG Release Slack thread: https://kubernetes.slack.com/archives/CJH2GBF7Y/p1707291164014399

Work items for Go1.22rc1

• kube-cross image update: golang: Build 1.22rc1 images #3421

  • image promotion: Image promotion for build-image v1.30.0-go1.22rc1-bullseye.0 k8s.io#6302

• go-runner image update: golang: Build 1.22rc1 images #3421

  • image promotion: Image promotion for build-image v2.3.1-go1.22rc1-bookworm.0 k8s.io#6301

• releng-ci image update: golang: Build 1.22rc1 images #3421

  • image promotion: not promoting this image

Work items for Go1.22rc2

• kube-cross image update: golang: Build 1.22rc2 images #3433

  • image promotion: Image promotion for build-image v1.30.0-go1.22rc2-bullseye.0 k8s.io#6331

• go-runner image update: golang: Build 1.22rc2 images #3433

  • image promotion: Image promotion for build-image v2.3.1-go1.22rc2-bookworm.0 k8s.io#6332

• releng-ci image update: golang: Build 1.22rc2 images #3433

  • image promotion: not promoting this image

After kube-cross image promotion

• kubernetes/kubernetes update (master): [go] Bump images, dependencies and versions to go 1.22rc2 kubernetes#122889

  Ensure the following have been updated within the PR:

  • kube-cross image
  • go-runner image
  • publishing bot rules
  • test image
  • .go-version file

Note
This update may require an update to go.sum files, for example: kubernetes/kubernetes#118507
This will require an API Review approval.

After go-runner image promotion

• distroless-iptables image update: build distroless-iptables with go1.22rc2 #3440

  • image promotion:

After kubernetes/kubernetes (master) has been updated

• k8s-cloud-builder image update: add configs for 1.30 with go1.22 for k8s-cloudbuilder #3439

• k8s-ci-builder image variants update: add configs for 1.30 with go1.22 for k8s-cloudbuilder #3439

• kubekins/krte image updates: Update kubekins to Go 1.22rc2 test-infra#31761

Work items for Go1.22

• kube-cross image update: Build Go 1.22 (1.22.0) images #3454

  • image promotion: Image promotion for build-image v1.30.0-go1.22-bullseye.0 k8s.io#6398

• go-runner image update: Build Go 1.22 (1.22.0) images #3454

  • image promotion: Image promotion for build-image v2.3.1-go1.22-bookworm.0 k8s.io#6399

• releng-ci image update: Build Go 1.22 (1.22.0) images #3454

  • image promotion: Image promotion for releng 1.22-bookworm-0 k8s.io#6400

After kube-cross image promotion

• kubernetes/kubernetes update (master): [go] Bump images, dependencies and versions to go 1.22 kubernetes#123217

  Ensure the following have been updated within the PR:

  • kube-cross image
  • go-runner image
  • publishing bot rules
  • test image
  • .go-version file

Note
This update may require an update to go.sum files, for example: kubernetes/kubernetes#118507
This will require an API Review approval.

After go-runner image promotion

• distroless-iptables image update: build distroless-iptables using go1.22 #3457

  • image promotion: Image promotion for build-image v0.5.1 k8s.io#6402

After distroless-iptables image promotion

• kubernetes/kubernetes update (master): [go] Bump images, dependencies and versions to go 1.22 kubernetes#123217

  Ensure the following have been updated within the PR:

  • distroless-iptables image
  • test image

After kubernetes/kubernetes (master) has been updated

• k8s-cloud-builder image update: Build k8s-ci-builder and k8s-cloud-bulider with Go 1.22.0 and 1.21.7 #3463

• k8s-ci-builder image variants update: Build k8s-ci-builder and k8s-cloud-bulider with Go 1.22.0 and 1.21.7 #3463

• kubekins/krte image updates: Update from golang 1.22rc2 to 1.22 final test-infra#31940

Follow-up items

• Ensure the Golang issue template is updated with any new requirements

cc: @kubernetes/release-engineering
cc: @liggitt

[MadhavJivrajani]

Tracking PRs related to go1.22 that we would need to pick to release branches as part of an eventual update to go1.22:

• e2e framework: adapt unit test to Go 1.22 kubernetes#120750

  • 1.29: merged during 1.29 release cycle.
  • 1.28: Automated cherry pick of #120750: e2e framework: adapt unit test to Go 1.22 kubernetes#122580
  • 1.27: Automated cherry pick of #120750: e2e framework: adapt unit test to Go 1.22 kubernetes#122581
  • 1.26: EoL
  • 1.25: EoL

• [go1.22] .*: bump golang.org/x/tools to v0.16.1 kubernetes#122412

  • 1.29: [1.29][go1.22] .*: bump golang.org/x/tools to v0.16.1 kubernetes#122429
  • 1.28: [1.28][go1.22] .*: bump golang.org/x/tools to v0.16.1 kubernetes#122428
  • 1.27: [1.27][go1.22] .*: bump golang.org/x/tools to v0.16.1 kubernetes#122427
  • 1.26: EoL
  • 1.25: EoL

• golangci-lint v1.55.1 kubernetes#121476

  • 1.29: merged during 1.29
  • 1.28: Automated cherry pick of #120750: e2e framework: adapt unit test to Go 1.22 kubernetes#122580
  • 1.27: Automated cherry pick of #120750: e2e framework: adapt unit test to Go 1.22 kubernetes#122581

---

Tracking verification of conditions for bumping go1.21 on release branches:

• ~6th May: 3 months after go 1.22 release (6th Feb 2024)
• ~17th May: 1 month after Kubernetes 1.30 release on go1.22 (17th Apr 2024)
• no regressions relative to go 1.22 known to impact Kubernetes
  • runtime: SIGSEGV after performing clone(CLONE_PARENT) via C constructor prior to runtime start golang/go#65625 - Fixed in go1.22.4
• pass unit and integration tests with both go 1.22 and the original go version for the release branch
  • 1.27: [1.27] DO NOT MERGE: test 1.27 with go1.22.3 kubernetes#124921
  • 1.28: [1.28] DO NOT MERGE: test 1.28 with go1.22.3  kubernetes#124922
  • 1.29: [1.29] DO NOT MERGE: test 1.29 with go1.22.3 kubernetes#124923
• any behavior changes in go 1.22 are mitigated to preserve existing behavior without requiring action by Kubernetes end-users
  • 1.26: NA
  • 1.27: NA
  • 1.28: NA
  • 1.29: NA

---

Tracking the PR that actually bumps to go1.22:

• • 1.26: EoL
  • 1.27: [release-1.27] [go] upgrade to go1.22 kubernetes#125701
  • 1.28: [release-1.28] [go] upgrade to go1.22 kubernetes#125670
  • 1.29: [release-1.29] [go] upgrade to go1.22 kubernetes#125668

[pohly]

kubernetes/kubernetes#120750 is getting cherry-picked in:

• 1.29: not needed (was included)
• 1.28: Automated cherry pick of #120750: e2e framework: adapt unit test to Go 1.22 kubernetes#122580
• 1.27: Automated cherry pick of #120750: e2e framework: adapt unit test to Go 1.22 kubernetes#122581

[cpanato]

/assign
i will work on the release engineering side, image bumps and other things :)

[liggitt]

add golang/go#65625 as a potential go1.22 regression to track (either until it is resolved, or until we are sure it doesn't impact Kubernetes' use of go)

[cpanato]

maybe we can close this issue and open a new one to track #3280 (comment)

wdyt? @liggitt @MadhavJivrajani

[MadhavJivrajani]

@cpanato I'm not sure I fully understand why we need a new issue - maybe I'm missing something?

@liggitt - tracked golang/go#65625 as a regression, thanks!

[liggitt]

maybe we can close this issue and open a new one to track #3280 (comment)

I have a slight preference for keeping everything associated with the minor update in a single issue, but it is a little weird for an issue titled go1.22.0 to track the minor update of release branches to an eventual go1.22.x... maybe we could truncate the title to go1.22?

[cpanato]

ok not a problem, was more a suggestion :) we keep this open

[liggitt]

kubernetes/kubernetes#121476 bumped honnef.co/go/tools to v0.4.6

it turns out that release specifically contained a fix for evaluating go1.22: https://github.com/dominikh/go-tools/releases/tag/2023.1.6

please add that to the list of updates tracked in #3280 (comment) needed for go1.22

[MadhavJivrajani]

@liggitt good catch! Thanks, I've tracked the PR.

[MadhavJivrajani]

Now that we are about to start the process of bumping the Go major version on release branches, we have one tracked regression that seems to affect runc: golang/go#65625

However, this issue is not a bug in Go itself but in glibc (specifically starting from glibc 2.25): golang/go#65625 (comment). However, the issue seems to be triggered in go1.22 because of it calling pthread_getattr_np(pthread_self(), &attr)

There is an attempt at gracefully handing this in go, however that also does not seem to be a sustainable solution: https://go-review.googlesource.com/c/go/+/585019

The workaround here needs to be in runc itself as mentioned here: opencontainers/runc#4233

cc @liggitt

[MadhavJivrajani]

cc @akhilerm

[BenTheElder]

Though also we're pretty sure that bug only affects runc with nsenter, which kubelet is not doing, so as long as your runc binary is not with go 1.22 kubelet can be as far as I know.

[humblec]

Now that we are about to start the process of bumping the Go major version on release branches, we have one tracked regression that seems to affect runc: golang/go#65625

However, this issue is not a bug in Go itself but in glibc (specifically starting from glibc 2.25): golang/go#65625 (comment). However, the issue seems to be triggered in go1.22 because of it calling pthread_getattr_np(pthread_self(), &attr)

There is an attempt at gracefully handing this in go, however that also does not seem to be a sustainable solution: https://go-review.googlesource.com/c/go/+/585019

The workaround here needs to be in runc itself as mentioned here: opencontainers/runc#4233

cc @liggitt

The GO cherry-pick has been approved golang/go#67650 and mostly its going to be part of Go 1.22.4.

[MadhavJivrajani]

I see an incoming Go CVE: golang/go#67682

Which means go1.22.4 might come sooner than expected. @dims / @liggitt thoughts on waiting it out? The other alternative is we can bump release branches to go1.22.3 since the regression does not affect K8s itself. We would however have to be on a differing Go version than runc for at least one patch release, which could be acceptable.

[humblec]

I see am incoming Go CVE: golang/go#67682

Which means go1.22.4 might come sooner than expected. @dims / @liggitt thoughts on waiting it out? The other alternative is we can bump release branches to go1.22.3 since the regression does not affect K8s itself. We would however have to be on a differing Go version than runc for at least one patch release, which could be acceptable.

@MadhavJivrajani imo, considering a CVE fix is also part of 1.22.4, I think it's good to bump k/k on 1.22.4.

[dims]

The other alternative is we can bump release branches to go1.22.3 since the regression does not affect K8s itself.

My vote is to keep the wheels turning as we may catch things in CI jobs from what we have now. differing Go version than runc is not a concern i have.

[BenTheElder]

+1 @dims, we're often having a different version anyhow when we're using prebuilt upstream binaries versus our source builds ...

Unless we know that the packages kubelet uses are not supported on the go version, this should be fine.

We continue, I think we only have the codegen issue which only applies to older branches when the go image isn't also updated?

[MadhavJivrajani]

@dims @BenTheElder go1.22.4 is slated for Tuesday (4th June): https://groups.google.com/d/msgid/golang-dev/hro34FA2QbSJZ-wwA7vkTQ%40geopod-ismtpd-25?utm_medium=email&utm_source=footer

Considering we've already kicked off testing with 1.22.3 and that hasn't surfaced failing jobs (except the codegen issue), I do think 4th June is reasonable to wait for. Please lmk if you have concerns with this and I can start working towards bumping to go1.22.3. Thanks!

[dims]

@MadhavJivrajani personally i like getting things in and watch the periodic ci jobs over the weekend when there is not that much activity and then come monday there is a set of things to do .. but it's your call :) i am ok either way in this instance.

[dims]

@MadhavJivrajani are we done here?

[cpanato]

i will build the images for the release branches :)

[cpanato]

all done :)
/close

[k8s-ci-robot]

@cpanato: Closing this issue.

In response to this:

all done :)
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[MadhavJivrajani]

Yay! 🎉
Thanks folks!