cmd/go: mod tidy reports toolchain not available with 'go 1.21'

[matthewhughes-uw]

Note: I think this is a question of: is "go 1.21" a valid go directive? since there was no 1.21 release, only 1.21.0, perhaps not

What version of Go are you using (go version)?

$ go version
go version go1.21.0 linux/amd64

Does this issue reproduce with the latest release?

Yes, reproduced on Go 1.21.0

What operating system and processor architecture are you using (go env)?

go env Output

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='0'
GOMOD='/proj/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build435086183=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Given a basic go.mod

$ cat go.mod 
module example.com/foo

go 1.21

Then run:

$ GOTOOLCHAIN="go1.20+auto" go mod tidy
go: downloading go1.21 (linux/amd64)
go: download go1.21 for linux/amd64: toolchain not available

What did you expect to see?

go mod tidy runs successfully. I guess I expect it would pick go1.21.0 as the toolchain if no toolchain is available as 1.21

What did you see instead?

The error output above with a non-zero exit code

More details

The issue was seen when running dependabot on a repo with a go 1.21 directive in go.mod, the GOTOOLCHAIN above was taken from their usage https://github.com/dependabot/dependabot-core/blob/08ac25ebd773cede0c00be9a98e5bb03b680870b/go_modules/Dockerfile#L34

Running the above command with GODEBUG=http2debug=1 I see it:

• requests go.dev/dl/mod/golang.org/toolchain/@v/v0.0.1-go1.21.linux-amd64.zip which returns Location: https://go.dev/dl/mod/golang.org/toolchain/@v/v0.0.1-go1.21.linux-amd64.zip
• requesting that URL then returns location: https://dl.google.com/go/v0.0.1-go1.21.linux-amd64.zip
• requesting that URL gives a 404

Updating the directive to: go 1.21.0 will permit things to run fine.

Update: closed per #62278 (comment)

[seankhliao]

how did you end up with go 1.21 as a directive?

[matthewhughes-uw]

how did you end up with go 1.21 as a directive?

I can't remember exactly, but I think it was rather manual like: go1.21.0 mod edit -go=1.21

[laboger]

I am seeing this problem on linux/ppc64le.
I have built a toolchain from the latest go1.21 branch and set up my PATH to use it.
When I cd'ed to cmd/compile/internal/ssa in a toolchain from master that changes the behavior to what is shown above.
I found it when I tried to do 'go generate' but I get the same behavior from just doing 'go version' from within that directory.
If I cd back to another directory then it provides the correct go version.

$ go version
go version go1.21.0 linux/ppc64le
$ cd ~/golang/plain/go/src/cmd/compile/internal/ssa
$ pwd
/home/boger/golang/plain/go/src/cmd/compile/internal/ssa
$ go version
go: downloading go1.22 (linux/ppc64le)
go: download go1.22 for linux/ppc64le: toolchain not available
$ cd ~/gotests
$ go version
go version go1.21.0 linux/ppc64le

[bcmills]

@matthewhughes-uw, this is as expected: go 1.21 was a development version of the language, for which there is no downloadable release (because it is not a specific version). The release version would be go 1.21.0.

In general if you want to specify a development version in the go line, you must also give a concrete toolchain version. So either of these should work:

go 1.21.0

or

go 1.21
toolchain go1.21.0

[bcmills]

@laboger, the problem you are seeing is similar. go1.21.0 cannot upgrade automatically to a toolchain that supports go 1.22, because no such toolchain has been released. In order to compile a package in a go 1.22 module, you need to use a development build of the toolchain that supports that language version.

In particular, if you are working within $GOROOT/src you should be using exactly the toolchain built by make.bash within that GOROOT — you may need to re-run make.bash and/or check your $PATH. 😅

[bcmills]

As far as I can tell this is all working as designed. @matthewhughes-uw, are there specific documentation changes that would have helped you solve or avoid this problem?

[matthewhughes-uw]

As far as I can tell this is all working as designed

👍

@matthewhughes-uw, are there specific documentation changes that would have helped you solve or avoid this problem?

I think the main issue was my assumption that, previously I could just specify a language version (per https://go.dev/doc/toolchain#version) i.e. go 1.X in my go.mod e.g. go 1.20 but with Go 1.21 I can no longer do this. The docs say:

The version must be a valid Go release version, such as 1.9, 1.14, or 1.21rc1

and within the linked page

The syntax ‘1.N’ is called a “language version”. It denotes the overall family of Go releases implementing that version of the Go language and standard library.

so I guess it wasn't clear to me that a "language version" != a "Go release version" (though it was true for every language version before Go 1.21)

In general if you want to specify a development version in the go line, you must also give a concrete toolchain version. So either of these should work:

🤔 Maybe it's worth documenting something like "If you want to specify a 'language version' in go.mod you should (must? But again, only true for Go 1.21 at the moment) also specify a toolchain version"?

[prymitive]

I think the main issue was my assumption that, previously I could just specify a language version (per https://go.dev/doc/toolchain#version) i.e. go 1.X in my go.mod e.g. go 1.20 but with Go 1.21 I can no longer do this.

To add to this: what really happened here that causes a bit of confusion is that the value of go keyword in go.mod changed it's format.

Before 1.21 release X.Y.Z value (1.20.1) would be invalid so you couldn't put it there.

With 1.21 release you most likely want to put X.Y.Z there since most user would want to put there a real working version, not a dev release.
To add to confusion X.Y value technically works, but it may or may not depending on what was released.

So the way I see it we went from never put .0 to always put .0 (0 being point release number).

Having now also toolchain (which from docs sounds like it's optional) just means that people need to wrap their heads around more complicated set of options to set, as per example from #62278 (comment).

[gopherbot]

Change https://go.dev/cl/643615 mentions this issue: _content: update go.mod examples to use full go versions

[dnwe]

After further consideration and discussion with @hyangah @rsc and @samthanawalla we've decided to change the behavior so that if the go.mod lists go 1.X and the current go version is less than that version, we'll attempt to download go 1.X.0 if it exists.

Many of our users are used to manually writing go 1.X in their go.mod files so it seems reasonable to allow that to continue to have a reasonable meaning. Though once this is fixed and released in 1.23 the fix will only be available to users with local toolchains of 1.23 and later.

Unfortunately even if as a library author you attempt to write go 1.23 as your go directive (to imply "I'm happy for people using any go1.23.x or newer toolchain to import my library) go1.23.6 mod tidy (or whatever version you use) still seems to just immediately replace that with the explicit 1.23.0 and add a toolchain directive:

-go 1.23
+go 1.23.0
+
+toolchain go1.23.6

So really you still end up having to specify a minor (i.e., go 1.23.0 or go 1.24.0)

[matloob]

@dnwe Do any of your module's dependencies have a go directive of go 1.23.0? If they don't, I don't think this should be happening. If they do have that requirement, which is higher than 1.23, it would cause the dependency on the go version to be increased to 1.23.0.

[dnwe]

Yep all the x/ repos have bumped to go 1.23.0 now so pretty much everyone does 😅 (e.g., golang/crypto@89ff08d)

My understanding was that 1.23 was ultimately meant to be a synonym of 1.23.0 though, so it seems like the go cli should treat them as comparable

[matloob]

@dnwe I think I missed something in your previous comment. If your goal is for your requirement to be that "I'm happy for people using any go1.23.x or newer toolchain to import my library" then 1.23.0 is the correct requirement. 1.23 allows pre-release versions of go, but those versions are not supported stable releases. That's why the x/ repos specify a minimum of 1.23.0 rather than 1.23.

About 1.23 vs 1.23.0:

Because 1.23 doesn't allow pre-release versions, 1.23.0 is a higher minimum go version requirement than 1.23. So if you've taken a transitive dependency on a module version (like your example of golang.org/x/crypro@89ff08d) that requires a minimum Go version of 1.23.0 then your module's minimum Go version requirement is increased to 1.23.0. You should still be able to set your module's minimum Go version requirement to 1.24 (rather than 1.24.0) if none of your module's transitive dependencies require a higher version, such as 1.24.0. To require 1.23 you'd have to downgrade all your transitive requirements to versions before the requirement on go 1.23.0 was introduced.

1.23 and 1.23.0 are not meant to be synonyms. There is one case where we collapse the two: in the default case when GOTOOLCHAIN is in auto mode and the go command needs to switch to a higher version of Go, it will download 1.23.0 to satisfy a minimum Go version requirement of 1.23. However, the go command does this because 1.23.0 is the minimum release version of Go that satisfies the minimum version requirement.

[dnwe]

@matloob I guess (as a library module author) I really just miss the behaviour that was present from Go 1.11 - Go 1.19, whereby I only ever had to touch the go directive in my go.mod if I'd chosen to adopt some new language or stdlib feature, like the ioutil → io migration in Go 1.16, or generic in Go 1.18 or the maps/slices packages in Go 1.21, and I just had to bump my go.mod to <major>.<minor> to indicate the minimum stdlib that was required. Most downstream authors took a similar approach and it was all relatively frictionless

Now unless I code my modules to be entirely dependency-free, my choice of go directive is almost invariably dictated by dependencies downstream of me who have arbitrarily bumped their directive to (e.g.,) go 1.23.3 for no particular reason, which means I have to bump mine to match if I want to continue to pull in updates and which means every consumer of my library also has to do the same.

I would have much preferred a model whereby go directive remained as major.minor language version and if toolchain was omitted then there was just an implicit major.minor.x — hence I'd hoped 1.23 being permitted again would have partially achieved that again