cmd/go: TestScript/mod_outside fails at HEAD when run after make.bash

[bcmills]

~$ gotip version
go version devel go1.20-85a2c19b32 Mon Nov 21 21:14:38 2022 +0000 linux/amd64

~$ gotip env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/usr/local/google/home/bcmills/.cache/go-build"
GOENV="/usr/local/google/home/bcmills/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/usr/local/google/home/bcmills/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/usr/local/google/home/bcmills"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/google/home/bcmills/sdk/gotip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/google/home/bcmills/sdk/gotip/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="devel go1.20-85a2c19b32 Mon Nov 21 21:14:38 2022 +0000"
GCCGO="/usr/local/google/home/bcmills/bin/gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="c++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build389363616=/tmp/go-build -gno-record-gcc-switches"

~$ gotip test cmd/go -run=TestScript/mod_outside
vcs-test.golang.org rerouted to http://127.0.0.1:36825
https://vcs-test.golang.org rerouted to https://127.0.0.1:35179
go test proxy running at GOPROXY=http://127.0.0.1:35737/mod
--- FAIL: TestScript (0.02s)
    --- FAIL: TestScript/mod_outside (1.32s)
        script_test.go:134: 2022-11-21T21:35:42Z
        script_test.go:136: $WORK=/tmp/cmd-go-test-592575045/tmpdir635258927/mod_outside2804733253
        script_test.go:154:
            > env GO111MODULE=on
            > [short] skip
            [condition not met]
            # This script tests commands in module mode outside of any module.
            #
            # First, ensure that we really are in module mode, and that we really don't have
            # a go.mod file. (0.005s)
            # 'go list' without arguments implicitly operates on the current directory,
            # which is not in a module. (0.012s)
            # 'go list' in the working directory should fail even if there is a a 'package
            # main' present: without a main module, we do not know its package path. (0.006s)
            # 'go list all' lists the transitive import graph of the main module,
            # which is empty if there is no main module. (0.038s)
            # 'go list' on standard-library packages should work, since they do not depend
            # on the contents of any module. (0.351s)
            # 'go list' should work with file arguments. (0.021s)
            # 'go list' on a package from a module should fail. (0.006s)
            # 'go list -m' with an explicit version should resolve that version. (0.015s)
            # 'go list -m -versions' should succeed even without an explicit version. (0.007s)
            # 'go list -m all' should fail. "all" is not meaningful outside of a module. (0.004s)
            # 'go list -m <mods> all' should also fail. (0.004s)
            # 'go list -m <mods>' should fail if any of the mods lacks an explicit version. (0.004s)
            # 'go list -m' with wildcards should fail. Wildcards match modules in the
            # build list, so they aren't meaningful outside a module. (0.009s)
            # 'go clean' should skip the current directory if it isn't in a module. (0.005s)
            # 'go mod graph' should fail, since there's no module graph. (0.004s)
            # 'go mod why' should fail, since there is no main module to depend on anything. (0.003s)
            # 'go mod edit', 'go mod tidy', and 'go mod fmt' should fail:
            # there is no go.mod file to edit. (0.012s)
            # 'go mod download' without arguments should report an error. (0.004s)
            # 'go mod download' should download exactly the requested module without dependencies. (0.012s)
            # 'go mod download all' should fail. "all" is not meaningful outside of a module. (0.005s)
            # 'go mod vendor' should fail: it starts by clearing the existing vendor
            # directory, and we don't know where that is. (0.005s)
            # 'go mod verify' should fail: we have no modules to verify. (0.004s)
            # 'go get' has no go.mod file to update outside a module and should fail. (0.017s)
            # 'go get' should not download anything. (0.011s)
            # 'go build' without arguments implicitly operates on the current directory, and should fail. (0.005s)
            # 'go build' of a non-module directory should fail too. (0.005s)
            # 'go build' of source files should fail if they import anything outside std. (0.018s)
            # 'go build' of source files should succeed if they do not import anything outside std. (0.068s)
            # 'go build' should succeed for standard-library packages. (0.059s)
            # 'go build' should use the latest version of the Go language. (0.080s)
            # 'go doc' without arguments implicitly operates on the current directory, and should fail.
            # TODO(golang.org/issue/32027): currently, it succeeds. (0.010s)
            # 'go doc' of a non-module directory should also succeed. (0.011s)
            # 'go doc' should succeed for standard-library packages. (0.021s)
            # 'go doc' should fail for a package path outside a module. (0.057s)
            # 'go install' with a version should succeed if all constraints are met.
            # See mod_install_pkg_version. (0.277s)
            # 'go install' should fail if a package argument must be resolved to a module. (0.006s)
            # 'go install' should fail if a source file imports a package that must be
            # resolved to a module. (0.018s)
            # 'go install' for a package in GOROOT should succeed.
            # (It must be non-stale here so that the test does not write to GOROOT). (0.112s)
            > ! stale cmd/addr2line
        script_test.go:154: FAIL: mod_outside.txt:208: stale cmd/addr2line: unexpected success
FAIL
FAIL    cmd/go  1.408s
FAIL

As with #56888, I have no idea why this test is not failing on the build dashboard.

[bcmills]

Bisecting now.

I have a test-only fix, but I'd like to understand why it's broken in the first place. It looks like the building packages and commands step in make.bash is somehow failing to cache the dependencies of cmd/addr2line.

[bcmills]

Bisected to CL 451284, which unfortunately means that it was probably broken before then but masked until Michael fixed the TestScript regression I introduced in refactoring.

[gopherbot]

Change https://go.dev/cl/452458 mentions this issue: cmd/go: in TestScript/mod_outside, run 'go build std cmd' before checking cmd/addr2line staleness

[bcmills]

Turns out that make.bash hasn't been priming the build cache since CL 432535, but it was masked on the builders on the one hand by go tool dist test (which was changed to refresh the cache on the builders in CL 432535), and on the other hand by the checkNotStale in the build phase using its own isolated GOCACHE.

But that's probably ok, because we expect make.bash to produce an environment equivalent to installing a toolchain from scratch, and the cache won't be primed then either..?

[gopherbot]

Change https://go.dev/cl/452677 mentions this issue: cmd/go: remove special case for prebuilt cgo library cache keys

[gopherbot]

Change https://go.dev/cl/452679 mentions this issue: cmd/dist: restore the original GOCACHE before building std and cmd