x/tools/internal/imports: TestModeGetmodeVendor failing due to vendoring change

[jayconrod]

What version of Go are you using (go version)?

$ go version
go version devel +46b7557836 Thu Oct 10 13:56:58 2019 +0000 darwin/amd64

golang.org/x/tools checked out at 0337d82405ff9ebcb2638658c98761e12f51693e

Does this issue reproduce with the latest release?

no, only tip

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jayconrod/Library/Caches/go-build"
GOENV="/Users/jayconrod/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/jayconrod/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/jayconrod/Code/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/jayconrod/Code/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/jayconrod/go/src/golang.org/x/tools/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/rq/x0692kqj6ml8cvrhcqh5bswc008xj1/T/go-build395310348=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

go test golang.org/x/tools/internal/imports -run=TestModeGetmodeVendor

What did you expect to see?

test passes

What did you see instead?

--- FAIL: TestModeGetmodeVendor (0.25s)
    mod_test.go:235: loading package name for rsc.io/quote: running go: exit status 1 (stderr:
        go list -m: can't list modules with -mod=vendor
        	use -mod=mod or -mod=readonly to ignore the vendor directory
        )
    mod_test.go:235: package name for rsc.io/quote = , want quote
    mod_test.go:235: scan failed: running go: exit status 1 (stderr:
        go list -m: can't list modules with -mod=vendor
        	use -mod=mod or -mod=readonly to ignore the vendor directory
        )
    mod_test.go:235: scanning for quote did not find rsc.io/quote
    mod_test.go:235: import path rsc.io/quote not found in active modules
    mod_test.go:240: package name for rsc.io/quote = , want quote
    mod_test.go:240: import path rsc.io/quote not found in active modules
FAIL
FAIL	golang.org/x/tools/internal/imports	0.291s
FAIL

https://build.golang.org/log/4ca136788f50edc4878fe80c76f1ec376539a9b2

[bcmills]

Argh, sorry for the lack of advance warning. I ran the x/tools tests with the vendor change but forgot to look at the output. 😞

[jayconrod]

Caused by some combination of:

• CL 199821 - cmd/go/internal/list: disallow 'list -m' with '-mod=vendor'
• CL 198319 - cmd/go: automatically check and use vendored packages

(I'm putting together a CL right now that skips the test while we figure out what to do)

[gopherbot]

Change https://golang.org/cl/200299 mentions this issue: internal/imports: skip TestModeGetmodeVendor

[bcmills]

Considering the test name, it's probably 199821.

I'm guessing that most of x/tools/internal/imports is actually interested in packages rather than modules per se. If so, the fix is probably to switch from something like

go list -f '{{with .Module}}{{.Path}} {{.Dir}}{{end}}' ...

More generally, though, go list -m is somewhat nonsensical in vendor mode: the vendor directory doesn't contain the entire module and may interleave packages from different modules.

[heschi]

No, it very specifically needs modules, I'm afraid. Listing packages is too slow. I'll talk it through with Jay.

[bcmills]

-mod=vendor does not have accurate module information — and never did, unfortunately.

(It previously reported incomplete module information in vendor mode, but we probably shouldn't build tooling around that.)

If you're detecting vendor mode explicitly, the easy way to list out the available modules is to parse the # module version and # module version => replacement lines from the vendor/modules.txt file. (But that won't get you coherent directories for individual modules, because the vendor directory intentionally flattens the modules into an interleaved directory structure.)

[gopherbot]

Change https://golang.org/cl/203257 mentions this issue: internal/imports: skip TestStdlibNotPrefixed

[gopherbot]

Change https://golang.org/cl/203017 mentions this issue: internal/imports: support vendoring in module mode