make.bash: GO111MODULE=on breaks build

[davecheney]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

tip

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

darwin/amd64

What did you do?

Daves-MBP(~/go/src) % env GO111MODULE=on ./make.bash
Building Go cmd/dist using /Users/dfc/go1.4.
Building Go toolchain1 using /Users/dfc/go1.4.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
go: cannot determine module path for source directory /Users/dfc/go (outside GOPATH, no import comments)
go tool dist: FAILED: /Users/dfc/go/pkg/tool/darwin_amd64/go_bootstrap install -gcflags=all= -ldflags=all= -i cmd/asm cmd/cgo cmd/compile cmd/link: exit status 1

What did you expect to see?

Successful completion of ./make.bash

What did you see instead?

go: cannot determine module path for source directory /Users/dfc/go (outside GOPATH, no import comments)
go tool dist: FAILED: /Users/dfc/go/pkg/tool/darwin_amd64/go_bootstrap install -gcflags=all= -ldflags=all= -i cmd/asm cmd/cgo cmd/compile cmd/link: exit status 1

This may seem like a contrived example but it is easily conceivable someone may add export GO111MODULE=on to their ~/.bashrc (or similar) to "opt in" to modules everywhere, then be unaware that this setting prevents building Go from source.

[andybons]

@rsc @bcmills @ianlancetaylor

[davecheney]

Here is another example of how GO111MODULE can unintentionally break building tip from source, travis-ci/gimme#158

[cespare]

I was confused when trying to run all.bash/make.bash and it failed (error attached below). It turns out it was because there was a go.mod in an ancestor directory. I mention this here because @bcmills assures me that the fix for my issue is the same as the fix for this one.

$ ./all.bash
Building Go cmd/dist using /Users/caleb/apps/gobootstrap.
Building Go toolchain1 using /Users/caleb/apps/gobootstrap.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for darwin/amd64.
src/cmd/internal/objfile/disasm.go:24:2: unknown import path "golang.org/x/arch/arm/armasm": cannot find module providing package golang.org/x/arch/arm/armasm
src/cmd/internal/objfile/disasm.go:25:2: unknown import path "golang.org/x/arch/arm64/arm64asm": cannot find module providing package golang.org/x/arch/arm64/arm64asm
src/cmd/internal/objfile/disasm.go:26:2: unknown import path "golang.org/x/arch/ppc64/ppc64asm": cannot find module providing package golang.org/x/arch/ppc64/ppc64asm
src/cmd/internal/objfile/disasm.go:27:2: unknown import path "golang.org/x/arch/x86/x86asm": cannot find module providing package golang.org/x/arch/x86/x86asm
can't load package: /Users/caleb/apps/go/src/cmd/pprof/pprof.go:28:2: non-standard import "github.com/google/pprof/driver" in standard package "cmd/pprof"
src/cmd/pprof/readlineui.go:21:2: unknown import path "golang.org/x/crypto/ssh/terminal": cannot find module providing package golang.org/x/crypto/ssh/terminal
can't load package: /Users/caleb/apps/go/src/cmd/trace/pprof.go:24:2: non-standard import "github.com/google/pprof/profile" in standard package "cmd/trace"
can't load package: /Users/caleb/apps/go/src/cmd/vendor/github.com/google/pprof/driver/driver.go:25:2: non-standard import "github.com/google/pprof/profile" in standard package "cmd/vendor/github.com/google/pprof/driver"
src/cmd/vendor/github.com/google/pprof/driver/driver.go:23:2: use of internal package github.com/google/pprof/internal/driver not allowed
src/cmd/vendor/github.com/google/pprof/driver/driver.go:24:2: use of internal package github.com/google/pprof/internal/plugin not allowed
can't load package: /Users/caleb/apps/go/src/cmd/vendor/github.com/google/pprof/internal/binutils/disasm.go:24:2: non-standard import "github.com/ianlancetaylor/demangle" in standard package "cmd/vendor/github.com/google/pprof/internal/binutils"
src/cmd/vendor/github.com/google/pprof/internal/binutils/binutils.go:29:2: use of internal package github.com/google/pprof/internal/elfexec not allowed
src/cmd/vendor/github.com/google/pprof/internal/binutils/addr2liner.go:26:2: use of internal package github.com/google/pprof/internal/plugin not allowed
can't load package: /Users/caleb/apps/go/src/cmd/vendor/github.com/google/pprof/internal/driver/driver.go:30:2: non-standard import "github.com/google/pprof/profile" in standard package "cmd/vendor/github.com/google/pprof/internal/driver"
src/cmd/vendor/github.com/google/pprof/internal/driver/cli.go:23:2: use of internal package github.com/google/pprof/internal/binutils not allowed
src/cmd/vendor/github.com/google/pprof/internal/driver/flamegraph.go:23:2: use of internal package github.com/google/pprof/internal/graph not allowed
src/cmd/vendor/github.com/google/pprof/internal/driver/driver_focus.go:23:2: use of internal package github.com/google/pprof/internal/measurement not allowed
src/cmd/vendor/github.com/google/pprof/internal/driver/cli.go:24:2: use of internal package github.com/google/pprof/internal/plugin not allowed
src/cmd/vendor/github.com/google/pprof/internal/driver/commands.go:30:2: use of internal package github.com/google/pprof/internal/report not allowed
src/cmd/vendor/github.com/google/pprof/internal/driver/options.go:27:2: use of internal package github.com/google/pprof/internal/symbolizer not allowed
can't load package: /Users/caleb/apps/go/src/cmd/vendor/github.com/google/pprof/internal/graph/graph.go:26:2: non-standard import "github.com/google/pprof/profile" in standard package "cmd/vendor/github.com/google/pprof/internal/graph"
src/cmd/vendor/github.com/google/pprof/internal/graph/dotgraph.go:24:2: use of internal package github.com/google/pprof/internal/measurement not allowed
can't load package: /Users/caleb/apps/go/src/cmd/vendor/github.com/google/pprof/internal/measurement/measurement.go:24:2: non-standard import "github.com/google/pprof/profile" in standard package "cmd/vendor/github.com/google/pprof/internal/measurement"
can't load package: /Users/caleb/apps/go/src/cmd/vendor/github.com/google/pprof/internal/plugin/plugin.go:24:2: non-standard import "github.com/google/pprof/profile" in standard package "cmd/vendor/github.com/google/pprof/internal/plugin"
can't load package: /Users/caleb/apps/go/src/cmd/vendor/github.com/google/pprof/internal/report/report.go:33:2: non-standard import "github.com/google/pprof/profile" in standard package "cmd/vendor/github.com/google/pprof/internal/report"
src/cmd/vendor/github.com/google/pprof/internal/report/report.go:30:2: use of internal package github.com/google/pprof/internal/graph not allowed
src/cmd/vendor/github.com/google/pprof/internal/report/report.go:31:2: use of internal package github.com/google/pprof/internal/measurement not allowed
src/cmd/vendor/github.com/google/pprof/internal/report/report.go:32:2: use of internal package github.com/google/pprof/internal/plugin not allowed
can't load package: /Users/caleb/apps/go/src/cmd/vendor/github.com/google/pprof/internal/symbolizer/symbolizer.go:32:2: non-standard import "github.com/google/pprof/profile" in standard package "cmd/vendor/github.com/google/pprof/internal/symbolizer"
src/cmd/vendor/github.com/google/pprof/internal/symbolizer/symbolizer.go:29:2: use of internal package github.com/google/pprof/internal/binutils not allowed
src/cmd/vendor/github.com/google/pprof/internal/symbolizer/symbolizer.go:30:2: use of internal package github.com/google/pprof/internal/plugin not allowed
src/cmd/vendor/github.com/google/pprof/internal/symbolizer/symbolizer.go:31:2: use of internal package github.com/google/pprof/internal/symbolz not allowed
can't load package: /Users/caleb/apps/go/src/cmd/vendor/github.com/google/pprof/internal/symbolz/symbolz.go:30:2: non-standard import "github.com/google/pprof/profile" in standard package "cmd/vendor/github.com/google/pprof/internal/symbolz"
src/cmd/vendor/github.com/google/pprof/internal/symbolz/symbolz.go:29:2: use of internal package github.com/google/pprof/internal/plugin not allowed
src/cmd/vendor/golang.org/x/crypto/ssh/terminal/util.go:20:2: unknown import path "golang.org/x/sys/unix": cannot find module providing package golang.org/x/sys/unix
go tool dist: FAILED: /Users/caleb/apps/go/pkg/tool/darwin_amd64/go_bootstrap install -gcflags=all= -ldflags=all= std cmd: exit status 1 

[hyangah]

I often used 'go install cmd' to rebuild go commands with tiny local changes, and that doesn't work with GO111MODULE=on - I guess for the same reason.

[bcmills]

This should be fixed as of CL 164621.