cmd/go: 'go mod why -m' and 'go mod graph' should report modules in the compatibility graph.

[bcmills]

To ease the migration path for pruned module graphs in Go 1.17 (#36460), we added a -compat flag to go mod tidy (#46141) to preserve additional checksums for Go 1.16.

Ideally, go mod why -m (and perhaps go mod graph?) should also report why those checksums are needed, since they're a bit mysterious otherwise.

CC @jayconrod @matloob

[gopherbot]

Change https://golang.org/cl/329529 mentions this issue: cmd/go: add a -go flag to 'go mod graph'

[bcmills]

CL 329529 updates go mod graph, which I think should suffice to unblock the release.

I would also like go mod why to report this information, but since go mod why is already underpowered for transitive module dependencies (#27900) it's perhaps not critical to improve for 1.17.

[bcmills]

go mod graph -go=1.16 should now show the compatibility graph, so this no longer blocks Go 1.17.

[gopherbot]

Change https://golang.org/cl/334375 mentions this issue: cmd/go: remove a duplicated word from 'go help mod graph'

[bcmills]

I think the remaining work is under the umbrella of #27900, so I'm going to go ahead and close this out — the portion unique to the Go 1.17 release is done.

[liggitt]

does this mean if we want to avoid seeing compatibility deps in go mod graph, we have to pass -go 1.16?

edit: no, that doesn't work

[liggitt]

IIUC, go 1.17 go mod tidy adds all indirect dependencies to go.mod and makes them explicit-yet-indirect requires.

we use go mod graph to get visibility to direct and indirect dependency counts, and with go1.17 it is seeing all of the compatibility deps added by go mod tidy as direct deps... this makes go mod graph significantly less useful for understanding where dependencies are coming from

[jayconrod]

@liggitt go mod graph -go 1.16 will show dependencies that go mod graph -go 1.17 would have pruned out.

I don't think there's any precise way to show direct / indirect dependencies with go mod graph. Even with everything pre-1.17, it won't distinguish // indirect requirements. Some go list fanciness may be the best option for this.

[liggitt]

pre-1.17, go mod tidy would omit indirect dependencies from the go.mod file (unless the go.mod file was requiring a newer version), so go mod graph would only show edges from the main module to deps it actually used or influenced version selection for... by default with 1.17, go mod tidy seems to make go mod graph unusable for understanding which deps the main module is depending directly on or influencing version selection of

[bcmills]

The go.mod file continues to mark indirect dependencies with // indirect comments, just as it did in prior Go versions. go mod graph continues to report the transitive dependency graph, just as it did in prior Go versions.

It is true that go mod graph in a go 1.17 module reports more dependency edges from the main module (to the explicitly-listed // indirect modules), and it is true that that makes go mod graph somewhat less useful for analyzing the modules that provide directly-imported packages.

I think we should address that use-case more directly (ha!), and the right way to do that is probably #40364. In the interim, you can use two passes of go list, I think:

$ PKGS=$(go list -f '{{range .Imports}}{{.}}{{"\n"}}{{end}}' -test ./... | sed 's/ \[.*\.test\]//' | sort -u)
$ go list -f '{{with .Module}}{{.Path}} {{.Version}}{{end}}' $PKGS | sort -u

[liggitt]

was the impact on tools built on top of go mod graph considered? the 1.17 changes to go mod tidy seem to make go mod graph unsuitable for evaluating the module graph

[bcmills]

go mod graph reports exactly the same things that it always reported. If your go.mod file included explicit // indirect dependencies in a pre-1.17 module, they would be included in exactly the same way that they are in a 1.17 module.

(That is: go mod graph has never been an appropriate tool for analyzing direct vs. indirect dependencies, because that requires some very specific assumptions about indirect dependencies not being upgraded.)

[liggitt]

this closed issue is probably the wrong place to have this discussion, I'll open a new issue

[liggitt]

opened #47648