Error installing tools when using Go version managers (asdf-vm)

[pecigonzalo]

What version of Go, VS Code & VS Code Go extension are you using?

• Run go version to get version of Go
  • go version go1.14 linux/amd64
• Run code -v or code-insiders -v to get version of VS Code or VS Code Insiders
  • 1.42.1
• Check your installed extensions to get the version of the VS Code Go extension
  • 0.13.1
• Run go env GOOS GOARCH to get the operating system and processor architecture details
  • linux
  • amd64
  • WSL2

Share the Go related settings you have added/edited

"go.autocompleteUnimportedPackages": true,
"go.useLanguageServer": true,
"go.toolsGopath": "~/.gotools",

Describe the bug

Im using asdf-vm and I have a similar but related issue to #2899 where vscode-go fails to install the required go tools because it runs go in a different path than the one the code is on.
This would affect anything that depends on the location potentially, like direnv as well (altho given direnv was run when the shell entered the directory and you open VSCode from that shell, it could be that it works).

Output:

gocode:
Error: Command failed: /home/pecigonzalo/.asdf/shims/go get -v github.com/mdempsky/gocode
asdf: No version set for command go
you might want to add one of the following in your .tool-versions file:

golang 1.14

This is because the exec of go get -v seems to happen outside of the code repository, causing asdf to not find the .tool-versions file and be able to select the correct binary.

I believe its this section:

vscode-go/src/goInstallTools.ts

Lines 177 to 178 in 7f301ed

	// Install tools in a temporary directory, to avoid altering go.mod files.
	const toolsTmpDir = fs.mkdtempSync(getTempFilePath('go-tools-'));

Steps to reproduce the behavior:

1. Install asdf-vm and the golang plugin
2. Install a Go version with it asdf install golang 1.14
3. Create a folder and add the managed version asdf local golang 1.14
   • This should create a .tool-versions containing the version that you selected
4. Use the Go: Install/Update Tools action
5. You will get the error.

Workaround

Set a global default for asdf and go to either system or a fixed version of go.

While this workaround works, it's sub-optimal as many other commands could also be running outside of the PWD of the code and be using an incorrect version of go.

[hyangah]

I don't know about asdf-vm but running go commands in a shared, current directory was not desirable because it affects go.mod file in the current directory, and also the existing go.mod files in the current directory affects the dependencies of installed binaries. If go1.14 is used, I think we can use the new -modfile flag, but this is not applicable for old go version users.

@pecigonzalo Is there an established guideline recommended for asdf users to avoid this go.mod mutation issue?

[pecigonzalo]

@hyangah I dont think there is, IMO its not an asdf-vm problem, but rather how do we retrieve the binaries, as asdf-vm only sets the go shim in the path and that is it, its not a version manager for the go packages.

Maybe using go get is not the best approach for this as AFAIK it retrieves and builds the tool, which is affected by the go version that we use as I understand. So instead we should be retrieving binaries directly.
The main question here would be if different versions of the tools have to match different versions of go, if that is not the case, then just getting binaries or requiring a valid global go version (like I do in the workaround) would be enough.

[hyangah]

@pecigonzalo If it is intended to work with go, it's better support the go eco system's de-facto standard way of installing a binary.

One workaround is to install the necessary tools manually and drop them in the vscode bin path or PATH the Go extension will look for binaries.

Personally, I don't think most tools need to be rebuilt to match the version of go in use. I just ignore all the reinstallation messages most of time (the popup is annoying but it seems to happen only once whenever I switch my go and goroot). Maybe there are some tools that I don't know about but are sensitive to the go version. @ramya-rao-a may know better about this version match requirement.

--
Edit: maybe some tools need to be updated to pick up the matching packages included in some backwards-incompatible standard libraries. Maybe it's time to identify such tools and revisit the version enforcement.

[pecigonzalo]

@hyangah It works with the de-facto way of installing binaries, I never said it does not support doing go get, but for asdf-vm (and many other version managers) the context of the folder where the command is being executed is key to the version detection.

There are in my experience 3 approaches to version managers:

• Modify the PATH when loaded (either venv sourcing, manually, direnv, etc): venv, node_module, nvm

It would work for this, but it requires sourcing or hooking and polluting PATH

• Set the binary globally (or globally in a contained folder): tfenv, g, n

It would work with this, but changing the version for one environment, changes the global go version, causing a lot of other unintended problems.

• Shim the binary and detect it on the fly: adsf-vm, pyenv

It does not affect other concurrently running activities but does not detect the version outside of the path.

As explained the problem with the ones that fall on the 3rd class is vscode-go is running in another working directory, which means the version manager cant detect the version to use, this is the actual workaround as users don't want the installation of those binaries in the context of the go.mod file and as a workaround for that.

Arguably I would say, if users want to install the tools are part of how the code is lined, formatted and inspected then having them in the go.mod is actually the way the go ecosystem would set them.

[stamblerre]

@hyangah It works with the de-facto way of installing binaries, I never said it does not support doing go get, but for asdf-vm (and many other version managers) the context of the folder where the command is being executed is key to the version detection.

I believe what @hyangah meant here is that the go command should always be able to run outside of a Go project. Many instructions suggest running the Go command in a temporary directory, so it seems like the version manager's job to handle this fairly common behavior. We can certainly address this issue in Go 1.14 with -modfile as @hyangah also suggested, but I don't think there is anything we can do for older Go versions.

Arguably I would say, if users want to install the tools are part of how the code is lined, formatted and inspected then having them in the go.mod is actually the way the go ecosystem would set them.

Tool installation has presented a challenge in the context of Go modules, because the majority of tools are not related to the project that a user is working on. Adding indirect dependencies to user's modules as they download tools is not acceptable behavior, which is why this work-around was introduced.

@hyangah: What do you think, are you okay with adding a special case for 1.14 to use -modfile? Seems like the right place to use it, IMO.

/cc @jayconrod, just so you see another use case in action 😄

[pecigonzalo]

I believe what @hyangah meant here is that the go command should always be able to run outside of a Go project. Many instructions suggest running the Go command in a temporary directory, so it seems like the version manager's job to handle this fairly common behavior.

I understand if you do not wish to support this use-case for this extension, but I have to disagree with this, I do not think it the job of the version manager to do that.
I do not recall any Go docs I have used advising to run commands in temporary directories but if you could you point me to those docs I can try and PR the version manager accordingly if possible.

IMO if this is the case, it would be great for Go to use a temp dir inside the working context, like its done for .venv or node_modules (I know its used more like the equivalent vendor but its just an example) as most version managers support inheritance from the parent folder.

Alternatively, it could be something to be resolved by Go supporting a global install param, as npm does with the --global.

As far as I understand this, it is not fair or possible to ask the version manager to understand that a random command run in a random directory wants a version set in a repository. It would sort of
asking it to do package management.

PS: The command is able to run outside of the Go project with my workaround, but it means its running a global Go version that could be different from the project one.

[stamblerre]

I do not recall any Go docs I have used advising to run commands in temporary directories but if you could you point me to those docs I can try and PR the version manager accordingly if possible.

It may not be in formal docs, but it's mentioned frequently in related issues: golang/go#27643, golang/go#24250. It's not ideal, but it is the current state with modules.

IMO if this is the case, it would be great for Go to use a temp dir inside the working context, like its done for .venv or node_modules (I know its used more like the equivalent vendor but its just an example) as most version managers support inheritance from the parent folder.

The only way to do this correctly would be to temporarily create a nested module in your project. This makes me pretty nervous - @hyangah: do you think this is a reasonable idea?

Alternatively, it could be something to be resolved by Go supporting a global install param, as npm does with the --global.

There have been proposals for such a flag, and we'll certainly use it if it gets added in a later version of Go.

PS: The command is able to run outside of the Go project with my workaround, but it means its running a global Go version that could be different from the project one.

This seems like reasonable behavior. As @hyangah said, it shouldn't be necessary to have tools compiled with the same version of Go as you are currently using, and I don't think we have the infrastructure to support easily switching between versions of tools based on your current version of Go.

[pecigonzalo]

If that is acceptable solution then I'll just use the global version configuration and if ok the PR this to indicate that, for other stumbling on the same issue. Thanks for replying so quickly.

…

On Wed, Mar 4, 2020, 10:12 PM Rebecca Stambler ***@***.***> wrote: I do not recall any Go docs I have used advising to run commands in temporary directories but if you could you point me to those docs I can try and PR the version manager accordingly if possible. It may not be in formal docs, but it's mentioned frequently in related issues: golang/go#27643 <golang/go#27643>, golang/go#24250 <golang/go#24250>. It's not ideal, but it is the current state with modules. IMO if this is the case, it would be great for Go to use a temp dir inside the working context, like its done for .venv or node_modules (I know its used more like the equivalent vendor but its just an example) as most version managers support inheritance from the parent folder. The only way to do this correctly would be to temporarily create a nested module in your project. This makes me pretty nervous - @hyangah <https://github.com/hyangah>: do you think this is a reasonable idea? Alternatively, it could be something to be resolved by Go supporting a global install param, as npm does with the --global. There have been proposals for such a flag, and we'll certainly use it if it gets added in a later version of Go. PS: The command is able to run outside of the Go project with my workaround, but it means its running a global Go version that could be different from the project one. This seems like reasonable behavior. As @hyangah <https://github.com/hyangah> said, it shouldn't be necessary to have tools compiled with the same version of Go as you are currently using, and I don't think we have the infrastructure to support easily switching between versions of tools based on your current version of Go. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3087?email_source=notifications&email_token=ACLS7CPOLHSZB53ECHCLSV3RF274ZA5CNFSM4LBBHJH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEN2KCPY#issuecomment-594846015>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACLS7CMMZFOYMKM7XJRZJ73RF274ZANCNFSM4LBBHJHQ> .

[stamblerre]

Looks like this has been resolved. Closing.

[pecigonzalo]

@stamblerre I was not able to comment back before as I did not had the time to generate a proper test env but I actually stumbled upon some issues, as some tools (lets say fmt) dont like to format code for other versions, they seem to conflict with the main go binary or something.

I was working on a clear example of the issue so it was easy to see, but just wanted to comment that the workaround its not 100 effective.

[pecigonzalo]

I have created

❯ tree -a
.
├── 1.13
│   ├── .tool-versions
│   └── main.go
├── 1.14
│   ├── .tool-versions
│   └── main.go
└── system
    └── main.go

system is go1.14.2 from ubuntu packages

// main.go
package main
import "fmt"
func main() {
    fmt.Println("hello world")
}

When I open code ./1.13/ I immediately get the following message

Your Go version is different than before, few Go tools may need re-compiling

which installs to

Using the value /home/user/.gotools from the go.toolsGopath setting.
Installing 17 tools at /home/user/.gotools/bin in module mode.

So having a global 1.14 or other version still makes the tool misbehave, as each time I work on a different version, I need to "recompile" the tools.

Additionally, the output on save shows:

/tmp/tmp.OeaCXp0xNh/1.13>Finished running tool: /home/user/.asdf/shims/go build -i -o /tmp/vscode-gojvanjz/go-code-check .
# math/bits
compile: version "go1.14.2" does not match go tool version "go1.13"
# unicode/utf8
compile: version "go1.14.2" does not match go tool version "go1.13"
# runtime/internal/sys
compile: version "go1.14.2" does not match go tool version "go1.13"
# internal/race
compile: version "go1.14.2" does not match go tool version "go1.13"
# unicode
compile: version "go1.14.2" does not match go tool version "go1.13"
# sync/atomic
compile: version "go1.14.2" does not match go tool version "go1.13"
# internal/cpu
compile: version "go1.14.2" does not match go tool version "go1.13"
# runtime/internal/atomic
compile: version "go1.14.2" does not match go tool version "go1.13"

But if I run /home/user/.asdf/shims/go build -i -o /tmp/vscode-gojvanjz/go-code-check . from the console, it works without issues.

[stamblerre]

Thanks for the clarification. I added golang/vscode-go#146 (comment) to golang/vscode-go#146.

We will be revisiting the way that this extension handles multiple Go versions, and as part of that, we will investigate how to best work with Go version managers. Let's continue the discussion on golang/vscode-go#146 as this needs to be part of the larger conversation.

/cc @hyangah