Doesn't support GOPATH with multiple directories

[TechWilk]

I've just been attempting to try out bud, but ran into issues before I could get going.
Granted, I've not got far into learning Go, I thought bud would help to give me a gentler start to creating a web app, so I may be missing something that is considered "basic Go knowledge".

I've followed your example to get going:

• curl -sf https://raw.githubusercontent.com/livebud/bud/main/install.sh | sh
• bud create test-bud (and followed the steps)
• cd test-bud
• bud run

and I get presented with the following:

~/code/go/src/github.com/techwilk/test-bud$ bud run
| Listening on http://127.0.0.1:3000
| conjure: generate "bud/.cli/program/program.go" > program: unable to wire > di: unable to find definition for param "github.com/livebud/bud/runtime/bud".*Flag within "github.com/techwilk/test-bud/bud/.cli/command".*CLI > stat /home/user/go:/home/user/code/go/pkg/mod/github.com/livebud/[email protected]: no such file or directory

I've previously had Go 1.17, but upgraded to 1.18.2 which didn't change anything.

[TechWilk]

$ bud version
     bud: 0.1.4
  svelte: 3.47.0
   react: 18.0.0

[matthewmueller]

Sorry for the trouble @TechWilk! Let me look into it. It's odd because it appears like you don't have the runtime installed. This should have happened when you ran bud create.

Two questions if you're around:

• What OS are you running?
• Are you installing into $GOPATH or somewhere else on your machine?

[matthewmueller]

Okay, so I just ran the steps you outlined and it's working correctly for me. However, if I manually delete the runtime:

$ trash `go env GOMODCACHE`/github.com/livebud/[email protected]/

I'll get the same error you're seeing:

| Listening on http://127.0.0.1:3000
| conjure: generate "bud/.cli/program/program.go" > program: unable to wire > di: unable to find definition for param "github.com/livebud/bud/runtime/bud".*Flag within "github.com/livebud/issue-78/test-bud/bud/.cli/command".*CLI > stat $GOMODCACHE/github.com/livebud/[email protected]: no such file or directory

I can't really explain why you don't have the runtime installed since it should do that automatically for you with bud create.

One thing that's odd in your error:

stat /home/user/go:/home/user/code/go/pkg/mod/github.com/livebud/[email protected]

Any idea why there's a colon? Can you confirm that the following directory exists? /home/user/code/go/pkg/mod/github.com/livebud/[email protected]

[TechWilk]

Thanks for your quick response!

1. I'm running Debian 11
2. I think it's in GOPATH

echo $GOPATH
/home/user/go:/home/user/code/go

Ah ok, so that reminds me... the first time I was playing with Go the getting started guide I used recommended splitting the GOPATH so that your own projects lived in a separate dir to all the imported modules. e.g. you work in /home/user/code/go but go get puts stuff in the first directory (in this case /home/user/go) to keep your working directories tidier if I'm not mistaken.

I think I've sussed the issue then, it's that bud doesn't support GOPATH with multiple directories.

$ ls /home/user/code/go/pkg/mod/github.com/livebud/[email protected]
ls: cannot access '/home/user/code/go/pkg/mod/github.com/livebud/[email protected]': No such file or directory

$ ls /home/user/go/pkg/mod/github.com/livebud/[email protected]
Changelog.md     go.sum      livebud   package.json       scripts
Contributing.md  install.sh  main.go   package-lock.json  tools.go
docs             internal    Makefile  Readme.md          version.txt
go.mod           License.md  package   runtime

[matthewmueller]

Woah, I had no idea this was possible! I guess it makes sense, $GOPATH being just like $PATH. I guess Go will install all your modules in the first path (e.g. /home/user/go/pkg/mod)? Do you have any packages in /home/user/code/go/pkg/mod?

I'm pretty sure this is the problem! I'll try and come up with a solution.

In the meantime, does the following work for you?

GOPATH=/home/user/go bud run

One more thing, you do remember which getting started guide you used that recommended this? I'd like to learn more.

[TechWilk]

Yes as far as I know that's correct. There's a bunch of stuff in there

$ ls -l /home/user/go/pkg/mod/ | wc -l
16

Thanks! GOPATH=/home/user/go bud run works great!

I don't remember which guide off the top of my head. I'll try and dig it out for you.
It's worth noting that I certainly don't know enough at this point to be able to differentiate between if is in fact a good practice, or, like anything on the web, just a random opinionated person's preference!

[syke99]

It's a bit old, but here is a a SO post that has some good links about multiple GOPATHs and why you'd use them.

I'm also really interested in this issue, so I'm currently researching solutions

[syke99]

@matthewmueller I think I might have found a solution, but it seems there are different solutions for Go1.17 and Go1.18. I think that the 1.17 solution will work with 1.18, but I'm not 100% certain. I'm running 1.18 locally, though, so I can test that out

[matthewmueller]

Awesome, I also got some feedback from the Go community on Twitter:

I used to do this before Modules: "go get" would clone into my first GOPATH, and I would use the second to manually manage the projects I maintained.

IMHO there's no point in a multi-directory GOPATH with Modules nowadays.

This makes total sense to me pre-modules, where you want to keep your maintained projects in a clean directory away from your downloaded modules. Nowadays, go modules are tucked away in $GOPATH/pkg/mod anyways, so not a huge reason to keep this. We certainly still want to support this behavior in Bud though.

@syke99 what are you encountering that leads you to believe there's different solutions for 1.17 and 1.18?

At first glance, I was thinking this would be a fairly simple solution, split on :, and choose the first path to find the runtime. Alternatively, look in each path one by one until you find it or fail trying. This is probably closer to the correct behavior of $GOPATH if it resembles $PATH.

[syke99]

Yeah, I was going to mention that it seems to be a bit outdated of a practice. In the end, yeah, I was going to just recurse into all directories listed, after separating out each path into its own substring. But what I found that leads me to believe the solution would be different is how you go about getting the GOPATH in 1.17 vs 1.18.

1.17: gopath := os.GetEnv("GOPATH")

1.18: gopath := build.Default.GOPATH

after closer examination, you can still use gopath := os.GetEnv("GOPATH") in 1.18, but it seems that it's being treated like println() vs fmt.PrintLn() i.e. println() is still included, but it's considered best practice to use fmt.PrintLn(). I suppose it's really up to you, though, on how you'd like that part to be implemented in the solution.

source: the first answer in this SO post

[matthewmueller]

1.17: gopath := os.GetEnv("GOPATH")

I'm pretty sure build.Default.GOPATH is available pre 1.18, since I've been using it since at least 2017: https://github.com/matthewmueller/joy/blob/e60ec6cf98bbbf7935742e6882fa3d33cadc6a78/internal/mains/mains.go#L32

I'd suggest we use that unless there's good reason not to and then:

At first glance, I was thinking this would be a fairly simple solution, split on :, and choose the first path to find the runtime. Alternatively, look in each path one by one until you find it or fail trying. This is probably closer to the correct behavior of $GOPATH if it resembles $PATH.

[TechWilk]

This makes total sense to me pre-modules, where you want to keep your maintained projects in a clean directory away from your downloaded modules. Nowadays, go modules are tucked away in $GOPATH/pkg/mod anyways, so not a huge reason to keep this. We certainly still want to support this behavior in Bud though.

Right, that makes a lot of sense. I must have been following a pre-modules guide. (bonus, I've now learned about modules!)

At first glance, I was thinking this would be a fairly simple solution, split on :, and choose the first path to find the runtime. Alternatively, look in each path one by one until you find it or fail trying. This is probably closer to the correct behavior of $GOPATH if it resembles $PATH.

According to one of the answers on @syke99's SO link it's worth noting that the delimiter in $GOPATH is OS-specific, so just splitting on : sounds like it wouldn't be suitable for windows

$ go help gopath 
The Go path is used to resolve import statements.
It is implemented by and documented in the go/build package.

The GOPATH environment variable lists places to look for Go code.
On Unix, the value is a colon-separated string.
On Windows, the value is a semicolon-separated string.
On Plan 9, the value is a list.
...

[matthewmueller]

so just splitting on : sounds like it wouldn't be suitable for windows

Thanks for the heads up! I always wondered what os.PathListSeparator was good for 😄

[syke99]

Ahh yeah, I forgot to mention that. Thanks @TechWilk!! depending on how you want to implement a fix, you could simply run a check for the value of the env variable GOOS like so:

    os := runtime.GOOS
    switch os {
    case "windows":
        // split on ";" and recurse through different GOPATHs to look for the module
    default:
        // split on ":" and recurse through different GOPATHs to look for the module
    }

[matthewmueller]

you could simply run a check for the value of the env variable GOOS like so

os.PathListSeparator does this already. Let's lean into Go's standard libraries as much as we can. This will help us on the path to cross-platform compatibility.

[syke99]

Ope, yep. I must’ve missed your comment about os.PathListSeparator and didn’t know it was part of the std lib. I definitely agree to use it!!

[edimoldovan]

Woah, I had no idea this was possible! I guess it makes sense, $GOPATH being just like $PATH. I guess Go will install all your modules in the first path (e.g. /home/user/go/pkg/mod)? Do you have any packages in /home/user/code/go/pkg/mod?

I'm pretty sure this is the problem! I'll try and come up with a solution.

In the meantime, does the following work for you?

GOPATH=/home/user/go bud run

One more thing, you do remember which getting started guide you used that recommended this? I'd like to learn more.

Hi folks,

I was planning to try bud but I ran into similar issues as mentioned in this thread. My setup:

• macOS Ventura, M1
• go1.19 darwin/arm64
• GOPATH not set at all (other go projects run just fine)
• tried GOPATH=/Users/ed/go bud run and received same error:

| Listening on http://127.0.0.1:3000
| app: unable to load state: app: unable to wire. di: unable to wire "change.me/bud/program".loadWeb function. di: unable to find definition for param "github.com/livebud/bud/package/router".*Router in "change.me/bud/internal/app/web".*Server . parser: unable to find declaration for "github.com/livebud/bud/package/router".Router in "bud/internal/app/web/web.go". stat /Users/ed/go/pkg/mod/github.com/livebud/[email protected]: no such file or directory```

Any suggestion about how to try it out?

[matthewmueller]

Hey @edimoldovan, that's weird. Can you share your go env? I'm curious what your default GOPATH is.

It seems like the runtime wasn't downloaded. What commands did you run before you reach this error?

Can you try go getting the runtime?

go get -u github.com/livebud/bud