Standard library JSON metadata is not properly populating CompiledGoFiles

[ellie-idb]

What version of rules_go are you using?

0.51.0-rc1 (a985583)

What version of gazelle are you using?

0.37.0

What version of Bazel are you using?

7.4.1

Does this issue reproduce with the latest releases of all the above?

I believe so.

What operating system and processor architecture are you using?

linux/amd64

Any other potentially useful information about your toolchain?

Remote caching

What did you do?

Use gopackagesdriver with Go 1.23.3 + VS Code

What did you expect to see?

Type checking & hints to work properly for VS Code

What did you see instead?

Errors for all standard library imports, saying that the go/packages driver was unable to load the package, and that there is missing metadata. As well, inside of the gopls console, the following logs are visible:

[Info  - 11:57:02 AM] 2024/11/27 11:57:02 go/packages.Load #1
	view_id="1"
	snapshot=0
	directory=/home/ellie/work/services
	package="@@rules_go~//stdlib:internal/runtime/exithook"
	files=[]

[Info  - 11:57:02 AM] 2024/11/27 11:57:02 go/packages.Load #1
	view_id="1"
	snapshot=0
	directory=/home/ellie/work/services
	package="@@rules_go~//stdlib:net"
	files=[]

[Info  - 11:57:02 AM] 2024/11/27 11:57:02 go/packages.Load #1
	view_id="1"
	snapshot=0
	directory=/home/ellie/work/services
	package="@@//backend/models/gen:gen"
	files=[/home/ellie/work/services/backend/models/gen/cmd.go]

[censored]

[Info  - 11:57:02 AM] 2024/11/27 11:57:02 go/packages.Load #1
	view_id="1"
	snapshot=0
	directory=/home/ellie/work/services
	package="@@rules_go~//stdlib:internal/goexperiment"
	files=[]

[Info  - 11:57:02 AM] 2024/11/27 11:57:02 go/packages.Load #1
	view_id="1"
	snapshot=0
	directory=/home/ellie/work/services
	package="@@rules_go~//stdlib:internal/nettrace"
	files=[]

We've internally forked the gopackagesdriver at @devzero-inc to fix this. It turns out that it's just as easy as setting CompiledGoFiles = GoFiles for all standard library imports. I've also taken the liberty to go ahead and move the driver to using the official driver structs - as called out by @sluongng in #3962. I'm currently waiting on approval to upstream these fixes, but thought I might make an issue to track this and bring it to everyone's attention.