x/tools/gopls: high cpu usage when make changes in package which have some cgo package dependents #50151
Comments
gopherbot
added
Tools
|
I have compiled a modified version of gopls with cgo disabled: it may better to add a command line option |
|
Thanks for the investigation and experiment @zhuah I wonder what's breaking when cgo is completely disabled. (related to #35721) @findleyr we have multiple cgo related issues (e.g. #43560) and I don't know where we are. what do you think about keeping track of cgo functionality & performance improvement under an uber issue? |
hyangah
added
the
gopls/performance
There are still an issue
Yes, but even for cgo packages, it's also acceptable for me without cgo type checking, just like GoLand.
Yes, it's already good enough for me. Also, after disabled cgo, there are large improvement of setting workspace speed. |
|
This needs to be further investigated, but probably won't make the v0.7.5 milestone. Moving to v0.8.0. |
findleyr
modified the milestones:
gopls/v0.8.2,
gopls/v0.8.3,
gopls/v0.8.4,
gopls/on-deck
|
Hey I'm the maintainer for https://github.com/go-gst/go-gst , the gstreamer bindings for golang. Also the accompanying https://github.com/go-gst/go-glib for the required GLib bindings. Since gstreamer is quite a large project, both projects contain a lot of C code and go exported functions. A simple compile from scratch takes around 1-2min depending on the machine. With the current language server this turns into a major headache, because after some typing in the package my machine has around 40-100 gcc processes running and I have to leave the machine to do it's thing, before I can start coding again. I have auto formatting turned on and auto save on focus change, which wasn't a problem in any other golang projects of mine. Also this might be a matter of project structure, in that case I'm happy to hear some improvements.
Since these is are repos for creating C FFIs, turning of cgo is not really an option for the LSP. |
|
@RSWilli I drafted a speculative CL recently that 'fakes' cgo: the type checker just treats C bindings as unknown, but doesn't report issues, and functionality works as normal for non-cgo references. In other words, if I'm in a package that imports "C", gopls won't know anything about "C.Foo" (but won't complain), and other references such as "pkg.Bar" will work as expected. So, this would theoretically reduce the CPU usage and latency of gopls significantly in your workspace, at the expense of losing information about C dependencies. Would such a feature be desirable for you? |
|
@findleyr not really for the bindings, because I need the type C information. Every struct in that project is a go struct that wraps a C pointer, about every method on the structs uses the unsafe package and calls into C in one way or another. In projects depending on go-gst the compilation situation is similar though. There such a feature is very useful, because the importing package doesn't touch cgo at all, and only type checking the go method calls is enough. EDIT: in projects depending on go-gst the situation is not hat bad, because the compiled go-gst is cached after the first run. |
|
I don't always need the type info to be there immediately though. If they arrive eventually this would be enough. I have the feeling though that the LSP starts a new "compilation" of the project, even though the last one hasn't finished yet. The old one is also never invalidated, which end up in a big herd of running c compiling steps. 2mins of importing new modules and typing random code, hovering over some symbols leaves me with this:
and several gcc processes in htop:
After around 30mins of programming I regularly reach 100% load on my machine, mostly coming from gcc. |
|
The worst dependencies I commonly come across are: These compilations are also retried with alarming frequency, it's not clear to me this data is properly cached, or it's too easy to trigger a recompilation. Given the extreme cost it would be great to reuse the data for as long as possible. IIUC - 'gopls' (or the toolchain) shouldn't need to recalculate anything provided the CGO using module and compiler versions don't change. |
|
The same happens for me with https://github.com/diamondburned/gotk4. It's the only package I remember triggering this But then often, in unclear circumstances while editing files, my computer would suddenly have >15 |
|
@mpx @fiatjaf Do you have autoformatting enabled? I feel like this issue is split between the autoformatter and the language server. Since go-gst is a project depending on glib-2.0 (and https://github.com/go-gst/go-glib is a fork of https://github.com/gotk3/gotk3) these are exactly the projects that I struggle with. htop views like this are common for me:
A single gcc compilation of gstreamer/glib isn't cheap. |
Sadly the caching doesn't work if you tinker in the project itself, then you are stuck with waiting after every change. The LSP/cache seems to be clever though and only really recompile when an import changes. |
I use I had some things wrong on my comment before, indeed the big number of processes I get are also I also use |
|
The caching issue may just be due to me only interacting with some projects less frequently. While the versions and toolchain haven't changed, it might be a week or so since I last touched them. That said, I do recall situations where it appears to happen multiple times within a week. Yes, I'm typicaly not attempting to use |
The language server
This is the same for me. I think that the formatting and autocompletion are somehow interfering with each other. If I disable |
gopls version
Build info
golang.org/x/tools/gopls v0.7.3
golang.org/x/tools/[email protected] h1:Lru57ht8vtDMouRskFC085VAjBAZRAISd/lwvwOOV0Q=
github.com/BurntSushi/[email protected] h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
github.com/google/[email protected] h1:BKbKCqvP6I+rmFHt06ZmyQtvB8xAkWdhFyr0ZUNZcxQ=
github.com/sergi/[email protected] h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
golang.org/x/[email protected] h1:Gz96sIWK3OalVv/I/qNygP42zyoKp3xptRVCWRFEBvo=
golang.org/x/[email protected] h1:5KslGYwFpkhGh+Q16bwMP3cOontH8FOep7tGV86Y7SQ=
golang.org/x/[email protected] h1:WUoyKPm6nCo1BnNUvPGnFG3T5DUVem42yDJZZ4CNxMA=
golang.org/x/[email protected] h1:aRYxNxv6iGQlyVaZmk6ZgYEDa+Jg18DxebPSrd6bg1M=
golang.org/x/[email protected] h1:hk7xRoeg0CG1nRLsd5BZLDUgVpA9bnKylGk1p2/BPH0=
golang.org/x/[email protected] h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
honnef.co/go/[email protected] h1:ws8AfbgTX3oIczLPNPCu5166oBg9ST2vNs0rcht+mDE=
mvdan.cc/[email protected] h1:bi/1aS/5W00E2ny5q65w9SnKpWEF/UIOqDYBILpo9rA=
mvdan.cc/xurls/[email protected] h1:59Olnbt67UKpxF1EwVBopJvkSUBmgtb468E4GVWIZ1I=
go env
What did you do?
I'm work on a monorepo with some cgo packages in VSCode, when i modify the core package that cgo packages depends on, the cpu usage goes 100%.
I have tried inspect it with
which shows that there are several
clangprocess created bygo listandcgocommand.i must run
pkill clanginstantly when cpu usage goes 100%, but when i am keep editing, the cpu goes 100% again.so i'm wondering that is there a way to disable cgo in gopls or let gopls do not compile cgo package.
What did you expect to see?
What did you see instead?
Editor and settings
Logs
The text was updated successfully, but these errors were encountered: