Cpptools consumes about 6GB memory on debian, and the system get no more response.

[nuaalyy]

I don't know why but it really confused me. Cpptools costs lots of memory, and the debian goes down, but on win10, cpptools works well. The only differece is that the boost libs is installed on debian but not installed on win10. Maybe large scale c++ projects are not supported well? The version of CPPTOOL is 0.27.0-insiders3.

[sean-mcmanus]

Have you tried 0.27.0-insiders5 (or the soon to be released 0.27.0)? What about 0.26.3 (i.e. is this a regression)?

Do you mean one cpptools process is using 6 GB or do you mean cpptools-srv? Does your system run out of memory after reaching 6 GB, i.e. does it appear like cpptools is requesting > 6 GB and is running out of memory?

Does this repro before you open any files?

[SantinoKeupp]

I experience similar problems with version 0.27.0 on a gentoo machine. It looks like 0.26.3 is more stable.

[nuaalyy]

The memory is cost by the C/C++ extension, not other applications or processes or services, which is for sure. And version 0.26.3 has been tried, which also has the problem. Once the extension is disabled, everything goes well. I will try the version 0.27, and update a snapshot of the task manager later. If any log is helpful please let me know.

[sean-mcmanus]

@nuaalyy cpptools-srv is part of our extension. Depending on how you're viewing C/C++ extension memory usage, cpptools and multiple cpptools-srv processes could be included in the memory usage count, so it's important for us to know what the per-process memory usage is. Most of our memory usage should be in those processes, but if you're seeing high memory usage in a Visual Studio Code process or "extension host", that would be important to know too.

@SantinoKeupp We are not yet aware of any instability regression in regards to memory usage with 0.27.0 (and we've been in Insider mode for a month). Can you provide more info on what you are seeing?

[nuaalyy]

I tried the latest version, 0.27, same issue. When Find All Reference or Go to Definition, status of cpptools-srv switches to Running, and Memory grows up to even 8GB, and the system hangs. When the cpptools-srv process is killed, system goes back to normal. Wish to be helpful.

[sean-mcmanus]

This is partly related to #4811 . We spawn up cpptools-srv processes based on the core count without checking the available memory. Both the 2.8 GB cpptools and the 6 GB cpptools-srv are unexpected. Are you aware of anything special about the TU you're compiling, such as using a large file or large libraries?

[nuaalyy]

I was just reviewing the codes, not compiling anything. Indeed, some large libraries is included the project sucn as BOOST, which may be an impact factor.

[bobbrow]

Is there an open source codebase we can clone to reproduce this error? Unfortunately, there's not much we can do to solve this without some additional help.

[skramm]

I think I got hit by this one too. VSCode is used only for editing purpose, no compiling.System becomes unresponsive, had to kill VS Code.
Problem is: even after killing it, it seems that cpptools stays in the list of process (see attached screenshot from htop), each of these eating up lots of virtual memory.

OS: Ubuntu 16.04
"About" dialog:
Version: 1.44.0
Commit: 2aae1f26c72891c399f860409176fe435a154b13
Date: 2020-04-08T11:22:13.689Z
Electron: 7.1.11
Chrome: 78.0.3904.130
Node.js: 12.8.1
V8: 7.8.279.23-electron.0
OS: Linux x64 4.4.0-177-generic

[Fla3inH0tCheet0s]

I found that it helps to add exclusion folders. For example, add the following to .vscode/settings.json

"files.exclude": {
  "folder_with_generated_files": true,
  ...
}

[staleh1]

Similar issue with 0.28.2 on Fedora, although not as extensive as OP. Seems to increase memory usage over time.

[skramm]

I even had to stop using it, it became unusable after a couple of minutes (freezes), had to switch to another IDE.
@bobbrow asked for some relevant code base. I'd say: any, with more than 10-20 files. For a really large one, try https://github.com/opencv/opencv/ for example. On a standard Linux-Ubuntu flavor box, just fiddle around the source files, going from usage to definition/declaration of functions and you should experience that in a matter of minutes.

[sean-mcmanus]

@skramm Does the issue repro if you disable our extension? The problem you're hitting may not be caused by our extension -- what process is using CPU or memory? The virtual memory usage shouldn't affect performance. If you kill the VS Code process, you'll need to do it in a way that also kills all the child process (cpptools, cpptools-srv, etc.).

[nuaalyy]

I even had to stop using it, it became unusable after a couple of minutes (freezes), had to switch to another IDE.
@bobbrow asked for some relevant code base. I'd say: any, with more than 10-20 files. For a really large one, try https://github.com/opencv/opencv/ for example. On a standard Linux-Ubuntu flavor box, just fiddle around the source files, going from usage to definition/declaration of functions and you should experience that in a matter of minutes.

No doubt that large scale of projects will take lots of memory to parse the definitions and reference relationships. I will apply more memory for my desktop. Hope it will do. By the way, QT may consume less memory when parsing the projects.

[mgsteinkamp]

Seem to be having the same issue here. cpptools can easily run up 8GB of memory.
VSCode used only for editing, no compiling.
Also, I rarely use the Go-To-Definition feature and it hangs before I even hover over a variable.

Environment:

• Ubuntu 18.04,
• 6 (12) core
• 16GB RAM
• v0.29.0-insiders

Some large, open source libraries used:

• OpenCV
• PCL

[OnkelDon]

Same issue here. It seems as soon as linux kernel is in the search/intellisense path, the leak happens after some time, it eat up all the 32GB! Sometimes I replace the cpptools binary with a script which changes ulimit settings (ulimit -v 8000000), but it's gone after every update. Would be nice if this become a setting or so.

Environment:

• Ubuntu 18.04
• 8 (16) cores
• 32GB RAM
• v0.29.0

Linux kernel 5.6 and some litte binaries compiled against it.

[qfunq]

To confirm and recap: out of the box, linux cpptools/vscode has two bugs, which combine to render the suite almost unusable:

1: A memory leak/over eager searching. cpptools consumes all cpu resources.
2: The cpptools service is slow to/never terminates after vscode has exited, and vscode server seems keen to respawn another cpptools service.

Workarounds:

1: Along with OnkelDons' suggestion of using a script wrapper and ulimit, subsequently launching with cpulimit -l 0 helps slow it's damage, with almost no(!) impact to the user.
To expand on Onkels' suggestion, having UI limits for any extension is a nice general way to code it up, greatly expanding the range of usable extensions. For the moment, here is my cpptools script: (I'm running it on a cloud server, with 12G ram).

#!/bin/bash

ulimit -v 4000000

cpulimit -l 0 /home/qfunq/.vscode-server/extensions/ms-vscode.cpptools-0.29.0/bin/cpptools-raw $@

2: Now the persistence of instances of cpptools is much more general. One apparent cause of the bug is that vscode server is invoked using a shell script .... and scripts are not apps ... but by using dumb-init, they can be converted to have application like behaviour. Since the outer invocation uses sh, we can insert the following into .bashrc

alias sh='/bin/dumb-init /bin/bash'

with the side effect of upgrading every sh invocation to an app. This may be not what is required all the time ... so, in principle, by writing a more sophisticated script to replace sh, it's possible to trap the call to invoke cpptools (or other misbehaving extension/script), and inject it's complete resource configuration dynamically, allowing vscode to be updated without patching it, and fully localising the effect of the resource injection.

On testing, the dumb alias technique transforms vscode server from a script to an application, assuming a clean shutdown. In preliminary testing, there were no additional instances of cpptools created, they terminate as soon as the code server does (which may be after the app has exited), and as of yet, there are no side effects to report of locally aliasing sh (I tend to invoke bash with dumb-init anyway).

A remaining issue is that vscode still seems to want to spawn new cpptools after reloading the developers window. It appears reload is not signalling the child services? This might be patchable by using dumb-init at a higher level?

Edit: The entire workaround for 2) is not working, the alias is never being called because of path issues. When sh is patched globally, the dumb-init environment is not directly compatible with terminal control, requiring some additional wiring to be effective.

[sean-mcmanus]

@sskorol Setting C_Cpp.intelliSenseMemoryLimit to 256 might help, but if the TU require more memory then that won't be sufficient. Setting C_Cpp.intelliSenseEngine to "Tag Parser" would be another workaround. Our minimum spec for memory is probably around 4 GB.

[sskorol]

@sean-mcmanus seems better, thanks. But do you have any plans to optimize cpptoops for remote development on resource-limited boards?

[sean-mcmanus]

@sskorol The cases where memory > 1 GB we may be able to fix/improve, but getting memory usage < 1 GB with cpptools-srv may not be possible given our parser's need for memory. If you have a Windows machine you could try using VS 2019's remote Linux support, which will run the IntelliSense on the host Windows machine with more memory: https://devblogs.microsoft.com/cppblog/linux-development-with-c-in-visual-studio/ .

[sskorol]

@sean-mcmanus well, I have Windows as a second OS on PC. But primarily I use macOS and Linux. Anyway, thanks for the link.

[kefins]

I have encountered the same problem, cpptools took too much memory and triggered OOM.

[Mon Jul 12 15:51:56 2021] Out of memory: Kill process 1631 (cpptools) score 342 or sacrifice child
[Mon Jul 12 15:51:56 2021] Killed process 1631 (cpptools) total-vm:7280960kB, anon-rss:5549740kB, file-rss:0kB, shmem-rss:0kB
[Mon Jul 12 15:51:57 2021] oom_reaper: reaped process 1631 (cpptools), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

OS version info:

 cat /etc/os-release 
NAME="Ubuntu"
VERSION="18.04.1 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.1 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

VSCODE version:

版本: 1.58.0 (system setup)
提交: 2d23c42a936db1c7b3b06f918cde29561cc47cd6
日期: 2021-07-08T06:54:55.083Z
Electron: 12.0.13
Chrome: 89.0.4389.128
Node.js: 14.16.0
V8: 8.9.255.25-electron.0
OS: Windows_NT x64 10.0.19042

C/CPP version:

Version 1.5.1: July 9, 2021
Bug Fixes
cppvsdbg Debugging becomes no-op between 1.4.1 and 1.5.0 #7808

[sean-mcmanus]

@kefins Running out of memory is a general symptom for which there could be many causes. We'd need more repro info to determine what the cause is in your case.

[kefins]

@kefins Running out of memory is a general symptom for which there could be many causes. We'd need more repro info to determine what the cause is in your case.

I know little about the cpptools process and I don't know how to fix it, if you need more info, you can tell me how to gather them. But I think it was caused by intelliSenseEngine, after making it to be disable, the OOM seems never take place again.

[sean-mcmanus]

@kefins It's best to file a new issue: https://github.com/microsoft/vscode-cpptools/issues/new?assignees=&labels=&template=language-service.md&title= , and mention which process is using the memory and how much memory, what version of the C/C++ extension is being used, what OS is being used, what compiler is being used, and any repro project or file you can provide, and if there are any particular operations that trigger the issue, and a description of how fast the memory accumulates, and potentially a call stack of the process when the memory is increasing quickly (https://github.com/microsoft/vscode-cpptools/wiki/Attaching-debugger-to-cpptools-or-cpptools%E2%80%90srv )...some of the C/C++ logging might help (C_Cpp.loggingLevel set to "Debug").

[kefins]

@sean-mcmanus OK, I will file a new issue later.

[github-actions]

Hey @sean-mcmanus, this issue might need further attention.

@nuaalyy, you can help us out by closing this issue if the problem no longer exists, or adding more information.

[riban-bw]

There was mention earlier that the extension uses memory based on quantity of cores. I am running VSCode on a Windows laptop (i5 8GB) connecting remotely to a Raspberry Pi 4 with 2GB RAM and 4 cores. With larger projects the service on the Pi consumes all the available memory over a period of some minutes then it all locks up. There was also mention of disabling some features to make it less memory hungry or running parsers on the local rather than remote machine. Are any of these possible work-arounds for this issue?

[sean-mcmanus]

The memory usage based on cores is just in regards to the number of IntelliSense processes kept alive. If you're hitting the memory issue with just 1 file then you'd be hitting a different issue. 2-4 GB of RAM may not be enough for using our extension (8+ GB would be recommended). We have some additional settings like C_Cpp.maxCachedProcesses in development (unreleased) that could be used to override that behavior (if you're hitting the issue when opening > 1 file).

[fwilmsh]

I'm experiencing a similar issue with memory usage of cpptools process constantly increasing. over about an hour of usage now:

Version of the Cpp Extension is 1.7.1.
I don't have any files exclusions defined.

[sean-mcmanus]

@fwilmsh We need more info. Are you able to set the C_Cpp.loggingLevel to "Debug" and check the C/C++ output to determine what is going on when the memory usage is increasing? Or attach a debugger to cpptools to provide a call stack? See https://github.com/microsoft/vscode-cpptools/wiki/Attaching-debugger-to-cpptools-or-cpptools%E2%80%90srv

[moodboom]

sean-mcmanus This problem seems pretty easily reproducible - if not the leak, then just the massive resource usage - have you tested with boost libraries? With a small boost project, with just the editor open, right now I have 5 cpptools-srv processes each taking up 1.2GB of memory. Completely unreasonable.

Update: crashes left a ton of cpptools-srv and cpptools (and code) background processes running. Had to manually kill them all. That left me with one 1.2GB cpptools-srv process in the background. Still seems high... but better.

[sean-mcmanus]

@moodboom 1.2 GB memory usage with a file that uses boost sounds "by design" (expected). The preprocessed TU for such a file could be hundreds of thousands of lines long. Implementation of #3628 might solve the issue.

[github-actions]

This issue has been closed automatically because it needs more information and has not had recent activity.