No semantic tokens (for custom colorization) generated by C/C++ Intellisense

[dasupradyumna]

Bug type: Language Service

• OS and Version: Ubuntu 20.04.3 LTS ( Linux x64 5.13.0-28-generic )
• VS Code Version: 1.64.0
• C/C++ Extension Version: 1.8.2
• Other extensions you installed (and if the issue persists after disabling them): Python, Pylance, VSCodeVim and issue persists after disabling / uninstalling all of them and reloading VS Code
• If using SSH remote, specify OS of remote machine: Local machine

Loading the extension for a C++ file, semantic colorization is not rendered as expected.

Expected behavior

Current behavior

All syntax colorization seen in the above screenshot is the result of TextMate grammars. I confirmed this by uninstalling the C++ extension altogether and checking that all the colors persisted with no change. Upon running the command Developer: Inspect Editor Tokens and Scopes from the VS Code command palette, I noticed that no semantic tokens were being provided for the symbols in the files.
I suspect it's an issue with the Intellisense engine here, since VS Code is able to receive semantic tokens from Python extension (Pylance) in a python project without any problems. I have tried searching online for any setting or configuration I might've missed that's causing this, but to no avail. But I am still able to get autocomplete suggestions while typing and pop up hover information about symbols, so the engine is definitely running and parsing the files.
Only the semantic tokens for custom syntax coloring are missing.

• "editor.semanticHighlighting.enabled": true is present in ~/.config/Code/User/settings.json

• Configurations in c_cpp_properties.json
  {
  "configurations": [
  {
  "name": "Game-Engine",
  "includePath": [
  "${workspaceFolder}/Engine/include",
  "${workspaceFolder}/Engine/vendor/spdlog/include"
  ],
  "intelliSenseMode": "linux-gcc-x64",
  "compilerPath": "/usr/bin/g++",
  "cStandard": "gnu17",
  "cppStandard": "gnu++17"
  }
  ],
  "version": 4
  }

• Logs from running C/C++: Log Diagnostics from the VS Code command palette
  -------- Diagnostics - 6/2/2022, 12:27:27 pm
  Version: 1.8.2
  Current Configuration:
  {
  "name": "Game-Engine",
  "includePath": [
  "${workspaceFolder}/Engine/include",
  "${workspaceFolder}/Engine/vendor/spdlog/include"
  ],
  "intelliSenseMode": "linux-gcc-x64",
  "compilerPath": "/usr/bin/g++",
  "cStandard": "gnu17",
  "cppStandard": "gnu++17",
  "compilerPathIsExplicit": true,
  "cStandardIsExplicit": true,
  "cppStandardIsExplicit": true,
  "intelliSenseModeIsExplicit": true,
  "compilerArgs": [],
  "mergeConfigurations": false,
  "browse": {
  "path": [
  "${workspaceFolder}/Engine/include",
  "${workspaceFolder}/Engine/vendor/spdlog/include",
  "${workspaceFolder}"
  ],
  "limitSymbolsToIncludedHeaders": true
  }
  }
  Translation Unit Mappings:
  [ /home/kenja/CppProjects/Game-Engine/Sandbox/src/app.cpp ]:
  /home/kenja/CppProjects/Game-Engine/Engine/src/EntryPoint.h *
  [ /home/kenja/CppProjects/Game-Engine/Sandbox/src/main.cpp ]:
  /home/kenja/CppProjects/Game-Engine/Sandbox/src/main.cpp
  Translation Unit Configurations:
  [ /home/kenja/CppProjects/Game-Engine/Sandbox/src/app.cpp ]:
  Process ID: 293272
  Memory Usage: 208 MB
  Compiler Path: /usr/bin/g++
  Includes:
  /home/kenja/CppProjects/Game-Engine/Engine/include
  /home/kenja/CppProjects/Game-Engine/Engine/vendor/spdlog/include
  /usr/include/c++/9
  /usr/include/x86_64-linux-gnu/c++/9
  /usr/include/c++/9/backward
  /usr/lib/gcc/x86_64-linux-gnu/9/include
  /usr/local/include
  /usr/include/x86_64-linux-gnu
  /usr/include
  Standard Version: c++17
  IntelliSense Mode: linux-gcc-x64
  Other Flags:
  --g++
  --gnu_version=90300
  [ /home/kenja/CppProjects/Game-Engine/Sandbox/src/main.cpp ]:
  Process ID: 293927
  Memory Usage: 94 MB
  Compiler Path: /usr/bin/g++
  Includes:
  /home/kenja/CppProjects/Game-Engine/Engine/include
  /home/kenja/CppProjects/Game-Engine/Engine/vendor/spdlog/include
  /usr/include/c++/9
  /usr/include/x86_64-linux-gnu/c++/9
  /usr/include/c++/9/backward
  /usr/lib/gcc/x86_64-linux-gnu/9/include
  /usr/local/include
  /usr/include/x86_64-linux-gnu
  /usr/include
  Standard Version: c++17
  IntelliSense Mode: linux-gcc-x64
  Other Flags:
  --g++
  --gnu_version=90300
  Total Memory Usage: 302 MB

------- Workspace parsing diagnostics -------
Number of files discovered (not excluded): 2735

[sean-mcmanus]

I'm not reproing the issue. Can you check to make sure the C_Cpp.enhancedColorization setting is not set to "Disabled"? Also, if you set C_Cpp.loggingLevel to "Debug" do you see cpptools/getSemanticTokens in your logging?

[dasupradyumna]

Hello! Sorry for the late response.
Yes, C_Cpp.enhancedColorization setting is Enabled.
When I open a Cpp file, I see getCodeActions, textEditorSelectionChange, getDocumentSymbols, activeDocumentChange, getFoldingRanges, finishUpdateSquiggles; but no getSemanticTokens in the logging (Debug level).

[sean-mcmanus]

That is odd. @Colengms Any idea what could be going wrong?

The only known issue I'm aware of is #8799 , but it doesn't seem like you're hitting that.

[Colengms]

Hi @pradyudasu . I'm unable to easily repro this. There would seem to be some detail about your environment or settings that we may need to uncover to understand what's going wrong.

Could you provide all of the output from your C/C++ output channel leading up to the issue? It may provide some clues.

[dasupradyumna]

Hello, I have no idea what happened but it's working now.
I can even see the cpptools/getSemanticTokens function in the C/C++ output channel. I recently updated VS Code via apt; that might have fixed it, I guess. I will close the issue for now, and will reopen it if this problem resurfaces. Thank you all for the immediate responses!

[sean-mcmanus]

@pradyudasu You may have hit #8867 .