Cache results of Find All References

[jandyman]

I'm not sure if others are having this problem, but I'm finding Find All References to be very slow on several projects which aren't even that big. In addition, I haven't figured a way limit the scope of the searches. In all other senses, the navigation and Intellisense are better than Eclipse (my previous goto for this), but Find All References is both too slow and something I use ALL the time. It's a continual thorn in my side. Is there a solution to this? It seems like once an index is built for a project, this function should be instantaneous, as it is in Eclipse

[sean-mcmanus]

Find All References scope limitation is being tracked at #4102 .

We need more info on what stage is too slow for you. What does the status indicator in the bottom right show for the majority of your wait time? Does it mention "files confirmed"?

Our telemetry indicates that most of our users are experiencing acceptable performance for Find All References, but some user's with particular projects or hardware are not. During the operation, what does your memory and CPU usage look like? We scale "confirmation" via the core count, so if your machine has too few cores, that could be one factor, or if your machine has too little memory with a high core count, that could be the cause of slowness.

The index we build is able to quickly get a subset of "potential" references, but our implementation has to compile the code in order to "confirm" references (i.e. to type check it) in the target files where references are suspected (same as what VS does). You can click the search icon to get preview of the results.

We're tracking Find All References performance with #4051 .

[jandyman]

I moved my comment about scope to #4102

I'm running a brand new Macbook Pro, so there is no problem with cores or such. I suspect that the reason I see a difference with Eclipse is that it builds an index once and reuses it, whereas VS code is compiling all the files. I don't see anything but a progress bar at the top of the UI until the Find All References search is almost completely done, so I don't really know what you are talking about WRT the "search icon". Maybe there is one in the dialog box that pops up in the lower right corner when the scan is almost completely done, but it is there so briefly (a small portion of the total time to Find All References) that I can't tell.

I hope that helps, but I feel that I'm misunderstanding something.

[sean-mcmanus]

You're supposed to see a UX like the following after 3 seconds:

How long is Find All References taking for you?

[jandyman]

Why the three second wait? I looked at it a bit more, and it looks like in both the projects I've been looking at, the dialog comes up at something like three seconds but then the search is finished not too long afterwards. So the total search may be five seconds or so. You may say that five seconds isn't bad. But when you are using the IDE to navigate across an existing codebase and diagnose call trees, it feels very slow.

Plus, I compared to two other tools. In one project, I compared against Eclipse, in the other, I compared against Xcode. In both cases, the reference search gave results essentially instantaneously. So the user experience is not nearly as good. It is a shame, because the rest of the navigation and code completion experience is top notch.

Plus of course there is issue #4102, where the advertised solutions are not working for me at all.

[wymhuster]

I met the same issue. It seems that vscode just search file every time and don't have index in memory

[sean-mcmanus]

@jandyman @wymhuster Yeah, our implementation is based on VS's implementation, which doesn't store all the reference info in a database (which Eclipse/XCode might do), so 3-5 seconds is currently "by design". Our database just enables us to quickly locate files that have the references, but those files still need to be re-parsed if an existing TU doesn't exist already. We could cache those: #4038 . The manual way to see how much faster that would be would be to open the files with references before re-running find all refereces.

[jandyman]

I get it. But 3-5 seconds is painful when you are using to instantaneous navigation, it is THE major pain point when using VSCode as a C++ dev tool. And previous search results aren't saved either, so you can't go back to the search you did three minutes ago. Combine this with the #4102 issue (unsolved for me), and you get an experience that is not fun. Especially when VS Code feels "so close" in so many other ways. With CodeLLDB, debugging is decent, especially when you fill in the holes with lldb command line commands. The intellisense, code completion, all great.

[sean-mcmanus]

After (if?) VS Code supports streaming of references results, then some results could be showed immediately and then more would be added as they're confirmed (which is what VS does) which might make it seem faster (e.g. the files with the most textual references could have their results streamed first).

I've filed a bug on VS at https://developercommunity.visualstudio.com/content/problem/937753/c-intellisense-find-all-references-takes-seconds-i.html that you could upvote if you have a Microsoft account. I don't believe they have plans to fix it though, since it would involve major architectural changes.

[jandyman]

Well, thanks for your help at least. There is one piece of behavior that is a little suspicious. When I do a Find All References for a symbol that has no "other references" (which are usually worthless anyway), the results are instantaneous. Makes me suspicious of what is going on getting those "other references".

Do you have any comments on my experience with #4102? Is that expected behavior?

[sean-mcmanus]

What are the exact types of the "other references" when you see the slowness? The other references are not expect to increase the performance. Are the "other" references in different TUs and of type "Cannot confirm" or "Not a Reference" -- if so, then the extra time would be from our trying to parse those extra TUs.

[merlynomsft]

Just a quick comment from the C++ IntelliSense team -- we're listening and appreciate the feedback on Find All References. We understand how important it is to provide results as quickly as possible and we're continuing to invest in and improve the responsiveness of Find All References.

[carlhua]

What is the rationale to not store reference into a database? Space is cheap, and when we are dealing with large codebases, I would argue its extremely painful to wait 5+ seconds for the references to show up. It would be highly desirable to give user the option to store the references like Eclipse. This is one of the few reason (I think the last one) that I cannot switch back to Vscode. I would highly recommend the team to consider this option :)

[sean-mcmanus]

@carlhua One reason is that we would have to run our IntelliSense parser over every source file (TU) in the workspace, which would be a lot slower than our fast "tag parser" which doesn't parse symbols in function bodies (or macros/includes/etc.), i.e. we'd have to "build everything" in the workspace instead of just the open TUs like we currently do, and when a shared header file is changed it could cause a lot of processing to rebuild hundreds of TUs. Another reason is that it would take a lot of work to implement, so it would need to become a high enough priority issue compared to other issues the VS and VS Code team are working on. The issue that has been tracking Find All References performance is around number 41 in our most votes list: #4051 (in list: https://github.com/Microsoft/vscode-cpptools/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc ). The VS issue tracking is at https://developercommunity.visualstudio.com/spaces/62/index.html , and the issue I filed for improving Find All References performance has 0 votes compared to the others (https://developercommunity.visualstudio.com/idea/939036/c-intellisense-find-all-references-takes-seconds-i.html).

[jandyman]

I'm not sure that I get this. What is being requested is the that Find All References results are cached, that's all. I don't see why the method used to obtain these results needs to change. I get what is being said about the header files, but in that case the Find All References cache could be invalidated, triggering a new reference search, or (as a nice to have) a dialog could be displayed allowing the user to choose whether to do a new search.

A benefit of this approach is that the results of several Find All Reference searches could be cached, allowing the user to go back to a reference search for a previous symbol. This is also a feature available in Eclipse.

[sean-mcmanus]

Yeah, caching of results could probably be done. Caching in memory would be easier than using than using the database. #4038 is related in that it caches the TUs to make confirmation faster.

[heartacker]

If this feature is simple to implement, implement it as soon as possible. There is a long wait for each looking up the invocation,Even the repeated action has to wait for a long time.

[Spongman]

yeah, this feature is useless in all but the simplest projects. i'm sure it works great on a stage, but in real life where it can take several minutes to complete, it's just not worth using.

$ find . -name '*.[ch]*' | wc -l 
69827

[heartacker]

hi, @sean-mcmanus
anything changed to be able to add fixed (release pending) label?

[sean-mcmanus]

@heartacker Yeah, it should be available in 1.7.0, unless we find some reason to revert it.

[heartacker]

@sean-mcmanus is this available now within newest insider?

[sean-mcmanus]

Not yet, it's still in my working branch, which is still undergoing more testing and bug fixes. The goal is for it to make 1.7.0-insiders2.

[jwbecalm]

it is THE major pain point when using VSCode as a C++ dev tool
agree + 1

[sean-mcmanus]

@heartacker We implemented #4038 to potentially reduce the time needed for the confirmation phase for our pending 1.8.0-insiders (opt-in via C_Cpp.references.maxCachedProcesses), but the "results" are not cached and the lexing phase results (described as "searching") in the progress UI are not cached, so I've removed the "fixed" label from this issue to track further potential improvements.

[YuntianCheng]

for me, it will freeze on confirming the last file.

[bdsoha]

@jandyman
CC: @Spongman

Any updates on this topic?

[Spongman]

No, I gave up and installed the clangd extension long ago…