Spring Language Server hangs forever

[youcangetme]

Describe the bug
On restart, first start or any changes to a project the Spring Language Server process can start but never complete. If you cancel it it just says it is canceling and never does.

To Reproduce
Just like with #925 I cannot replicated it at will but it does seem to happen after a lot of refreshing of the project.

Sample

Reports that include a sample will take priority over reports that do not.
At times, we may require a sample, so it is good to try and include a sample up front.

[BoykoAlex]

I suspect this is unlikely to be a harmful issue... Most likely what happens is LS starts a progress but then something fails or server re-starts and there is nothing to terminate the progress...
I'll try to reproduce it...

[martinlippert]

Maybe we can find a way to let the progress message to contain more details about what exactly is going on and what exactly the progress is waiting for.

[ts-62020]

If I may add, the CPU utilization goes through the roof in the meantime until the spring job is done.

[BoykoAlex]

This is likely something in LSP4E... all progress tasks started from Boot LS side have title set which means that under the progress bar the title would be displayed such as "Loading Rewrite Recipes..." etc. Likely lsp4e starts a job on the client that waits for the server response which never arrives...

[martinlippert]

@ts-62020

If I may add, the CPU utilization goes through the roof in the meantime until the spring job is done.

What exactly do you see in the progress view when the CPU utilization goes up? Maybe this specific observation is related to #1068 (just guessing here). When you see jobs that talk about "reconciling" while the CPU goes up, that would be a strong indicator that you are observing #1068.

If you see something else, please mention what exactly appears in the progress view when the CPU utilization is high and maybe capture a thread dump (using jps and jstack, while the CPU utilization is high) of the IDE process as well as the BootLanguageServer... progress, that might point us in the right direction for further investigations.

[martinlippert]

@youcangetme In case you come across this item in the progress bar again, it would be good to know what it said before you tried to cancel it and maybe also capture a thread dump of the IDE as well as the BootLanguageServer... process while this appears in the progress view, maybe that gives an idea about what is going on here.

[youcangetme]

@martinlippert I will screen shot it. It happened again last night and I didn't notice and weird parts of my compile and runs started failing. I was really confused as to why the beans were suddenly missing when I started from the Boot Dashboard and finally I looked down and the Spring Lang servers were spinning helplessly. I had to restart Eclipse to kill them. Next time I will screenshot everything.

[martinlippert]

@youcangetme Sounds good, please do not only screenshot, but also remember taking thread dumps as well... :-)

[youcangetme]

threaddump.txt

Hopefully this is what you need.

[martinlippert]

Thanks for capturing the thread dump. Did you observe a high-running CPU while this happens? And if so, on the BootLanguageServerApp process or the main IDE process?

[youcangetme]

Hmmm, Eclipse was nearly frozen. The rest of my computer was fine. I did not observe the CPU directly but it struck me as odd that Eclipse was like 20 chars behind my typing but when I clicked into any other window it was like nothing was wrong.

I will check the CPU next time.

Of note though, I was refactoring both times this happened. Not sure if that is a detail or not but that was the context of what I was doing before the lang server when nuts.

[BoykoAlex]

Looks like this is another instance of Rewrite reconciling that downgrades performance. I see this thread:

"Project-Reconciler-18" #2935 daemon prio=5 os_prio=0 cpu=953.12ms elapsed=1196.36s tid=0x00000252b5e901f0 nid=0x3710 runnable  [0x0000009bc41fe000]
   java.lang.Thread.State: RUNNABLE
	at java.io.FileOutputStream.writeBytes([email protected]/Native Method)
	at java.io.FileOutputStream.write([email protected]/FileOutputStream.java:349)
	at java.io.BufferedOutputStream.write([email protected]/BufferedOutputStream.java:123)
	- locked <0x00000000c089f0f0> (a java.io.BufferedOutputStream)
	at java.io.PrintStream.write([email protected]/PrintStream.java:568)
	- locked <0x00000000c089f0c8> (a java.io.PrintStream)
	at java.io.PrintStream.write([email protected]/PrintStream.java:616)
	at org.eclipse.lsp4j.jsonrpc.json.StreamMessageConsumer.consume(StreamMessageConsumer.java:68)
	- locked <0x00000000c1466f98> (a java.lang.Object)
	at org.springframework.ide.vscode.commons.languageserver.util.ParentProcessWatcher.lambda$apply$0(ParentProcessWatcher.java:129)
	at org.springframework.ide.vscode.commons.languageserver.util.ParentProcessWatcher$$Lambda$673/0x00000001004c96c8.consume(Unknown Source)
	at org.eclipse.lsp4j.jsonrpc.RemoteEndpoint.notify(RemoteEndpoint.java:126)
	at org.eclipse.lsp4j.jsonrpc.services.EndpointProxy.invoke(EndpointProxy.java:88)
	at jdk.proxy2.$Proxy40.publishDiagnostics(jdk.proxy2/Unknown Source)
	at org.springframework.ide.vscode.commons.languageserver.util.SimpleTextDocumentService.publishDiagnostics(SimpleTextDocumentService.java:641)
	at org.springframework.ide.vscode.commons.languageserver.util.SimpleLanguageServer$3.endCollecting(SimpleLanguageServer.java:763)
	at org.springframework.ide.vscode.boot.java.handlers.BootJavaReconcileEngine.lambda$reconcile$12(BootJavaReconcileEngine.java:164)
	at org.springframework.ide.vscode.boot.java.handlers.BootJavaReconcileEngine$$Lambda$1392/0x0000000100a4c370.accept(Unknown Source)
	at java.util.HashMap$Values.forEach([email protected]/HashMap.java:1065)

Looks like this thread isn't blocked and it keeps publishing diagnostics while blocking other threads to read/write json messages to the IDE client. We are working on moving from reconciling Rewrite ASTs to perform the same reconciling on JDT ASTs which is faster by an order of magnitude...
For now for cases like this one i'd disable under Spring -> OpenRewrite preference page "Reconciling of Java Sources" setting.

[ts-62020]

For now for cases like this one i'd disable under Spring -> OpenRewrite preference page "Reconciling of Java Sources" setting.

That seems to have helped. Thank you very much

[martinlippert]

@youcangetme I hope you have that preference Spring -> OpenRewrite -> Reconciling of Java Sources disabled as well. If not, you should definitely disable it and see if things are moving into the right direction.

[youcangetme]

Sorry, Github emails get buried in the Spam folder these days :( Let me do that.

[BoykoAlex]

This is duplicate of #1103 which just got fixed. Try latest snapshots from https://dist.springsource.com/snapshot/STS4/nightly-distributions.html