Texlab exits with non-zero exit code

[clason]

I am using texlab with neovim's builtin LSP client. Everything works fine, but when I exit neovim (which then sends a shutdown rpc request to texlab), it prints

Client 1 quit with exit code 101 and signal 0

to the terminal (because the exit code is non-zero). Do you know why this is happening? I couldn't find anything regarding code 101 in the sources...

[pfoerster]

Thanks for the report. To me, it looks like, texlab is panicking after receiving the exit notification because the client closes the console streams. The panic should result in a 101 exit code. Can you try 56e5228, please? Terminating the process before this happens should resolve the issue.

[clason]

I think you're right; the lsp client log contains

[ DEBUG ] 2021-04-30T16:37:29+0200 ] /usr/local/share/nvim/runtime/lua/vim/lsp/rpc.lua:390 ]^I"rpc.send.payload"^I{  id = 3,  jsonrpc = "2.0",  method = "shutdown"}
[ ERROR ] 2021-04-30T16:37:29+0200 ] /usr/local/share/nvim/runtime/lua/vim/lsp/rpc.lua:457 ]^I"rpc"^I"/Users/clason/.cache/nvim/lspconfig/texlab"^I"stderr"^I"thread 'main' panicked at 'Unexpected response received',crates/jsonrpc/src/client.rs"
[ ERROR ] 2021-04-30T16:37:29+0200 ] /usr/local/share/nvim/runtime/lua/vim/lsp/rpc.lua:457 ]^I"rpc"^I"/Users/clason/.cache/nvim/lspconfig/texlab"^I"stderr"^I":74:14\nnote: run with `RUST_BACKTRACE=1` environment variable to display a backtrace\n"

(Sorry, should have added logs from the beginning.)

Unfortunately, the commit doesn't seem to help, but I'm doing a full rebuild right now to make sure.

Should neovim be doing something in another order?

[pfoerster]

Hmm, it looks like texlab crashes because it receives a message, which it does not deserialize correctly. I don't think its a fault of neovim. Instead, the error originates from our jsonrpc library, which I already replaced with the rust-analyzer lsp library on my development branch. I will probably push the branch the next few days. This error (along with many other bugs) should not occur with the new version. I will add a comment here once you can try it out.

[clason]

Great, thank you! It's only cosmetic, so no hurry!

[pfoerster]

@clason You can now try the breaking-changes branch, which fixes a lot of stuff and is a huge internal refactoring.
As the name implies, there are some breaking changes:

• The options format is a bit different. Everything is now under the texlab setting and some settings have been reorganized:
  texlab.rootDirectory is the same as before, texlab.auxDirectory is the directory containing the .aux and .log files (the old buildDirectory setting, the old name was a bit confusing because some people thought that this would affect the actual output directory, which it does not, texlab.bibtexFormatter is the formatter used for BibTeX (either texlab or latexindent))
• One unreliable feature has been removed for now until there is a better solution available, namely previewing math equations. In the old version, we used to call the TeX engine, but this approach is very slow and a bit unreliable because it can randomly break because of errors in the preamble.
• ChkTeX is not implemented yet because I am still trying to find a way to avoid the bug. For now, however, there is some static linting based on the parser like mismatched environments or unbalanced braces (see Add static linting (light) for TeX files #323).
• The parsers have been rewritten from scratch because their old approach was quite limitating (see for example Complete cite= in acros \DeclareAcronym #324).
• The LSP layer has been replaced with a rust-analyzer crate, which is easier to use and has a smaller dependency footprint. This should fix this issue in particular.
• Parsing the log files now relies on file system watchers, which makes it easier to pick up build errors (Build errors undetected #281).
• The document symbols should now be way more accurate thanks to the new parser.
• Latexindent now picks up the local config YAML file in the workspace folder (Allow for passing extra arguments to latexindent #365).
• I did not end up using tree-sitter after all because it made error handling quite difficult and the performance was actually worse because traversing the tree turned out to be quite slow. A fast non-incremental parser is speedy enough.
• Distribution detection now longer produces a message in the client. Instead it uses a log message in the stderr log.
• Some features are not implemented yet like ChkTeX linting and for some features I am not sure if they really belong into the server like building or forward search. For example, building turned out to be quite buggy and it provides little benefit if implemented in the server. For most cases, a shell with latexmk -pvc suffices and now texlab picks up the build log errors without any additional work. With forward search, it is basically the same thing. The only thing texlab does in this case is reading the config and calling an external process. If we were to re-add those features, then, they should be implemented as commands and not as custom requests instead to be more in line with the LSP spec.

There are probably more changes that I have not mentioned here, but it would be really awesome if you and anyone else can try to find any regressions, be it in functionality or performance-wise. Once ChkTeX is added again and the regressions are fixed, I will think about adding a feature like semantic highlighting or maybe the grammar checker and then publish a 3.0 release.

[clason]

Awesome! I'll start test-driving it for my work. From what you write, it should be a drop-in replacement and even improvement for me since a) I never liked chktex and b) used VimTeX for building and forward search; I agree that an external build system like latexmk or arara is the better option. (The only thing that got me curious about the LSP request was the possibility for progress messages, but I never got around to testing it; these messages also sounded quite brittle, in general.)

I am a bit sad to see the equation preview go, as I used that as an example for pushing image support in console :] But I agree that it's too fragile and never worked for me for complex projects with many macros.

(And thanks for releasing the parsers, especially if you didn't end up using them! They're already used by nvim-treesitter and work really well!)

[clason]

It even builds more quickly now (but not much more quickly -- it's still Rust, after all 😁 )

[clason]

First, it seems noticeably snappier! 👍🏻

One major regression I noticed, though, is that hover on a \ref (and friends) no longer works? At least on one larger project; on a single-file project it still works.

On the same larger project, the workspace/Symbol request also returns more candidates now for an empty string -- some sections from a class file, where these are styled, e.g. % \Part[{#1}]{\MakeUppercase{#2}}%… (note that it's commented out, too!) -- I suspect that this is related?

And the textDocument/reference request stopped working? It seems to take a second but never return anything.

Finally, the completion stopped working reliably -- at least for autocompletion. For example, typing \a offers no candidates. When it does, It seems to return less candidates, too? It's definitely noticeable for bibtex; I'll have to poke around for other completions.
EDIT It's definitely messing with autocompletion (only); if I trigger it manually on \cite{Foo| and FooBar exists, it tries to insert it after the Foo, so I get \cite{FooFoobar|. Would it help if I logged and compared the responses?
EDIT 2 And now I cannot reproduce the completion issue any more.

[clason]

I love the new static parser errors!

[pfoerster]

It even builds more quickly now (but not much more quickly -- it's still Rust, after all 😁 )

Yeah, the compile times aren't the best, especially for release builds but reducing the numer of dependencies certainly helps a lot. Right now, the biggest culprit is actually citeproc-rs but at the moment, there is not really a good option available.

I am a bit sad to see the equation preview go, as I used that as an example for pushing image support in console :] But I agree that it's too fragile and never worked for me for complex projects with many macros.

Maybe I can get MathJax-Node to work inside a Duktape instance, which should work way better than the current approach.

First, it seems noticeably snappier! 👍🏻

That's very good to hear.

One major regression I noticed, though, is that hover on a \ref (and friends) no longer works? At least on one larger project; on a single-file project it still works.

It looks like hover range was actually to small. Before, you had to be inside the token to pick up the hover and not just hover over the complete command. Now, it should perform better. If it still does not work, I added an additional log statement, which should help to fix the issue (it looks the token the cursor is positioned at).

On the same larger project, the workspace/Symbol request also returns more candidates now for an empty string -- some sections from a class file, where these are styled, e.g. % \Part[{#1}]{\MakeUppercase{#2}}%… (note that it's commented out, too!) -- I suspect that this is related?

That's a bit odd. For now, I removed symbols from document classes and packages but it looks like a lexer issue to me.

And the textDocument/reference request stopped working? It seems to take a second but never return anything.

This should be related to the hover issue and is hopefully fixed now. If not, it would be awesome if you could provide a log or a small example.

Finally, the completion stopped working reliably -- at least for autocompletion. For example, typing \a offers no candidates. When it does, It seems to return less candidates, too? It's definitely noticeable for bibtex; I'll have to poke around for other completions.
EDIT It's definitely messing with autocompletion (only); if I trigger it manually on \cite{Foo| and FooBar exists, it tries to insert it after the Foo, so I get \cite{FooFoobar|. Would it help if I logged and compared the responses?
EDIT 2 And now I cannot reproduce the completion issue any more.

For testing purposes, I ended up reducing the number of completion candidates but I forgot to revert it to the old value. Now, it has the old value again. For the other issue, a log would indeed be helpful if it happens again.

[clason]

Yeah, the compile times aren't the best, especially for release builds but reducing the numer of dependencies certainly helps a lot. Right now, the biggest culprit is actually citeproc-rs but at the moment, there is not really a good option available.

It's OK; I just like poking fun at Rust's slow compile time :)

Maybe I can get MathJax-Node to work inside a Duktape instance, which should work way better than the current approach.

That's an interesting approach. But I don't really consider that a necessary feature -- more fun than actually helpful (for me).

It looks like hover range was actually to small. Before, you had to be inside the token to pick up the hover and not just hover over the complete command. Now, it should perform better. If it still does not work, I added an additional log statement, which should help to fix the issue (it looks the token the cursor is positioned at).

No, that's not it -- it doesn't show anything no matter where I trigger it. It's only \ref and \cref, though -- \eqref works, as does \label (for symbols where \cref doesn't work).

If it doesn't work, I don't see any response from the server. Does texlab write a log file somewhere?

That's a bit odd. For now, I removed symbols from document classes and packages but it looks like a lexer issue to me.

That seems to fix these issues, thanks!

And the textDocument/reference request stopped working? It seems to take a second but never return anything.

This should be related to the hover issue and is hopefully fixed now. If not, it would be awesome if you could provide a log or a small example.

Same problem still, unfortunately.

Minimal example is

\documentclass{article}

\begin{document}
\begin{equation}\label{eq:foo}
  1+1=2
\end{equation}

See \eqref{eq:foo}.
\end{document}

I don't see anything in the (client) logs about the references request. For some reason, it doesn't recognize texlab as supporting this request? resolved_capabilities has find_references = false; not sure why...

(For the record, my "stress test" project is https://arxiv.org/abs/2001.00216 -- you can download the TeX sources under "Other Formats" -- the downloaded archive for some reason doesn't get a file extension; just add it and you should be able to extract it.)

For testing purposes, I ended up reducing the number of completion candidates but I forgot to revert it to the old value. Now, it has the old value again. For the other issue, a log would indeed be helpful if it happens again.

Will do! Haven't seen it since...

[clason]

If you open a PR for the branch, I can comment there instead of here, if that's better?

[clason]

Looking at the workspace/Symbol output, I see that it now includes the minimal example test file in my home directory root in a project that is in a subdirectory (two levels down). Maybe the project parser is too greedy? This is not the case with 2.2.

[pfoerster]

If you open a PR for the branch, I can comment there instead of here, if that's better?

I created a PR over at #408.

I don't see anything in the (client) logs about the references request. For some reason, it doesn't recognize texlab as supporting this request? resolved_capabilities has find_references = false; not sure why...

This should be fixed now: I had the capability still commented out 🤦

[clason]

If you open a PR for the branch, I can comment there instead of here, if that's better?

I created a PR over at #408.

Great, I'll comment there from now on!

This should be fixed now: I had the capability still commented out 🤦

Nice one :)

[clason]

Last one here, I promise: references now work indeed 👍🏻

However, it seems to be noticeably slower now (~1s), but only for some symbols and only the first time? (An example would be the \label{chap:...} in any chapter in the arXiv project I linked above.)

[tiagovla]

I see the issue got closed. Would it be possible to make a release with this bug fixed? Still present in version 2.2.2.

[pfoerster]

I see the issue got closed. Would it be possible to make a release with this bug fixed? Still present in version 2.2.2.

Yes, I will publish a release once GitHub sorts out their actions issue. See https://www.githubstatus.com/

[clason]

And status is green again :)

[pfoerster]

Looks like it was successful this time :)

[clason]

Homebrew tests fail on ARM, unfortunately :(

[clason]

I think the tests need to be adapted since the initialization changed. Do you have a minimal request/response I can use for a test?

[pfoerster]

@clason You can try sending the following "handshake" to stdout:

Content-Length: 76

{"jsonrpc": "2.0","id":0,"method":"initialize","params":{"capabilities":{}}}Content-Length: 53

{"jsonrpc": "2.0","method":"initialized","params":{}}

TexLab should respond with a message containing the server capabilities.

[clason]

Shouldn't that be stdin?

(Just pasting this to the terminal after starting texlab gives Error: expected initialize request, got Err(RecvError).)

[pfoerster]

Shouldn't that be stdin?

Yeah, i meant stdin of course.

(Just pasting this to the terminal after starting texlab gives Error: expected initialize request, got Err(RecvError).)

I think the reason why pasting to the terminal does not work is because of the Unix line breaks (LSP uses Windows line breaks) and it looks like the lsp-server crate from Rust Analyzer rejects them.

Looking at other LSP tests, I think you can just copy the test of Rust Analyzer and it should work just fine.

[clason]

Ah, that explains it (you did switch the underlying language-server framework). Rust-analyzer tests seem to work indeed, thank you!

[pfoerster]

Ah, that explains it (you did switch the underlying language-server framework).

Yeah, we used to implement the LSP layer on our own (the jsonrpc crate in the crates folder) but now we use the same LSP framework as Rust analyzer (the lsp-server crate). This reduces the maintenance effort and also allows easier publishing on crates.io.

Rust-analyzer tests seem to work indeed, thank you!

👍

[clason]

Worked like a charm, homebrew now has 3.0.0 :)