XText Language Servers Support

[silvergl]

Description

A year ago I used this plugin (version 0.7.1) with the XText language servers. In that old version, there were no problems and everything worked as expected (diagnostics, completion, etc.). The latest version, however, seems not to work with XText LS anymore. On the first look, I managed to make it seemingly work following the latest installation and configuration guides. The status of the language server in the bottom shows "Fully Initialized" and the terminal log shows messages like "Handling a message" every time I type something in the cell. But the diagnostics and autocompletion do not work.

I think that the problem on the LS side can be ruled out since it works with LSP plugins of other text editors.
Is there a way to see the complete message exchange between LS and Jupyter?
The console output via 'Language Server Setting -> loggingLevel: "debug" does not provide this kind of information.
The terminal output in debug mode prints only the fact, that messages are handled and not their content.
Is there generally a better way to debug the communication between the language server and Jupyter?

Context

• Operation System: Windows 10
• Browser and version: Chrome 89
• JupyterLab version: 3.0.10
• jupyterlab-lsp version: 3.4.1
• Language Server: Language Server generated with XText framework
• Language Server installed with: Manually: XText generates a .jar executable
• Language Server Spec

# jupyter_server_config.json
{
  "LanguageServerManager": {
    "language_servers": {
      "a-language-server-implementation": {
        "version": 2,
        "argv": [ "java", "-jar", "path/to/my.jar" ],
        "languages": [ "oconf" ],
        "mime_types":[text/oconf]
      }
    }
  }
}

[krassowski]

I'm sorry that it is not working as intended. In general, if this language server is popular enough we could consider including in our testing pipeline (and auto-detect by default) to avoid such regressions in the future. What do you think?

I usually debug by setting a debug flag on the language server, but indeed only some of the servers support it. If you want to peek the content of the message you could modify:

https://github.com/krassowski/jupyterlab-lsp/blob/39010530eba400bffc56282709343e9fcf8bc778/python_packages/jupyter_lsp/jupyter_lsp/handlers.py#L32

e.g.

self.log.debug("[{}] Handling a message: {}".format(self.language_server, message))

I think it would be fine to have that by default actually - feel free to send a PR.

The console output via 'Language Server Setting -> loggingLevel: "debug" does not provide this kind of information.

Just to be extra sure: this modifies the web developer console output; It should show quite a lot of information. Not all messages, but things related to completion etc. Did you check the developer console in the browser?

[krassowski]

Other things to check:

• is the language server running in stdio mode? If it doesn't, it would be consistent with "it works in some editors but not here"
• what are the full logs from jupyter lab --debug
• what are the full logs from browser developer console?

[silvergl]

Thank you for your feedback.

if this language server is popular enough we could consider including in our testing pipeline (and auto-detect by default) to avoid such regressions in the future. What do you think?

As for this language it is not popular. And as for XText it is not a language but a framework for the language development.

is the language server running in stdio mode? If it doesn't, it would be consistent with "it works in some editors but not here"

Yes, it is. Otherwise, as I think, it wouldn't run on the older version too, because it also required stdio mode.

what are the full logs from jupyter lab --debug
what are the full logs from browser developer console?

Console.txt

localhost-1618691956606.log

[silvergl]

I found a workaround for my problem. First of all thank you again, @krassowski because your feedback inspired me to write a Pipe-Intercept-Skript. With its help I could log the LSP communication between server and client.

'Input.log' contains the messages sent from Jupyter to LS and the 'output.log' contains the LS responses. As you can see, the language server sends empty messages to the client. That was the reason for missing feedback in the notebooks.
And the reason for empty messages was the false file extension.

{"jsonrpc":"2.0","method":"textDocument/didOpen","params":{"textDocument":{"uri":"file:///home/ser/.virtual_documents/Untitled7.ipynb","languageId":"oconf","text":"","version":0}}}

My LS expected the file extension to be ".oconf". So I renamed "Untitled7.ipynb" to "Untitled7.oconf" and it started working.
However, this "dirty hack" is pretty unpracticle because once I close the "Untitled7.oconf" I have to rename it back to ".ipynb", so I can open it again with the Jupyter notebook.

And this point I ask if it could be possible to implement some "fake mode", so that clinet sends a URI with an appropriate extension? Or is there an existing workaround?

input.log
output.log

[krassowski]

We do that automatically already, assuming that the file_extension is returned by the kernel. Does the kernel that you use for oconf notebooks return file_extension in language_info section of kernel_info_reply https://jupyter-client.readthedocs.io/en/stable/messaging.html#kernel-info ?

In past we saw quite a few kernels failing to comply with specs here:

• https://github.com/krassowski/jupyterlab-lsp/issues/407#issuecomment-733685559
• https://github.com/krassowski/jupyterlab-lsp/issues/384#issuecomment-714504630

So we should probably document this requirement it better.

[silvergl]

We do that automatically already, assuming that the file_extension is returned by the kernel. Does the kernel that you use for oconf notebooks return file_extension in language_info section of kernel_info_reply https://jupyter-client.readthedocs.io/en/stable/messaging.html#kernel-info ?

Here is some output in terminal running Jupyterlab

Received kernel info: {'protocol_version': '5.3', 'implementation': 'jupyter-jvm-basekernel', 'implementation_version': '2.3.0', 'language_info': {'name': 'oconf', 'version': '1.0', 'mimetype': 'text/oconf', 'file_extension': '.oconf'}, 'banner': 'oconf - 1.0'}

And in the gui:

So, file_extension is recognized at least bu Jupyterlab.

[krassowski]

Thanks for the update. Hmm.... So it looks like a problem at out end too. I thought that we were adding the file extension (.oconf) but (surprisingly) we do not do that for top-level documents but only for documents embedded in other documents:

https://github.com/krassowski/jupyterlab-lsp/blob/39010530eba400bffc56282709343e9fcf8bc778/packages/jupyterlab-lsp/src/virtual/document.ts#L741-L747

which was introduced by myself in krassowski@5eb345b. I think that instead we should keep adding them if the encodedPath ends with .ipynb, but this will require quite a bit of testing to ensure that the jump functions keep working correctly after such change.