Language server not starting for custom language type #384

mikesligo · 2020-10-22T10:26:59Z

Hi. Firstly jupyter-lsp is an awesome project.

Description

I'm struggling to get my custom language server to actually start when editing my custom language.

What's particularly strange is, it starts without an issue if I configure it to work with python. I just can't get it to start when I'm using my custom kernel.

I wonder if you could help me work around this issue? I've looked at all the existing issues and get the impression there might be ambiguity around how languages are discovered (eg. https://github.com/krassowski/jupyterlab-lsp/issues/190, robocorp/robotframework-lsp#13).

Thanks in advance.

Steps to reproduce

$ mkdir test

test/kernel.json:

{
 "argv": ["python", "-m",
          "testkernel", "-f",
          "{connection_file}"],
 "display_name": "test",
 "name": "test",
 "language": "test"
}

$ jupyter kernelspec install --user test

testkernel.py:

from ipykernel.kernelbase import Kernel

class TestKernel(Kernel):
    implementation = 'test'
    implementation_version = '2'
    banner = "Test banner"
    language_version = '2'
    language = 'test'
    language_info = {
                     'mimetype': 'text/x-test',
                     'name': 'test'
    }

    def do_execute(self, code, silent, store_history=True, user_expressions=None, allow_stdin=False):
        if not silent:
            stream_content = {'name': 'stdout', 'text': code}
            self.send_response(self.iopub_socket, 'stream', stream_content)

        return {'status': 'ok',
                'execution_count': self.execution_count,
                'payload': [],
                'user_expressions': {},
               }

if __name__ == '__main__':
    from ipykernel.kernelapp import IPKernelApp
    IPKernelApp.launch_instance(kernel_class=TestKernel)

~/.jupyter/jupyter_notebook_config.py
(put any language server in the argv here if desired)

c.LanguageServerManager.language_servers = {
    "test-language-server": {
        "version": 2,
        "argv": ["touch", "language-server-started"],
        "languages": ["test"],
        "mime_types": [
            "text/x-test",
        ]
    }
}

Findings

When a new notebook is made with the Test kernel, the kernel works as expected. However the langauge server doesn't start up.

Logs

jupyter lab --log-level=DEBUG

[D 09:32:58.078 LabApp] [lsp] The following Language Servers will be available: {
      "test-language-server": {
        "argv": [
          "touch",
          "language-server-started"
        ],
        "languages": [
          "test"
        ],
        "mime_types": [
          "text/x-test"
        ],
        "version": 2
      }
    }

[D 10:17:15.959 LabApp] Received kernel info: {'status': 'ok', 'protocol_version': '5.3', 'implementation': 'test', 'implementation_version': '2', 'language_info': {'mimetype': 'text/x-test', 'name': 'test'}, 'banner': 'Test banner', 'help_links': []}

Enironment

Running in latest scipy-notebook docker. Using conda for package management.
jupyter-lsp 0.9.2
jupyterlab 2.2.8

The text was updated successfully, but these errors were encountered:

krassowski · 2020-10-22T10:38:41Z

Seems like an interesting challenge! I can have a look at the weekend, but a quick - possibly completely wrong - idea would be to set some file_extension too and try to set mimetype (in both places) to the same string as language name.

Also, it would greatly help if you could (before attempting the workaround mentioned above) try to use a browser console to see if there are any errors in there.

bollwyvl · 2020-10-22T11:56:28Z

Does it work with the Editor for text files? But I'll agree, especially notebooks will need the file extension metadata, so they they can be turned into plain files.

bollwyvl · 2020-10-22T12:48:59Z

Having a cookiecutter for a pygls-based language server and jupyter-lsp spec would probably be a good addition to the docs/tests once we get this figured out... For example, we can turn this issue into a gist/binder, then go back through and add templates, and robot test a lab with the new thing running. Until we get the new org setup, it could just be a long-running branch....

mikesligo · 2020-10-22T13:45:10Z

Hey

You're both totally right, I'm missing the file_extension everywhere. I assumed that wasn't important as the files were .ipynb, but clearly jupyter needs them.

It works now really well, thanks for your help!

bollwyvl · 2020-10-22T14:51:09Z

Yeah LSP doesn't (yet) care about ipynb (though there's some M$ EEE brewing, I have a feeling), and we have to write the files to something on disk that a language server will understand. Just for asking: what language do you plan to build your language server in?

mikesligo · 2020-10-22T15:11:22Z

The language server is implemented in scala, using LSP4J as a base (similiar to scala meta https://github.com/krassowski/jupyterlab-lsp/issues/17). At the moment I only have hover implemented but it looks like it works well with jupyter-lsp.

mike-vogel · 2021-01-25T21:52:32Z

My custom language kernel starts and works. I see my custom LSP listed in the LSP Servers -> 'Available' but is also says 'Missing: 1' -> 'plain'. I've tried many variations on the mime_types but no change. It seems like I have the issue with the file extension mentioned in a few issues. Where do I set the file_extension for my language and LSP that is mentioned above?

krassowski · 2021-01-25T21:57:47Z

What kernel or file type are you trying to support? It might not be the file_extension issue after all. Two likely scenarios:

the kernel does not return mime type in KernelInfo response
the file editor does not have appropriate CodeMirror mode installed so it cannot determine the mimetype

In other words if you are seeing "plain" this indicates the issue is not on your side, but on the kernel side or JupyterLab needing additional CodeMirror extension (depending on whether you encounter this issue in notebook or file editor). I can look into the respective kennel of it's open source.

mike-vogel · 2021-01-25T22:03:35Z

Just after typing the above I realized I been just looking at the config files. I added the file_extension into the language_info returned by the Kernel and things connected.

mike-vogel · 2021-01-26T13:11:07Z

My language kernel is named 'sdtm'. My language server is the minimum example from the pygls doc that does completions in response to the comma character. The completion doesn't work. Does seeing the following status display point to what the problem is after I opened a new notebook using my kernel? I don't see any errors in the Jupyterlab debug output.

Status shows:
"Initialized (additional servers needed)"

Status hover shows:
"0/1 virtual document connected (0 connections; waiting for: sdtm)

Full status shows:

LSP servers
Available: 2
pyls
python
r-languageserver
r
Running: 1
sdtm ()
Untitled.ipynb connected

krassowski · 2021-01-26T13:34:43Z

I don't see any errors in the Jupyterlab debug output.

What about browser developer console?
You can enable more verbose logging in the Advanced Settings Editor -> Language Server -> loggingLevel

pygls

pygls in the most recent version violates the Language Server Protocol specification and we had to jump via some hoops to make the jedi-language-server work correctly; in short it returns null where the fields are not set, which is mostly forbidden by the specs except for very specific circumstances. There was a PR from the jedi-language-server author to fix that in pygls but I see that it is now closed; the pygls team works on their own fix to this issue, but it still a work in progress. Could it be that pygls added nulls to your CompletionItem[] response where those are not allowed to be?

My language server is the minimum example from the pygls doc that does completions in response to the comma character.

A code of the JSON response sent from your server would certainly help to understand the situation better. Sometimes a GIF from the UI helps too.

mike-vogel · 2021-01-26T14:04:02Z

Browser Console output when opening a new notebook:

[Log] LSP: waiting for – "Untitled.ipynb" – "to fully load" (501.d0dee3f7a514bb00ee98.js, line 1)
[Debug] Starting WebSocket: ws://localhost:8888/api/kernels/c4713301-f12d-47ee-8dea-27e12ca8f5c4 (jlab_core.ec15f8c091a46864d4c7.js, line 2)
[Log] LSP: – "Untitled.ipynb" – "ready for connection" (501.d0dee3f7a514bb00ee98.js, line 1)
[Log] LSP: will connect using language: sdtm (501.d0dee3f7a514bb00ee98.js, line 1)
[Log] LSP: connection requested – {virtual_document: O, language: "sdtm", document_path: "Untitled.ipynb"} (501.d0dee3f7a514bb00ee98.js, line 1)
{virtual_document: O, language: "sdtm", document_path: "Untitled.ipynb"}Object
[Log] LSP: Connection Socket – {virtual_document: O, language: "sdtm", document_path: "Untitled.ipynb"} (501.d0dee3f7a514bb00ee98.js, line 1)
{virtual_document: O, language: "sdtm", document_path: "Untitled.ipynb"}Object
[Log] LSP: Skipping document update signal: connection not ready (501.d0dee3f7a514bb00ee98.js, line 1, x10)
[Warning] LSP: Connect timed out for sdtm (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] TypeError: undefined is not an object (evaluating 'this.connection.on')
register — 501.d0dee3f7a514bb00ee98.js:1:42734
register — 501.d0dee3f7a514bb00ee98.js:1:72269
dt — 501.d0dee3f7a514bb00ee98.js:1:85727
create_adapter — 501.d0dee3f7a514bb00ee98.js:1:93000
connect_adapter — 501.d0dee3f7a514bb00ee98.js:1:91991
http://localhost:8888/lab/extensions/
asyncFunctionResume
http://localhost:8888/lab/extensions/
asyncFunctionResume
[native code]
promiseReactionJobWithoutPromise
promiseReactionJob

[Warning] TypeError: undefined is not an object (evaluating 'this.connection.getDocumentHighlights') — 501.d0dee3f7a514bb00ee98.js:1:81260

mike-vogel · 2021-01-26T14:19:42Z

More complete log from starting fresh after restarting Jupyter Lab and creating a new notebook:

[Debug] pkiClientExtension.js (pkiClientExtension.js, line 3)
[Debug] __symantecPKIClientMessenger.common (pkiClientExtension.js, line 23)
[Debug] bridge.js (bridge.js, line 7)
[Debug] detection.js (detection.js, line 10)
[Debug] +content/detection.js (detection.js, line 12)
[Debug] -content/detection.js (detection.js, line 24)
[Debug] pkiClientExtension.js after DOM load event:1.0.1---true (pkiClientExtension.js, line 74)
[Debug] bridge.js after DOM load event (bridge.js, line 16)
[Debug] Connecting to PORT_NAME_SYMANTEC_PKI_CLIENT_CONTENT_TO_BACKGROUND... (bridge.js, line 17)
[Debug] Connected: – SafariAppExtension (bridge.js, line 19)
SafariAppExtension
[Debug] checkDatasetProperties: supportsFlavorConfiguration:true (pkiClientExtension.js, line 59)
[Debug] checkDatasetProperties: extensionVersion:1.0.1 (pkiClientExtension.js, line 59)
[Debug] Done. (bridge.js, line 38)
[Warning] Ignored setting registry preload errors. – Array (1) (jlab_core.ec15f8c091a46864d4c7.js, line 2)
Array (1)
[Warning] Ignored setting registry preload errors. – Array (1) (jlab_core.ec15f8c091a46864d4c7.js, line 2)
Array (1)
[Debug] Starting application in workspace: "default" (jlab_core.ec15f8c091a46864d4c7.js, line 2)
[Warning] @krassowski/jupyterlab-lsp:signature settings schema could not be found and was not loaded (501.d0dee3f7a514bb00ee98.js, line 1)
> Selected Element
< <div class="jp-Launcher-cardContainer">…</div>
[Log] LSP: waiting for – "Untitled1.ipynb" – "to fully load" (501.d0dee3f7a514bb00ee98.js, line 1)
[Debug] Starting WebSocket: ws://localhost:8888/api/kernels/8221db6a-fef9-4508-a0ca-709a5e73c923 (jlab_core.ec15f8c091a46864d4c7.js, line 2)
[Debug] Starting WebSocket: ws://localhost:8888/api/kernels/8221db6a-fef9-4508-a0ca-709a5e73c923 (jlab_core.ec15f8c091a46864d4c7.js, line 2)
[Debug] Starting WebSocket: ws://localhost:8888/api/kernels/8221db6a-fef9-4508-a0ca-709a5e73c923 (jlab_core.ec15f8c091a46864d4c7.js, line 2)
[Log] LSP: – "Untitled1.ipynb" – "ready for connection" (501.d0dee3f7a514bb00ee98.js, line 1)
[Log] LSP: will connect using language: sdtm (501.d0dee3f7a514bb00ee98.js, line 1)
[Log] LSP: connection requested – {virtual_document: O, language: "sdtm", document_path: "Untitled1.ipynb"} (501.d0dee3f7a514bb00ee98.js, line 1)
{virtual_document: O, language: "sdtm", document_path: "Untitled1.ipynb"}Object
[Log] LSP: Connection Socket – {virtual_document: O, language: "sdtm", document_path: "Untitled1.ipynb"} (501.d0dee3f7a514bb00ee98.js, line 1)
{virtual_document: O, language: "sdtm", document_path: "Untitled1.ipynb"}Object
[Warning] Editor not found in editor_to_source_line map (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] Could not retrieve current context – Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, …} (501.d0dee3f7a514bb00ee98.js, line 1)
Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, adapter: mt, virtual_document: O, …}Proxy
[Warning] Editor not found in editor_to_source_line map (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] Could not retrieve current context – Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, …} (501.d0dee3f7a514bb00ee98.js, line 1)
Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, adapter: mt, virtual_document: O, …}Proxy
[Warning] Editor not found in editor_to_source_line map (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] Could not retrieve current context – Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, …} (501.d0dee3f7a514bb00ee98.js, line 1)
Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, adapter: mt, virtual_document: O, …}Proxy
[Warning] Editor not found in editor_to_source_line map (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] Could not retrieve current context – Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, …} (501.d0dee3f7a514bb00ee98.js, line 1)
Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, adapter: mt, virtual_document: O, …}Proxy
[Warning] Editor not found in editor_to_source_line map (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] Could not retrieve current context – Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, …} (501.d0dee3f7a514bb00ee98.js, line 1)
Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, adapter: mt, virtual_document: O, …}Proxy
[Warning] Editor not found in editor_to_source_line map (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] Could not retrieve current context – Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, …} (501.d0dee3f7a514bb00ee98.js, line 1)
Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, adapter: mt, virtual_document: O, …}Proxy
[Warning] Editor not found in editor_to_source_line map (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] Could not retrieve current context – Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, …} (501.d0dee3f7a514bb00ee98.js, line 1)
Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, adapter: mt, virtual_document: O, …}Proxy
[Warning] Editor not found in editor_to_source_line map (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] Could not retrieve current context – Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, …} (501.d0dee3f7a514bb00ee98.js, line 1)
Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, adapter: mt, virtual_document: O, …}Proxy
[Warning] Editor not found in editor_to_source_line map (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] Could not retrieve current context – Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, …} (501.d0dee3f7a514bb00ee98.js, line 1)
Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, adapter: mt, virtual_document: O, …}Proxy
[Warning] Editor not found in editor_to_source_line map (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] Could not retrieve current context – Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, …} (501.d0dee3f7a514bb00ee98.js, line 1)
Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, adapter: mt, virtual_document: O, …}Proxy
[Warning] Editor not found in editor_to_source_line map (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] Could not retrieve current context – Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, …} (501.d0dee3f7a514bb00ee98.js, line 1)
Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, adapter: mt, virtual_document: O, …}Proxy
[Warning] Editor not found in editor_to_source_line map (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] Could not retrieve current context – Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, …} (501.d0dee3f7a514bb00ee98.js, line 1)
Proxy {editor_name: "CodeMirrorEditor", isDisposed: false, _event_wrappers: Map, adapter: mt, virtual_document: O, …}Proxy
[Warning] LSP: Connect timed out for sdtm (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] TypeError: undefined is not an object (evaluating 'this.connection.on') — 501.d0dee3f7a514bb00ee98.js:1:42734
[Warning] Editor not found in editor_to_source_line map (501.d0dee3f7a514bb00ee98.js, line 1)
[Error] Unhandled Promise Rejection: TypeError: null is not an object (evaluating 'e.line')
	get_editor_at_source_line (501.d0dee3f7a514bb00ee98.js:1:29917)
	get_editor_at_root_line (501.d0dee3f7a514bb00ee98.js:1:105542)
	getTokenAt (501.d0dee3f7a514bb00ee98.js:1:105653)
	get_token_at (501.d0dee3f7a514bb00ee98.js:1:103115)
	(anonymous function) (501.d0dee3f7a514bb00ee98.js:1:81753)
	asyncFunctionResume
	(anonymous function)
	promiseReactionJobWithoutPromise
[Log] LSP: Skipping document update signal: connection not ready (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] TypeError: undefined is not an object (evaluating 'this.connection.getDocumentHighlights') — 501.d0dee3f7a514bb00ee98.js:1:81260
[Log] LSP: Skipping document update signal: connection not ready (501.d0dee3f7a514bb00ee98.js, line 1)
[Warning] TypeError: undefined is not an object (evaluating 'this.connection.getDocumentHighlights') — 501.d0dee3f7a514bb00ee98.js:1:81260
[Warning] TypeError: undefined is not an object (evaluating 'this.connection.getDocumentHighlights')
http://localhost:8888/lab/extensions/
asyncFunctionResume
http://localhost:8888/lab/extensions/
http://localhost:8888/lab/extensions/
http://localhost:8888/lab/extensions/
Promise
l — 770.4e19279c417629a6a0e2.js:1:750
http://localhost:8888/lab/extensions/
l — 770.4e19279c417629a6a0e2.js:1:5032

krassowski · 2021-01-26T14:42:14Z

This is good (as in: useful), but I will need to see the log of actual responses returned from your server to be able to understand what is going on.
Also, could you please show the result of jupyter server extension list and jupyter labextension list?

mike-vogel · 2021-01-26T16:23:16Z

Server log attached. Output of commands below.

jupyterlab-debug-log.txt

$ jupyter serverextension list
config dir: /Users/mike.vogel/.jupyter
jupyter_lsp enabled
- Validating...
Error loading server extension jupyter_lsp
X is jupyter_lsp importable?
config dir: /Users/mike.vogel/robotlab/etc/jupyter
jupyterlab enabled
- Validating...
jupyterlab 1.1.4 OK

$ jupyter labextension list
JupyterLab v1.1.4
Known labextensions:
app dir: /Users/mike.vogel/robotlab/share/jupyter/lab
verdant-log v1.0.0 enabled OK

krassowski · 2021-01-26T16:28:04Z

JupyterLab v1.1.4

Ouch, you are running a very old JupyterLab. We no longer support JupyterLab 1.x.
JupyterLab 2.2 is the latest version I could help you with. JupyterLab 3.0 is preferable.

This is because the extension has progressed significantly since the times of JupyterLab 1.x, we fixed countless bugs and the extension system has changed significantly so backporting is not practical. We are just a small group of volunteers doing this in our free time and cannot support versions two major releases back.

You should create a new virtual environment and install JupyterLab 3.0 there along with the latest extension version.

krassowski · 2021-01-26T16:32:53Z

Though your logs seem to be using ServerApp which is indicative of JupyterLab 3.x. I do not understand the discrepancy. Could it be that you run the jupyter serverextension list and jupyter labextension list from a different environment?

Also please note that it is jupyter server extension list for JupyterLab 3.x

mike-vogel · 2021-01-26T16:34:34Z

Something crazy must have happened in my environment or where I ran those commands. I was running 3. I'll check and run again.

mike-vogel · 2021-01-26T16:36:42Z

Sorry, ran the commands in the wrong window and didn't look at the output.

$ jupyter serverextension list
config dir: /Users/mike.vogel/.jupyter
jupyter_lsp enabled
- Validating...
jupyter_lsp 1.1.0 OK
config dir: /Users/mike.vogel/robotlab/envs/jupyterlab-lsp/etc/jupyter
jupyterlab enabled
- Validating...
jupyterlab 3.0.5 OK

$ jupyter labextension list
JupyterLab v3.0.5
/Users/mike.vogel/robotlab/envs/jupyterlab-lsp/share/jupyter/labextensions
@krassowski/jupyterlab-lsp v3.0.0 enabled OK (python, jupyterlab_lsp)

krassowski · 2021-01-26T16:44:42Z

Please do try upgrading to v3.2 (lab extension) and 1.1.1 (server extension): https://github.com/krassowski/jupyterlab-lsp/releases/tag/v3.2.0
The output you showed is indicative of a bugs fixed in 3.1 and 3.2.

Also, please provide messages sent by your LSP server (sdtm) as this is not included in JupyterLab debug output. You may need to use some logging mechanism to get those (e.g. print to a file or to the standard error stream) as the standard output is actualy being directed to our extension. I see that we should probably include an option to show what the LSP servers provide us with in the debug mode..

mike-vogel · 2021-01-26T20:48:57Z

Upgraded to v3.2

$ jupyter serverextension list
config dir: /Users/mike.vogel/.jupyter
jupyter_lsp enabled
- Validating...
jupyter_lsp 1.1.1 OK
config dir: /Users/mike.vogel/robotlab/envs/jupyterlab-lsp/etc/jupyter
jupyterlab enabled
- Validating...
jupyterlab 3.0.5 OK

$ jupyter labextension list
JupyterLab v3.0.5
/Users/mike.vogel/robotlab/envs/jupyterlab-lsp/share/jupyter/labextensions
@krassowski/jupyterlab-lsp v3.2.0 enabled OK (python, jupyterlab_lsp)

jupyterlab-lsp-server-log.txt
Console-log.txt

I switched to using a copy of the pygls example app at: https://github.com/openlawlibrary/pygls/blob/master/examples/json-extension/server/server.py. I cut out everything that looked non-essential and uploaded it as 'core copy.py.txt'.
core copy.py.txt

This is the output logged by the above LSP program:
pygls.log

mike-vogel · 2021-01-26T21:49:31Z

Forgot to say that it is still not working but is failing for new reasons.

krassowski · 2021-01-26T22:55:53Z

Let's break down the console logs first. In the beginning we see that your notebooks gets opened and kernel gets connected properly:

[Log] LSP.WidgetAdapter: – "Untitled.ipynb" – "ready for connection" (571.fcec033f8e8059d1222c.js, line 1)

and the connection going to your language server gets established (connected but not initialized yet):

[Log] LSP.WidgetAdapter: – "will connect using language: sdtm" (571.fcec033f8e8059d1222c.js, line 1)
[Log] LSP: – "connection requested" – {virtual_document: N, language: "sdtm", document_path: "Untitled.ipynb"} (571.fcec033f8e8059d1222c.js, line 1)
{virtual_document: N, language: "sdtm", document_path: "Untitled.ipynb"}Object
[Log] LSP: – "Connection Socket" – {virtual_document: N, language: "sdtm", document_path: "Untitled.ipynb"} (571.fcec033f8e8059d1222c.js, line 1)
{virtual_document: N, language: "sdtm", document_path: "Untitled.ipynb"}Object

You can see that the connection has not been initialized yet based on this message:

[Log] LSP.WidgetAdapter: – "Skipping document update signal: connection not ready" (571.fcec033f8e8059d1222c.js, line 1, x9)

And on all the Editor not found in editor_to_source_line warnings.

You should wait for the statusbar indicator to show "Fully initialized".
If it is not "Fully initialized" it means that there is an issue in the LSP server (pygls) initialisation; it may be just slow, or it may not implement the initialization shakedown properly.

You then attempt to fetch completions for x, (but the LSP server has not initialized yet!):

[Debug] LSP.CompletionLab: – "Fetching" (571.fcec033f8e8059d1222c.js, line 1)
[Debug] LSP.CompletionLab: – "Token:" (3) (571.fcec033f8e8059d1222c.js, line 1)
{offset: 0, value: "x,", type: undefined}
{ch: 0, line: 3}
{ch: 2, line: 3}

The kernel reply arrives (it is empty - matches: []) and gets transformed properly, however it is empty (items: []):

[Debug] LSP.CompletionLab: – "Transforming" (571.fcec033f8e8059d1222c.js, line 1)
[Debug] LSP.CompletionLab: – "Transformed" (571.fcec033f8e8059d1222c.js, line 1)
[Log] LSP.CompletionLab: – "Transforming kernel reply:" – {start: 2, end: 2, matches: [], …} (571.fcec033f8e8059d1222c.js, line 1)
{start: 2, end: 2, matches: [], metadata: {}}Object
[Debug] LSP.CompletionLab: – "Merging completions:" (2) (571.fcec033f8e8059d1222c.js, line 1)
{start: 2, end: 2, items: []}
{start: 2, end: 2, items: []}

You do not get reply from your language server because it is still not ready.
The initial 30 seconds have passed but your language server has not confirmed initialization, so we switch into a passive mode checking for initialization update less frequently for another 5 minutes:

[Warning] LSP: – "Connection to Untitled.ipynb timed out after 30 seconds, will continue retrying for another 5 minutes" (571.fcec033f8e8059d1222c.js, line 1)

And as this is the end of the logs, I presume that your language server has never initialized.

krassowski · 2021-01-26T23:08:47Z

I think that your python code is ok. I wonder if you may have some buffering which prevents the code initialization request from being received. I suspect that you may need to:

make sure your run-sdtm-lsp.sh connects streams properly (I do not know what is inside)
set -u to your Python call or set PYTHONUNBUFFERED environment variable to ensure that the messages are not being lost in the Python buffer (see https://docs.python.org/3/using/cmdline.html#cmdoption-u). We do that for jedi-language-server which is pygls based.

mike-vogel · 2021-01-27T02:07:11Z

You were right about the messages being lost. They were completely lost because I was starting the pygls language server in TCP instead of STDIO so I was in the wrong communication mode. The pygls modes are documented at https://pygls.readthedocs.io/en/latest/pages/advanced_usage.html?highlight=logging#connections. It was a mix of my lack of knowledge about the different modes and me thinking it automatically started in STDIO mode (a.k.a. user error). When I added an explicit call of server.start_io() my Language Server got to the 'Fully Initialized' status and completions started working.

The shell file I was running doesn't do anything but run python to execute my language server code. I tried the -u switch and it doesn't seem to change anything.

krassowski · 2021-01-27T02:25:40Z

Great to hear it was resolved!

I think that we need to improve the documentation on our side to note down all of this knowledge. Would you like to contribute a short note on what is required to make a new kernel/language server pair work with the lsp extension to our documentation? It would be a great addition and you may be a person who has done this most recently and have it in fresh memory. It could (but doesn't have to) include an example of steps needed to make a minimal pygls server work.

mike-vogel · 2021-01-27T12:44:17Z

I'd be happy to work on that documentation note. Where should I put it?

krassowski · 2021-01-27T14:32:41Z

Probably the best place would be a new section on "What is needed to connect a kernel with a language server" and an example on creating and configuring a new pygls-based language server below the scala example in https://github.com/krassowski/jupyterlab-lsp/blob/master/docs/Configuring.ipynb

We gathered some useful notes for new contributors in the Contributing section: it explains how to install required developemenmt dependencies, how to build the documentation locally (though you might skip this step and on PR our RTD integration will do that for you) and how to lint the code with scripts/lint.py.

krassowski mentioned this issue Nov 12, 2020

Metals (scala language server) integration with jupyterlab-lsp #407

Closed

krassowski added the documentation Improvements or additions to documentation label Jan 26, 2021

krassowski added this to the 3.3 milestone Jan 26, 2021

krassowski removed this from the 3.3 milestone Feb 14, 2021

This was referenced Apr 22, 2021

XText Language Servers Support #578

Open

Switch to MyST and MyST-{NB} with book-theme #581

Merged

krassowski closed this as completed in #581 Apr 25, 2021

j-vm mentioned this issue Jan 10, 2024

Feature request: Mimetype set in LanguageInfo InseeFrLab/Trevas-Jupyter#100

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Language server not starting for custom language type #384

Language server not starting for custom language type #384

mikesligo commented Oct 22, 2020

krassowski commented Oct 22, 2020

bollwyvl commented Oct 22, 2020 via email

bollwyvl commented Oct 22, 2020 via email

mikesligo commented Oct 22, 2020

bollwyvl commented Oct 22, 2020 via email

mikesligo commented Oct 22, 2020

mike-vogel commented Jan 25, 2021

krassowski commented Jan 25, 2021 •

edited

Loading

mike-vogel commented Jan 25, 2021

mike-vogel commented Jan 26, 2021

krassowski commented Jan 26, 2021

mike-vogel commented Jan 26, 2021 •

edited by krassowski

Loading

mike-vogel commented Jan 26, 2021 •

edited by krassowski

Loading

krassowski commented Jan 26, 2021 •

edited

Loading

mike-vogel commented Jan 26, 2021 •

edited

Loading

krassowski commented Jan 26, 2021

krassowski commented Jan 26, 2021 •

edited

Loading

mike-vogel commented Jan 26, 2021

mike-vogel commented Jan 26, 2021

krassowski commented Jan 26, 2021

mike-vogel commented Jan 26, 2021

mike-vogel commented Jan 26, 2021

krassowski commented Jan 26, 2021

krassowski commented Jan 26, 2021 •

edited

Loading

mike-vogel commented Jan 27, 2021 •

edited

Loading

krassowski commented Jan 27, 2021

mike-vogel commented Jan 27, 2021

krassowski commented Jan 27, 2021

Language server not starting for custom language type #384

Language server not starting for custom language type #384

Comments

mikesligo commented Oct 22, 2020

Description

Steps to reproduce

Findings

Logs

Enironment

krassowski commented Oct 22, 2020

bollwyvl commented Oct 22, 2020 via email

bollwyvl commented Oct 22, 2020 via email

mikesligo commented Oct 22, 2020

bollwyvl commented Oct 22, 2020 via email

mikesligo commented Oct 22, 2020

mike-vogel commented Jan 25, 2021

krassowski commented Jan 25, 2021 • edited Loading

mike-vogel commented Jan 25, 2021

mike-vogel commented Jan 26, 2021

krassowski commented Jan 26, 2021

mike-vogel commented Jan 26, 2021 • edited by krassowski Loading

mike-vogel commented Jan 26, 2021 • edited by krassowski Loading

krassowski commented Jan 26, 2021 • edited Loading

mike-vogel commented Jan 26, 2021 • edited Loading

krassowski commented Jan 26, 2021

krassowski commented Jan 26, 2021 • edited Loading

mike-vogel commented Jan 26, 2021

mike-vogel commented Jan 26, 2021

krassowski commented Jan 26, 2021

mike-vogel commented Jan 26, 2021

mike-vogel commented Jan 26, 2021

krassowski commented Jan 26, 2021

krassowski commented Jan 26, 2021 • edited Loading

mike-vogel commented Jan 27, 2021 • edited Loading

krassowski commented Jan 27, 2021

mike-vogel commented Jan 27, 2021

krassowski commented Jan 27, 2021

krassowski commented Jan 25, 2021 •

edited

Loading

mike-vogel commented Jan 26, 2021 •

edited by krassowski

Loading

mike-vogel commented Jan 26, 2021 •

edited by krassowski

Loading

krassowski commented Jan 26, 2021 •

edited

Loading

mike-vogel commented Jan 26, 2021 •

edited

Loading

krassowski commented Jan 26, 2021 •

edited

Loading

krassowski commented Jan 26, 2021 •

edited

Loading

mike-vogel commented Jan 27, 2021 •

edited

Loading