Discussion on auto-installing servers

[mjlbach]

What would you need from lspconfig to make installing servers automatic?

Two options:

• No lspconfig config enabled, you open a lua file, lsp-installer detects this, enables the lsp config "blessed" configuration (pyright for python, clangd for c, etc.) and installs the language server for you, then lspconfig "just works"
• Lspconfig config enabled, lsp-installer detects this, and automatically installs the language servers matching your configs.

Happy to discuss.

I personally find this to be an anti-pattern, but there's an extremely vocal segment of users who probably will never use the built-in client until there is a supported option for this workflow.

[williamboman]

Hey! Not sure I'm following what lspconfig enabled means, do you mean individual servers that users have called .setup() on?

My initial thought would be that I guess this could be solved via ftplugins, for example:

" autoload/lsp/autoinstall.vim
function lsp#autoinstall#setup(server_name)
    lua <<EOF
-- check if server is installed, if not install it
EOF
endfunc

" ftplugin/typescript.vim
call lsp#autoinstall#setup("tsserver")

" ftplugin/html.vim
call lsp#autoinstall#setup("html")

" ftplugin/terraform.vim
call lsp#autoinstall#setup("tflint")

" ...

This would require the user to only do the following in their config:

require("nvim-lsp-installer").on_server_ready(function (server)
  local opts = {...}
  server:setup(opts)
end)

But I guess this could also be abstracted away to turn it into a 100% no-code setup for users. But I'm starting to feel like that's out of scope for this plugin.

I personally find this to be an anti-pattern, but there's an extremely vocal segment of users who probably will never use the built-in client until there is a supported option for this workflow.

Yeah I'm not too keen on automatically installing servers upon opening buffers. I've been wanting to continue building on the UI in this plugin to make LSPs more accessible. But ultimately the choice to install one should always be initiated by the user.

So just one concrete example is that we could scan 1) all currently open buffers, and/or 2) v:oldfiles to build a list of most common filetypes and from that identify whether the user is missing an LSP, in which case we could promote these in the UI window.

Btw, by UI window I mean :LspInstallInfo:

[mjlbach]

Hey! Not sure I'm following what lspconfig enabled means, do you mean individual servers that users have called .setup() on?

Yes, kinda like ensure_installed from treesitter.

Yeah I'm not too keen on automatically installing servers upon opening buffers. I've been wanting to continue building on the UI in this plugin to make LSPs more accessible. But ultimately the choice to install one should always be initiated by the user.

I am conflicted. I 100% agree with you on this, but I get a lot of heat from reddit/users that this is not intuitive. I would like to accommodate them somehow. Maybe a contrib module that could optionally be required?

[kylo252]

• No lspconfig config enabled, you open a lua file, lsp-installer detects this, enables the lsp config "blessed" configuration (pyright for python, clangd for c, etc.) and installs the language server for you, then lspconfig "just works"

That's basically how it's done in LunarVim, with the caveat of having to use that weird hack for ftplugin. Related: neovim/nvim-lspconfig#970.

[mjlbach]

Just to give a preview of what I am thinking (sorry, this is kinda a brain dump):

• In a few months, once the core client is "feature complete", I'm going to turn my attention onto the front ends of the client. It's always been my intention that lspconfig is a fallback monorepo (like filetype plugins are) with per-language extension like nvim-jdtls, nvim-metals, etc. serving as more bespoke solutions accommodating off-protocol weirdness.

• We will always have the "low level" client API in core, but we will create more high level utilities in core for things like project management, per-server configuration (per project), and move all of the attach logic from lspconfig to core. For building a plugin, you will not need lspconfig (you don't really as it stands, nvim-jdtls and nvim-metals don't use it), but you can assume functionality currently in lspconfig via more core APIs.

• As part of a meta-plugin, I've been talking with @wbthomason about exposing additional functionality in packer to allow plugins to manage other plugins. This will allow for a "coc-like" lsp plugin that does not involve implementing its own plugin manager.

• I created the nvim-lsp organization. I am open to creating a coc-like frontend which provides a declarative and imperative option for installing extensions/servers.

• As part of this plugin, I would like the minute you open a java file the plugin to be able to prompt the user "would you like to install nvim-jdtls and jdtls?".

• This plugin will also handle integrating snippets/autocompletion/other goodness until those functionalities (if they exist are moved into core).

I'm going to basically lay the groundwork for everything but running/managing this new "meta-plugin". I think until we basically get to 0 config, there are always going to be angry users.

[kylo252]

We will always have the "low level" client API in core, but we will create more high level utilities in core for things like project management.

I know we've talked briefly about the security aspects of this before, but it would this would really be a killer feature that will make life easier a bit easier for a lot of people.

per-server configuration (per project)

Maybe nlsp-settings is worth a serious consideration? if nothing else, just the settings injection model.

• I created the nvim-lsp organization. I am open to creating a coc-like frontend which provides a declarative and imperative option for installing extensions/servers.
• As part of this plugin, I would like the minute you open a java file the plugin to be able to prompt the user "would you like to install nvim-jdtls and jdtls?".

You can take a look at how LunarVim is adding, mostly simple, glow code for this to work. Auto-installtion of servers is already possible, demo from the current rolling branch: https://asciinema.org/a/LjPTI4ZlnhngMfztpE3nQlW3p

I quickly put together the changes for the installation prompt to make what's happening more obvious. Not sure why there's a duplicate question since this is my first time ever trying confirm 😅. Demo: https://asciinema.org/a/OGQ5ATOkh7gwtzaCllQNz4y8F

I'm going to basically lay the groundwork for everything but running/managing this new "meta-plugin". I think until we basically get to 0 config, there are always going to be angry users.

Wait until someones asks for 0 configs for 3 servers at a time, while running eslint_d as both a formatter and linter, and letting prettier run after it but only on typescriptreact files and not javascript.... 😢

[mjlbach]

Maybe nlsp-settings is worth a serious consideration? if nothing else, just the settings injection model... You can take a look at how LunarVim is adding, mostly simple, glow code for this to work.

I don't really need inspiration, I'm aware how the implementation would look (and they aren't really compatible with the future of lspconfig without adaption). This issue is about coordinating with a plugin about what functionality it should/should not offer, and how that fits into the grander 0 config path for servers.

[williamboman]

I am conflicted. I 100% agree with you on this, but I get a lot of heat from reddit/users that this is not intuitive.

I'm not very involved as far as user feedback goes, but I wonder how much of it is really about automatic installations of LSP servers? If you were to ask me, and I think you already touch on all of this in your other comment, much of the perceived friction stems from the lack of "in-editor", no-code, utilities for managing all-things-LSP. I think an important insight to help guide direction would be to learn what people generally consider to be a part of their core editor configuration (relative vs non-relative line numbers, status line, etc.) - which they don't mind writing version controlled "code" for - and what is considered an add-on which is to be managed through other interfaces. I think most users would put LSP extensions under the latter category.

I would like to accommodate them somehow. Maybe a contrib module that could optionally be required?

Yeah I had this thought too. I think for starters that'd be my preferred approach. As mentioned earlier, this should already be really easy to implement.

As for this plugin, I'm starting to feel like it's somewhat feature complete. There are few more things that I'd consider universally needed features (like viewing and updating outdated servers), but for the most part I think it's pretty much complete as far as installing LSP servers goes. Then there's the tasks of continuously adding more servers and ensuring continuous cross-platform stability, but that's maintenance.

With that said, I'd like to continue exploring what can be done in this plugin to make LSPs more accessible. This will mostly be in the UI space, where we have a custom render engine that makes prototyping and iterating really easy (if you ask me). I do find it a bit discouraging to experiment as it's very difficult to measure things as a plugin author (I know the amount of git clones and stars, that's pretty much it). I would love to have some telemetry in place to help understand actual user behavior, but that's generally a big no-no.

Perhaps going a bit off-topic now, but I've also been playing with the idea of creating a registry for LSP servers that would offer an as standardized API as possible for clients to gather metadata (releases, target languages, etc.) but more importantly pre-packaged distributions of the servers. I think such a registry would not only benefit neovim LSP users, which hopefully would incentivize server maintainers to ensure they interop well with it. I think long term that's sort of the direction things have to go as it'd allow clients to be dumber, and server maintainers to be accountable for interop with the registry. Not only that but I think it'd put spec compliance, correctness of implementations, and transparency in the forefront, benefiting all editors. I feel like doing all these things from a neovim core LSP client standpoint will always be an uphill battle.

[williamboman]

Yikes that turned out longer than I intended. I'd say the tl;dr of the original topic is: Totally doable already, but I'd prefer not to put it in this plugin directly but instead through some optional extension/contrib plugin. Reasons being I don't necessarily view it as core functionality (although it could eventually become one), as well as separating these would allow us to measure usage more easily.

[mjlbach]

I'm not very involved as far as user feedback goes, but I wonder how much of it is really about automatic installations of LSP servers?

Honestly, quite a bit. This is the number one thing I receive complaints about by far. The second and third are the lack of built-in autocompletion and snippets (which like, is a bigger thing than LSP and Prs for the latter at least are welcome to core as far as I know). The fourth being no default keybindings (although, this is the same as coc.nvim). The last thing is project local configuration/scratch servers which is WIP pending lua auto commands.

As for this plugin, I'm starting to feel like it's somewhat feature complete.

Totally understandable, I wanted to gauge your temperature on this. I think we could abstract away from having the "core" functionality in this plugin pending the ability to have a plugin manage other plugins with packer. This is essentially how coc-nvim works.

[williamboman]

• As part of this plugin, I would like the minute you open a java file the plugin to be able to prompt the user "would you like to install nvim-jdtls and jdtls?".
• This plugin will also handle integrating snippets/autocompletion/other goodness until those functionalities (if they exist are moved into core).

When you say "this plugin". Are you referring to the meta-plugin or literally this plugin 😅?

[mjlbach]

When you say "this plugin". Are you referring to the meta-plugin or literally this plugin 😅?

meta-plugin :)

[williamboman]

Cross-posting a comment that I felt was relevant to this discussion, especially in terms of the "meta-plugin" and cross-plugin interoperability: #192 (comment)

[mjlbach]

I think this could potentially be revived as an option with the new setup hook, such that calling setup for a given language server automatically ensures it's installed, sorta like an on-demand ensure_installed. I don't have strong feelings on this, so feel free to close.

[williamboman]

Yeah I thought of that too, I think it'd make sense. Maybe it could allow something along the lines of

ensure_installed = true
-- or as a string for less ambiguity
ensure_installed = 'on_demand' -- 'on_setup', 'configured', ...?

[mjlbach]

Yeah, it could be multi-typed, take a table for ensuring certain servers are installed (which can also be used in a loop to setup all of lspconfig) or a boolean which defaults to false that installs servers called by setup. I usually like having separate keys to keep the types consistent, for which perhaps automatic and ensure_installed could coexist (you can set up a server dynamically still even with lspconfig).

If you split this out into a separate companion plugin these could live in that plugin to avoid polluting the core installer plugin with things overfit to lspconfig.

[williamboman]

What about #638? I'm wondering if it also should make sure to uninstall any installed servers that haven't been set up..

[mjlbach]

Totally your call, I usually go for separate keys b/c I think the two aren't necessarily in conflict with each other. GC'ing would be complicated with the auto-install option.

[williamboman]

After some additional feedback from @MurdeRM3L0DY I went with introducing a new setting instead, keeping ensure_installed as-is. Reasons for this is to make sure ensure_installed can be used to target servers not managed via lspconfig (nvim-jdtls for example). Will however need to sleep on the naming of the new setting (automatic_installation)

[williamboman]

Added with the following variants:

-- 1. Will always make sure rust_analyzer is installed, no matter if you set the server up or not
require("nvim-lsp-installer").setup {
  ensure_installed = { "rust_analyzer" } 
}

-- 2. All servers set up via lspconfig will automatically be installed
require("nvim-lsp-installer").setup {
  automatic_installation = true,
}

-- 3. All servers set up via lspconfig, except "rust_analyzer", will automatically be installed
require("nvim-lsp-installer").setup {
  automatic_installation = { exclude = { "rust_analyzer" } },
}