Decide on a single vim plugin infrastructure

[timokau]

Issue description

When managing vim plugins through nix, we currently support multiple "backends":

• vim's native "packages" (which is relatively new and hasn't been around when the nix vim infrastructure was created)
• @MarcWeber's VAM, which I think was partly created with nix in mind
• vim-plug which is one of the most popular plugin managers
• pathogen which is also very popular, but much more simple (more on the vim-packages level)

As previously discussed in #52685 (comment), it is not clear what the advantage of multiple backends is. Since nix does most of the heavy lifting Nix does most of the heavy lifting:

• managing dependencies
• generating a vimrc snippet to load them
• keeping them up to date
• generating documentation and remote plugin manifests

The backend is only really needed to manage the "runtimepath" (i.e. actually load the plugins). The native "packages" implementation seems like a perfect fit for that.

Maintaining multiple backends makes development much harder. It takes a while to understand vim-utils.nix. It is not trivial to test changes on all provided backends. Examples for issues caused by the fragmentation (most by me):

• Fix vim-plugin dependencies #52767
• vim-addon-nix seems to no longer work with neovim #54585
• nixos-unstable broken by missing pname attribute in vim-plugins #53112
• vimPlugins.fzf-vim: automatically add fzfWrapper #52685 (comment)

I have the impression that most people use either VAM or vim-packages. Because it is the "official" package manager and minimal, I think it makes sense to focus on vim-packages.
If we decide to drop other backends, end-users will have to migrate. That is not hard, but involves manual work. It is unclear if the trade-off "nixpkgs development" and "one-time user action needed" is worth it. That is what this issue is about.

Configuration examples:

VAM:

neovim.override {
  configure.vam = {
    vimrcConfig.vam.knownPlugins = pkgs.vimPlugins; # optional
    vimrcConfig.vam.pluginDictionaries = [
      # load always
      { name = "youcompleteme"; }
      { names = ["youcompleteme" "foo"]; }

      # only load when opening a .php file
      { name = "phpCompletion"; ft_regex = "^php\$"; }
      { name = "phpCompletion"; filename_regex = "^.php\$"; }

      # provide plugin which can be loaded manually:
      { name = "phpCompletion"; tag = "lazy"; }
    ];
  };
}

vim-packages

neovim.override {
  configure.packages.myVimPackage = with pkgs.vimPlugins; {
    # loaded on launch
    start = [ youcompleteme fugitive ];
    # manually loadable by calling `:packadd $plugin-name` (or by autocommand)
    opt = [ phpCompletion elm-vim ];
  };
}

CC @Mic92 @Ma27 @MarcWeber @rvolosatovs

[timokau]

Also CC @teto

[rvolosatovs]

The only reason why I added https://github.com/junegunn/vim-plug as a backend was that I was unable to make https://github.com/fatih/vim-go work in neovim via neither the native vim packages nor VAM when migrating my config to Nix(see #46751 (comment)). vim-plug just worked.
That's why I'm forced to do this: https://github.com/rvolosatovs/infrastructure/blob/2f396c7f41c0b8e16aedcce417bec5017b76c4d7/nixpkgs/neovim/default.nix#L10-L48

[timokau]

What do you mean with "unable to make [it] work"? Did you re-try recently? I don't use go so I can't test properly, but I built a neovim with vim-go using packages:

nix-build -E 'with (import ./. {}); neovim.override { configure = { packages.pkg.start = [ vimPlugins.vim-go ]; }; }'

And was able to at least access the help pages of the commands vim-go adds.

[CrazedProgrammer]

My preference goes to vim-packages because of its minimalism and ability to manually lazy-load packages.

[rvolosatovs]

@timokau I just tried it and I had the same issue - everything builds perfectly and help and commands work, however going to definition via ^] does not. It does via :GoDef though.

1. Build Neovim using the command you provided in Decide on a single vim plugin infrastructure #56338 (comment)

2. ./result/bin/nvim test.go and paste following contents:

package main

import (
	"fmt"
)

func main() {
	fmt.Println()
}

3. Put cursor on fmt.Println line at first character of fmt - f
4. Press ^]
5. Observe following output below:

E433: No tags file
E426: tag not found: fmt
Press ENTER or type command to continue

6. From the same position do :GoDef
7. Cursor is moved above to "fmt" line in import() block. Observe following output below:

vim-go: [searching declaration] SUCCESS 

Now repeat the steps 1-4, but build nvim via:

nix-build -E 'with (import ./. {}); neovim.override { configure = { plug.plugins = [ vimPlugins.vim-go ]; }; }'

Now ^] works properly(same as :GoDef) and cursor is moved to "fmt"
Observe following output below:

vim-go: [searching declaration] SUCCESS

No idea why this happens - I tried to investigate what caused this difference in behavior, I assumed that vim-plug does some kind of "magic", since the native plugin manager failed to produce this behavior but did not find anything funky in the source code of it.

[Ma27]

I also think that it might be easier to only support one backend. I'm i.e. hesitant to merge PRs like #52685 which actually fixes problems, but I'm absolutely unsure about the fallout and possible follow-up issues that are even harder to fix.

I'd propose to gather all issues with vim-packages (such as the vim-go issues), fix them and then start a deprecation process for all of the other packages.

[MarcWeber]

Let's clear things up. All solutions basically use vim plugins. A vim plugin is directory added to $VIMRUNTIME to be loaded.
Plugin managers added activation at runtime partially emulating Vim's behaviour (loading the plugin/*.vim files). So there might be differences.
Plugin managers added ways to specify hooks. And here the managers differ. Some of this is taken part of by nix eventually.
The managers have different behaviours such as Pathogen always running helptags on startups which VAM does not.
Is it really getting simple if you drop targets? Because the installation phase should be the same.
And Pathogen / VAM etc already worked an was tested by me long time ago.
I personally use :VAMActivate name if I forgot a plugin sometimes and it just works.
So I cannot switch but I can keep private patches.

But is this a nix related problem? I tried getting in touch with authors, but collaboration seemed hard also because VAM was writetn first, then Shougos solution popped up 'not knowing about mine' then the many plug* like more simple solutions were written for fun or for more features such as introducing parallel fetching. VAM was VimL only to be as simple as possible. But it allows the a file with JSON lines to be configured so that parallel building can be implemented but this never happened.

The hooks (such as compile a .c file) do not depend on the manager eventually.

VAM has some plug like emulation layer, but does not support the many hooks the way the plug does.

The story doesn't end here there is Vim and nvim. And there are many language-server implementations.
The problem is the Vim community is divided and that's also caused by vim.sf.net not having a guide or best practice recommendation.

Do what you want and what works for you. VAM is happy if you give a pair of name / store paths to plugins so should be trivial. VAM also introduced vim-pi trying to give each plugin a name so that dependencies can be specified (without version) because it was most simple to get the job done assuming that most people use latest versions anyway. VAM was inspired by NIX that a configuration should somewhat determine behavior. But its impossible because it can depend on loading order of plugins if you load plugins lazily.

Truly fixing this is beyond this discussion, because a plugin must have influence on Vim (eg with ruby/python). And then what to do if you have two plugins one using Py2 the other Py3 ? Vim plugins have history. There will always be some problems to be fixed which is why I introduced the possibilities to give vim a name such as vim-with-py-and-plugins so that you can put multiple variations in path.

[Mic92]

If we drop 2 out of 4 plugin implementations, we can half the test matrix. For transition period pathogen and the plug configuration can be mapped onto the native backend + generating some deprecation warnings for a NixOS release. This way migrating should be relative straight forward and future changes to vim-utils easier to test.

[timokau]

It's possible that the vim-go issue is related to neovim/neovim#9390, which would be a blocker either way.

[teto]

This is of interest to me since I am trying to make it easier to generate vim configurations (typically right now I generate a neovim wrapper for each nix-shell merging my global basic neovim configuration with project specific configurations #55635).
I am mostly familiar with vim-plug, but seems like many features of plugin managers are useless in our declarative context:

• deal with fetching sources (branch/source pinning): nix does it
• run some hook for compilation/ updating manifest: nix does it
• lazy loading : can lead to problems if handled by the plugin manager, it should be handled by plugins

so the native backend should be enough unless we want the vim config generator to be used outside of (running :PlugUpdate and all).

[MarcWeber]

I think its wrong to 'settle on one' implementation. Rather we should focus on what we want.

What is VAM?
vim-pi package index with some hooks giving many vim plugins a name
vim-addon-manager allowing to specify folder locations of plugins as already done in the nix implementation but being able to fetch missing plugins as required using viml (which is serial not parallel ATM). It also supports lazy loading / activation as needed fetching dependencies.
dependencies are declared in an addon-info.json file just by name (JSON file).
the idea was to add hooks in an implementation independent way to the .json file.
But in real world it doesn't work because Windows is different from Linux from cygwin etc.
Lazy loading is important because some plugins have 5K lines of .viml and have been written before autoload was invented they sum up making vim startup slowly. its a feature you might want to have in some way.

So VAM allows to specify your plugins in a .txt file, then run viml code and have nix code which can be copy pasted to have your setup working almost sharing your setup unless special dependencies (ocaml) are required.

Despite it has been working even with .zip for Windows: http://vam.mawercer.de/
not many people picked it up.

It fails at 'plug related' hook configurations which is not a flaw, just a different configuration.

The problem we're talking about is not related to NIx. Hundreds of vim repositories have 4+ instructions about how to install a plugin .. Its a Vim community issue as well in some cases touching package manager issues (such as compiling ruby or ocaml related code).

If plugins are passed and namings are taken care of so that dependencies are found it would get the job done.

I don't say you have to use it. Just putting the question what features you care about.

Thus documentation about VAM and how to make ti work is easy: pass {'name': nix-path} like and be done.

But there are many more things to be aware about. Eg vim-addon-nix implemented 1) checking (like syntastic) but I don't like syntastic because it ran code despite its name 2) goto import feature 3) some syntax for .nix files.

But today that the client-server protocol exists https://langserver.org/ maybe all but syntax should be moved in such implementation so that it can be shared for all editors.

To move forward I think it makes sense to create a list of 'features we want' and then think about 'how to implement them'. Eg teto says:

lazy loading : can lead to problems if handled by the plugin manager, it should be handled by plugins<<<
WTF? A plugin manager is a plugin. But I agree there could be shared implementation. But lazy loading implies 'fetching sources' and then you get into the trouble how parallel should it be done and how many dependencies do you want (outside of nix). Thus should there be a 'nix only package manager'.

Just activating a path can already be done by VAM> There is function you can define mapping a plugin name to a path.

There are many 'universes' eg eclipse. Packaging eclipse plugins eventually doesn't make sense.
Fedora years back had a 5K script file to compile a very old version of Eclipse.
Downloading ecilpse just works.

So I want to draw attention that part of this problem is lack of 'commonly used dependency format'.
And this does not only apply to VIm, it applies to all packages. If there existed a hackage for everything all linux distros could benefit. But again -> disruptive things happen such as 'vscode' just working reasonably well for many cases.

I care about

• lazy loading to have fast startup time but also because I load some code per project only using local .vimrc
• manage dependencies in a simple way because I have much shared code for my plugins

What does 'testing' mean? When does a plugin work? Not many plugins ship with much testing code.

I once added testing code to 3 different backup solutions and it got removed again. So eventually we should just put enough effort to keep things working.

[Mic92]

Specifying dependencies via addon-info.json is nice in theory however I don't see many plugins actually using this concept. Also since the most popular package managers are not using information. It is somewhat orthogonal since dependency resolution in the declarative nix case is independent from the the underlying plugin manager. We only have some simple test cases that simply create vim wrappers for the different backends. As those need to be tested interactively it is also in the interest to keep the complexity low here.

[teto]

exactly dependency resolution is not really a problem since in practice there are no dependencies between vim plugins (except fugitive+rhubarb, ingosript library and quickfix.vim etc) so it can be done in nix directly. With neovim (which embeds a lua interpreter) we might see more lua-based plugins and a need to describe more dependencies but they might end up using luarocks mechanism anyway.

[LovingMelody]

I do believe that the greater vim community has been open to a single package manager for quite some time. I have been using vim-plug as it has made lazy loading easy for me. Some of the conditions that I would define can be safely assumed by Nix such as having python3-pynvim, there are still others that I cannot handle with just Nix:
vim-tmux - load if the $TMUX environment variable is set
codi.vim - load on :Codi command
While I am sure these kinds of can be done in pure vim-script and I could just the built-in plugin manager, the appeal of using some popular plugins such as vim-sensible is to avoid re-inventing the wheel. I believe a similar position should be taken with plugin managers. If the vim-plug tool on Nix supports these, please add it to the wiki.

[rvolosatovs]

@Fuzen-py vim-plug implementation just passes absolute paths of plugins, so it basically does this: https://github.com/junegunn/vim-plug/blob/93b702512d07bcad4c0d47913db587febe8cc054/doc/plug.txt#L178-L179
There's no support for parameters (e.g. for or on), but it's not too hard to add that

[timokau]

You can do a lot with nix, so something as powerful as vim-plug is not really needed for plugin management with nix.

[MarcWeber]

To be honest 'the one solution rules everything' does not work for Vim I think. Because there is also vundle, neobundle and many more. Trivial plugins work with all anyway. Maybe its also ok to accept that you cannot be perfect - sry.
VAM lost against vundle in popularity.
Also Vim (neovim) raise questions such as what plugin language to use.
There are more questions such as 'what plugin systems to push providing completions'
.. so many more questions. It depends on community or authors what they knew best.

So maybe rather than focusing one plugin manager we could also focus on standards.
Eg only add plugins which work with all :) Because in the end plugins are let rtp+= ... and that's it (in theory). In reality you get conflicts to to loading order etc.

Also how to ship depending codes. Old and famous tlib etc etc etc... Saying there are no dependencies just doesn't match the reality. If you remove dependencies you'll end up somebody asking for it again.

Anyway: VAM has a small core. VAM can install additional plugins outside of nix later using :VAMActivate. Those are uniq features of the implementations. I also have to admit that I don't even know about status of maintainance of vim-pi neither do I have much time to spend on it because I think all the problems must be solved on different level but I don't have the resources to even think about doing it.

Also words like "I noticed that plugin dependencies simply did not work with the native vim plugin manager. It simply was not implemented, which was surprising." found here #52767 well its the default behavior of VAM. But VAM allows to activate plugins lazily to reduce startup time. Eg if you open .php only then load the php plugins (people found multiple levels / implementations) to do so. I don't say VAM is close to being best. eg it doesn't care about stub commands. I think we had a sample implementation.

Summing up: Most plugins installed as only plugin should work with most managers. Otherwise the plugin broken. Focusing on one manager I think is not going to fix much.

Accepting some breakage eg (#broken-with-vam) like comments might help maintaining the plugins.

I tried making a complete list and differences of plugins once:
http://vim-wiki.mawercer.de/wiki/topic/vim%20plugin%20managment.html
-> but ... you see it .. its pointless to some degree because there are too many and due to how the community is (not) managed this will continue to happen.

vim.org does not even have a main wiki page where you could talk about such.

Words like "If we drop 2 out of 4 plugin implementations, we can half the test matrix." see above just say "I don't understand what a Vim plugin is at all" .. because a vim plugin should be working with most managers. Period.

I think that starting up time today matters less than 5-10 years ago. I agree. So people using rtp manipulation (vinalla vim) might not even understand the needs of the past today.
But even 100ms for each git commit you do in your lifetime might sum up..

I started VAM. I wrote the initial nix code. I also assume that most don't know that Vim's core is broken. Eg viml code can be interrupted at any stage when you resize the window. I sent the proof to the vim or vim-dev mailiglist long ago. There is nothing you can do about it but swich to neovim. There are more 'race conditions like this' Vim still happens to work good enough in most everyday cases.

Thus only try to be 'good enough'. Many packages break regularly on updates. Why ? Because waiting for all updates might be the bigger problem because most users will not care about random games... This applies to vim plugins as well. What is good enough is debatable again.

[LovingMelody]

You can do a lot with nix, so something as powerful as vim-plug is not really needed for plugin management with nix.

I under stand this, but it is far more work than it takes to use vim-plug outside of nix. Isn't the entire point of using plugins to not re-invent the wheel? I'm sure vim-sensible is popular for a reason.

Because in the end plugins are let rtp+= ... and that's it (in theory).

I believe neovim is a bit more than this due rplugin. If a decision like this is going to be made, both neovim & vim should be supported.

[stale]

Thank you for your contributions.
This has been automatically marked as stale because it has had no activity for 180 days.
If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.
Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the
   related code and @ mention them in a comment.
2. Ask on the NixOS Discourse. 3. Ask on the #nixos channel on
   irc.freenode.net.

[cideM]

Still important to me.

I read through the most recent comments here and with regards to

To move forward I think it makes sense to create a list of 'features we want'

Wouldn't the goal of the Vim/Neovim infrastructure in Nix be simply to make the necessary runtimepath adjustments to work with the native package solution that both editors nowadays offer? I'd personally consider anything above that a cherry on the cake thing. For example, it's nice if a plugin manager handles generating helptags for you, but every feature that is added needs to be tested and supported and once something is there it's hard to remove.

This is of course just my personal opinion but I'd focus on delivering the minimum functionality to make most plugins work and by work I mean Vim/Neovim are aware of the plugin, either by loading it as a start plugin or by making it available through packadd as an optional plugin. It's probably obvious that I don't know the full scope of what modern plugin managers do but it looks like minpac for example isn't doing a whole lot.

I hope I don't come across as ignorant of all of the work put into the ecosystem here and all the posts written on this topic, I'm infinitely thankful for all the work that's been done on Vim/Neovim in Nix

[stale]

I marked this as stale due to inactivity. → More info

[teto]

I've deprecated pathogen in #154814 because it brings nothing that I know of compared to vim packaging system. I would like to remove the rest as well. Well-designed plugins should not need plugin manager customizations to shave off 2ms. More advanced frameworks can live in flakes as for instance https://github.com/vi-tality/neovitality