async pickers take 3

[oberblastmeister]

note: this is only implemented for oneshot jobfinders. If you want to test this, use find_files on your home directory.
note2: this requires the async await pr

implementation

nvim_buf_attach

First we use channels to get the buf_lines_event. Using channels means that we can set the polling rate which I have set to 50ms. This means that we poll for a buf_lines_event every 50ms, and only get the last event. What this means is that if multiple events occur within 50ms, we throw them out and ignore them. This is good because this makes it so that we don't sort unnecesarily and this also means that we are always soring the latest event.

non blocking sorting

Sorting entries in a regular loop will completely block the editor. We sort in an idle loop instead to put each sort on one libuv event loop iteration.

canceling

Because we used an idle, cancelling is very easy. When the user types before the previous sort has been completed, we cancel the previous sort. This is done by using a oneshot channel. In the telescope main loop if the user typed before the previous sort completed, we will send a signal to cancel the previous sort. This makes everything smooth and further makes sure that we are only sorting the latest event.

displaying the entries

Previously, we display every entry while it is sorted. For synchronous finding, this is okay. However, we are doing everything in an idle loop, so this will cause extreme flickering and highlighting issues. This was the 'weird' sorting animation that @elianiva was seeing. Now this is completely fixed because when we process an entry, we do not immediately display it, we only display all the entries when we are finished with processing everything. This is also what fzf does. One thing that we could do is add a percentage indicator which shows how far through processing we are. This is again from fzf.

the future

This will be very easy to implement for other finders, I just don't have enough time yet. The only finder that we cannot directly apply this to is grep_string. The reason is that there are so many entries that it completely overloads the jobs that we currently have. The solution is the new jobs pr. Once we have those we can again apply this implementation. If you want to see the speed difference, run the benchmark in the pr, it will test rg.

small issues

The only issue with this is that it doesn't highlight the current line correctly. This is likely due to the fact that we are only calling display code when we are done filtering everything. We need to separate display code from the other code so we can call the highlighting stuff separate from the other stuff. Another issue is that it doesn't display the first entry correctly. This is also the same issue. To see the correct entry just move your cursor back and forth.

[elianiva]

I think I'm missing something here..

Error executing lua callback: ...ck/packer/start/telescope.nvim/lua/telescope/pickers.lua:413: attempt to index upvalue 'tx' (a nil value)             1,3           All
Error executing luv callback:
...acker/start/plenary.nvim/lua/plenary/async_lib/async.lua:24: The coroutine failed with this message: ...ck/packer/start/telescope.nvim/lua/telescope/pickers.lua:435: 
attempt to index upvalue 'rx' (a nil value)

[oberblastmeister]

Make sure that you are using the latest version of my plenary pr because I added the channels today

[elianiva]

yup, the commit is 2765ee5 and it still doesn't work for me

[oberblastmeister]

hmm I'm not sure what is happening here

[oberblastmeister]

oh I forgot to commit my local repo.
pull the latest plenary async await again and it should work.

[elianiva]

ok nice, it works :)
for anyone who want to see how it works but haven't got time to try it, here's a short screencast

edit: it might look like it's just debouncing, but it's not. If the directory is too big (at least on my machine), it looks like this, but if it's small enough like Neovim repo, the visual representation acts normally, it's just faster and not blocking the input.

stuff.mp4

[oberblastmeister]

please use tjs version #709

[tjdevries]

TY :) thjis made it super easy to implement u r the bomb ober <3