completions comments

[quinnj]

So I've been playing around with the allcompletions code as we talked about for Sublime-IJulia and I had a few comments/requests:

• Can the return type of [:hints] be made more consistent? From playing around with it, it seems like the result is sometimes an array of ASCIIString, sometimes UTF8String, and sometimes just an Any array. This is probably because a comprehension or map or something is being used. I can look into it more, but I figured you may know where the return array code is and could pre-allocate a consistent type
• It seems like sometimes the return list is a little excessive, take the following example:

julia> Jewel.allcompletions("P", LNR.cursor(1,1))[:hints]
1465-element Array{Any,1}:
 "%"
 ">>"
 ".\u2265"
 "\u220c"
 "-"
 ">="
 "|"
 "eval"
 "ndims"
 "~"
 ">:"
 "colon"
 "^"
 "|>"
 ".<<"
 "ENV"
 "transpose"
 "\u2288"
 "include"
 "LightTable"
 "×"
 ⋮
 "end"
 "export"
 "finally"
 "for"
 "function"
 "global"
 "if"
 "immutable"
 "import"
 "importall"
 "let"
 "local"
 "macro"
 "module"
 "quote"
 "return"
 "try"
 "type"
 "typealias"
 "using"
 "while"

Any idea why things like try or while are coming up as completions for P? It might also be good to be able to specify a cap limit on the number of completions to return.

• Are the completions ranked at all? Like, best matches first? Could be a possible solution to the cap limit, because the user could just take the top 20 or whatever

[MikeInnes]

So, I should clarify that all the system is meant to do is find the right context (latex, module, function etc.) and give you all possible completions, while filtering and ranking is left to the front end. It would definitely be nice to implement the filtering/ranking done in pure Julia at some point, but I haven't needed this yet and I think in your case it will make the most sense to take advantage of Sublime's great fuzzy searching.

So in your example the idea would be to provide Sublime with all 1400 completions once, even with spurious completions like "try" and "while", so that if the user deletes the p and starts typing something else Sublime can update instantly, without having to wait for the backend. You'd still want to call out to the backend for updates but asynchronously and periodically rather than on every keypress.

About the type consistency, I can definitely look into that, but can I ask why you need it to be more consistent? Since there's no real processing being done on them before passing them off it shouldn't be a performance issue.

[quinnj]

That makes sense and I actually figured out how to hook into Sublime's completions which does the fuzzy matching, so that makes sense.

About the type consistency, it's actually to make parsing the results easier. Basically, I'm passing in the call for completions to julia and then reading the output on completions[:hints] back into python to parse to give to Sublime. It would just make the parsing code easier if the result was always a consistent type, i.e. being read as UTF8String["while", "try"..., instead of sometimes ASCIIString["while", "try".... or Any["while", "try".....

One more thought, could we look into precompiling the allcompletions method? I've noticed it takes a few seconds the first time compiling before it can finally start rapid-fire completions.

[MikeInnes]

Ok, fair enough. I'll take a look at where the type instability is coming from.

Some more precompilation would definitely be nice. It should just be a case of calling the precompile function, right?

[quinnj]

Yeah, I think so. All of Base's precompilation is in base/precompile.jl, so I think it just works if you pass in the argument types.

[MikeInnes]

Actually, would it be OK just to convert it to an Any array on your end? Or, equivalently, serialise it via JSON instead of just rendering to a string directly?

The main issue is that completions come from a large number of places, including user-defined completion functions, so enforcing a specific type across all of them is a bit tricky.

Also, note that it's possible for completions to be a list of Dicts rather than strings. This is there to support LaTeX completions, where the displayed text is different from the inserted text, but I could add a flag to disable LaTeX completions if you want it.