inconsistent treatment of emoji completions

[stevengj]

JuliaLang/julia#10709 (@jakebolewski ) added emoji tab completions to the Julia REPL, e.g. \:koala: tab-completes to 🐨. Since IJulia uses the same tab-completion functions, it gets the same completions, but something weird happens when you type \:koala:<TAB>:

Note that it oddly thinks that there are two completions, not one. Not sure where this is coming from. It doesn't happen in the REPL, and Base.REPLCompletions.completions("\\:koala:", length("\\:koala:")) returns only a single completion.

[stevengj]

The message that is being sent to IPython (via IJulia.set_verbose(true)) is:

SENDING IPython Msg [ idents C485926233F4497B8BCC8AEEB9E87167 ] {
  header = Dict("msg_id"=>"ae06dfec-2397-4c20-9725-01d130d72cb9","msg_type"=>"complete_reply","username"=>"username","session"=>"C485926233F4497B8BCC8AEEB9E87167"),
  metadata = Dict{AbstractString,Any}(),
  content = Dict{ASCIIString,Any}("matches"=>UTF8String["🐨"],"status"=>"ok","matched_text"=>"\\:koala:")
}

which only has a single match. So, I suspect that something funky is happening on the IPython side here. @Carreau, @minrk?

[StefanKarpinski]

🐔

[minrk]

This might be a failure in the adaptation from the old v4 completion messages to v5.

[stevengj]

@minrk, where in the IPython codebase is this? I just ran into more weirdness today for 𝒦 (although the symptoms were different: tab completion seemed to delete some character that it wasn't supposed to; I have a hard time reproducing the exact circumstances). Like 🐨, this is outside the BMP, so it makes me think that you have a bug where you are using a UTF-16 encoding and are confusing 16-bit codeunits for codepoints.

[minrk]

Sorry for dropping this one. The koala completion is coming from the CodeMirror context hints. The adaptation is doing nothing wrong as far as I can tell. CodeMirror is seeing :koala as an identifier, and providing :koala as a local completion on the second :. I'm guessing this is a bug in CodeMirror's Julia mode.

What was happening with 𝒦? What were you typing and what did you see? I don't see any other unicode problems, and UTF16 is highly unlikely.

[stevengj]

@minrk, I don't remember exactly; I was doing something like \mscrK<tab> (𝒦) and some editing, and I remember at one point it seemed like a couple of characters suddenly disappeared. But I can't seem to reproduce it. I'll file a new issue if I can figure out what happened.

@malmaud, can the CodeMirror mode be fixed?

[malmaud]

I'll take a look.

[MarcMush]

I have the same issue, 7 years later