Surrogate pairs (emojis, etc.) are not handled well in insert mode

[athos]

Description

When I'm typing in a text containing surrogate pairs (like emoji, relatively complicated kanji, etc.) using my input method in the insert mode, the text gets garbled as shown in the GIF below:

After digging into the codebase, I found that this line of code does not seem to handle surrogate pairs well:

    const char = this.keysPressed[this.keysPressed.length - 1];

This can be confirmed as follows:

> var text = '🤔'
> text.length
2
> text[text.length-1]
'�'
> var text2 = '𩸽'
> text2.length
2
> text2[text2.length-1]
'�'

To fix this behavior, I think we need to do something like what VSCode does in getPrevCodePoint. If we had it, we should be able to handle text containing surrogate pairs well like the following:

> String.fromCodePoint(getPrevCodePoint(text, text.length))
'🤔'

To Reproduce
Typing in a text containing surrogate pairs in the insert mode.

Expected behavior
Expected characters are successfully entered in the editor.

Environment:

• Extension (VsCodeVim) version: 1.18.5
• VSCode version: 1.52.1
• OS: macOS Big Sur (Darwin x64 20.2.0)

[xiekeyi98]

I have the same issue.

Would you mind taking a look @J-Fields , thanks.

[J-Fields]

Can you confirm this is still an issue? This implementation was changed a bit in 1.19.0

[athos]

Thank you for letting me know! 🤝

I've checked with 1.19.2, but I still see the same issue, unfortunately.

[midorizemi]

Hello. I still have same issue too.

However, I found that I can get around this problem by using :emojisense: . Please refer to it.

Environment:

• Windows 10 Pro, Ver.: 2004, (build: 19041.928)
• VSCode Ver.: 1.55.2
• VSCodeVim Ver.: 1.20.2

Thank you.

[athos]

Yeah, I know that extension exists. Moreover, VS Code itself has built-in support for emoji input (Ctrl-Cmd-Space on Mac), which works fine in the insert mode of VSCodeVim.

But, this issue is not only about emoji, but also about any other characters that are represented as surrogate pairs, so it still needs to be resolved, even then.

[uiii]

This also happens to me when I try to insert emoji using Windows + .. It prints the emoji for a short moment and then changes to two question marks. It is very annoying because now I have to type the emoji in some other editor and then paste it to vscode.

[HouCoder]

Spend an hour on this issue, finally realized it's vscodevim's issue for not having emojis properly displayed.

[athos]

Wow, I've confirmed that the latest release resolves the issue. Excellent! 🎉 🎉
Thank you so much from the bottom of my heart for tackling this longstanding issue!

[dubaaron]

Hi, according to above, this was fixed, but I am experiencing this problem now on Mac, vscodevim version 1.24.3, VSCode version: 1.68.1 ... in insert mode, if I try to trigger the emoji insertion dialog (Ctrl + Cmd + Space), nothing happens.

I can trigger it in another app, and then paste into insert mode in vscode vim, but can't seen to trigger it in vscode itself now. 😥

PS -- this is very cool extension, thanks for making this. Now I can improve my vim skills while using VSCode, haha 😁