Keyboard layout for NLD + US International

[totaam]

Issue migrated from trac ticket # 2983

component: client | priority: trivial | resolution: fixed

2020-12-18 09:00:00: TijZwa created the issue

---

Hello,

I dont think this is a defect, but cant figure out how the local keyboard layout affects the Xpra client.

My case:
In windows I've two type of locales/keyboard settings. ENG + Us Keyboard and NLD + US international keyboard. (see screengrab 1)
The ENG + Us variant s detected as 0x409 and works as expected.
The NLD + Us internation variant is detected as 0x20409 and doesn't send quotes to the server.
See screengrab 2 for the output of Keyboard_info.exe.

With the ENG + Us keyboard the quote (or apostrophe) is logged as:

2020-12-17 18:04:56,735 client  26 @54.144 send_key_action(24, <GTKKeyEvent object, contents: {'modifiers': ['mod2'], 'keyname': 'apostrophe', 'keyval': 39, 'keycode': 222, 'group': 1, 'string': "'", 'pressed': True}>)

With the NLD + Us international keyboard the quote/apostrophe is logged as:

2020-12-17 18:04:11,007 client  26 @08.416 send_key_action(24, <GTKKeyEvent object, contents: {'modifiers': ['mod2'], 'keyname': 'dead_acute', 'keyval': 65105, 'keycode': 222, 'group': 0, 'string': '', 'pressed': False}>)

The US International and US English layout is the same (as far as I know...)
I just want the client to always send the key out as apostrophe, because my server is running Ibus to handle the keystrokes (apostrophe + e = é)

Do you have any tips on how to accomplish this?

[totaam]

2020-12-18 09:00:35: TijZwa uploaded file screengrab 1.png (9.2 KiB)

[totaam]

2020-12-18 09:00:47: TijZwa uploaded file screengrab2.PNG (21.2 KiB)

[totaam]

2020-12-18 09:01:57: TijZwa changed title from Keyboard layout for NLD + US Internation to Keyboard layout for NLD + US International

[totaam]

2020-12-18 09:01:57: TijZwa commented

---

[[Image(screengrab 1.png​)]]

[[Image(screengrab2.PNG​)]]

[totaam]

2020-12-18 09:02:53: TijZwa changed priority from major to trivial

[totaam]

2020-12-18 09:02:53: TijZwa changed component from android to client

[totaam]

2020-12-18 16:22:25: antoine changed status from new to assigned

[totaam]

2020-12-18 16:22:25: antoine commented

---

I can reproduce.

I assume this layout is the correct one to use for debugging:
[[Image(nl.png)]]

This looks difficult, depending on the keyboard layout chosen on the client, we get a different keyname: apostrophe vs dead_acute.
See Apostrophe and acute accent confusion which covers how different operating systems / layouts handle those keys.

---

What happens here is that the default us layout doesn't have a dead_acute key and so the server doesn't find the key to press:

2020-12-18 17:52:28,388 client   5 @44.397 mask_to_names(<flags 0 of type Gdk.ModifierType>) GetKeyState(VK_NUMLOCK)=1, names=['mod2']
2020-12-18 17:52:28,389 client   5 @44.398 parse_key_event(<Gdk.EventKey object at 0x0000000037d775e0 (void at 0x00000000036894d0)>, True)=<GTKKeyEvent object, contents: {'modifiers': ['mod2'], 'keyname': 'dead_acute', 'keyval': 65105, 'keycode': 222, 'group': 3, 'string': '', 'pressed': True}>
2020-12-18 17:52:28,405 client   5 @44.399 handle_key_action(ClientWindow(1), <GTKKeyEvent object, contents: {'modifiers': ['mod2'], 'keyname': 'dead_acute', 'keyval': 65105, 'keycode': 222, 'group': 3, 'string': '', 'pressed': True}>) wid=1
2020-12-18 17:52:28,405 client   5 @44.399 key_handled_as_shortcut(ClientWindow(1), 'dead_acute', ['mod2'], True) shortcuts_enabled=True, shortcuts=None
2020-12-18 17:52:28,405 client   5 @44.399 send_delayed_key() delayed_event=None
2020-12-18 17:52:28,405 client   5 @44.400 send_key_action(1, <GTKKeyEvent object, contents: {'modifiers': ['mod2'], 'keyname': 'dead_acute', 'keyval': 65105, 'keycode': 222, 'group': 3, 'string': '', 'pressed': True}>)
2020-12-18 17:52:28,406 will try levels: [4, 0, 5, 1, 6, 2, 7, 3]
2020-12-18 17:52:28,406 do_get_keycode(222, 'dead_acute', True, ['mod2'], 65105, '', 3)=-1, 3 (keyname translation)
2020-12-18 17:52:28,406 process_key_action(['key-action', 1, b'dead_acute', True, (b'mod2',), 65105, b'', 222, 3]) server keycode=-1, group=3
2020-12-18 17:52:28,406 set_keyboard_layout_group(3) ignored, value unchanged
2020-12-18 17:52:28,407 filtered_modifiers_set(['mod2'])={'mod2'}
2020-12-18 17:52:28,407 filtered_modifiers_set(['mod2'])={'mod2'}

TIL: process_key_action(['key-action', 1, b'dead_acute', True, (b'mod2',), 65105, b'', 222, 3]) server keycode=-1, group=3

I'm not 100% sure how to proceed here.
Generally speaking, we let the server do key translation and avoid doing any lookup client side.

Some options - which may be combined:

• have a fallback lookup table that maps common missing keys to their closest equivalent?
• add common missing keys to the keymap (since there are a few empty keycodes, usually)

[totaam]

2020-12-20 12:00:54: antoine uploaded file nl.png (25.2 KiB)

dutch keyboard layout according to wikipedia

[totaam]

2020-12-29 17:32:13: antoine commented

---

I think that the real problem is that we're trying to feed nl keys into a us server layout: everything works as expected if I launch my client with --keyboard-layout=nl.
We just need to detect that the nl keyboard layout is active.

---

Keyboard terminology

See also #1062 which is where we get the correct list of keyboard layouts (nl is in there), but without knowing which ones should be active?

#1282 ends with: The windows API only has a single keyboard, all the input goes there, no matter how many keyboards are attached

Calling ImmGetDescriptionA for the current language returns an empty string.

Also found this funny tidbit (includes python ctypes code): Windows 10 1803 and Keyboard Layouts which ends with Microsoft, please stop breaking things.!

In GetKeyboardLayout, I see: Beginning in Windows 8: The preferred method to retrieve the language associated with the current keyboard layout or input method is a call to Windows.Globalization.Language.CurrentInputMethodLanguageTag etc. But this looks like a dead end. Only for WinRT?
Maybe DotNET InputMethod.Current does what we want? It doesn't look like it, I get a NULL value.

Instead, it's the SUBLANGID that might give us what we want. See Language Identifiers.

According to Windows Language Code Identifier (LCID) Reference, the language id for nl should be 0x0013.
(there's also: 0x0413 for nl-NL, 0x0813 for nl-BE etc.)

You get 0x20409 and I get 0xd0020809 - not sure what those high bits are for. The important part is the sub-language, which is 0x1 in both cases - that's for United States - International layout.
So there's nothing there to indicate that the sub-language should be nl. Damn.

[totaam]

2020-12-30 05:14:50: antoine commented

---

Finally managed to get a more correct value using DotNET InputLanguage Class: InputLanguage.Culture contains nl-NL.

Now the problem is to extract this same data without using dotnet..

[totaam]

2020-12-30 06:22:12: antoine changed status from assigned to new

[totaam]

2020-12-30 06:22:12: antoine changed owner from antoine to TijZwa

[totaam]

2020-12-30 06:22:12: antoine commented

---

Fixed in r28280. Backported to older branches in 28282.

@TijZwa: please close if this works for you.
Updated beta builds have been uploaded.

[totaam]

2020-12-30 10:12:45: TijZwa changed owner from TijZwa to Antoine Martin

[totaam]

2020-12-30 10:12:45: TijZwa commented

---

Thanks! I am experiencing the same problem with 4.1.0.28280.
ENG/US works fine. NL/US INT doesnot work.

Is there anything I can test or information to gather?

[totaam]

2020-12-30 11:52:31: antoine changed owner from Antoine Martin to TijZwa

[totaam]

2020-12-30 11:52:31: antoine commented

---

Here's what I am seeing with this latest build (same as with 4.1-RC):

Python/GTK3 Microsoft Windows 10 aero client version 4.0.6-28284 64-bit
 OpenGL is disabled
 connected from 'DESKTOP-Q4IVN7P' as 'Windows 10 Test'
setting key repeat rate from client: 500ms delay / 33ms interval
setting keyboard layout to 'nl'

Then when I press the key, the server's -d keyboard log shows:

process_key_action(['key-action', 1, b'dead_acute', False, (b'mod2',), 65105, b'', 222, 3]) server keycode=48, group=0
set_keyboard_layout_group(0) ignored, value unchanged
filtered_modifiers_set(['mod2'])={'mod2'}
filtered_modifiers_set(['mod2'])={'mod2'}
is_modifier(48) not found
handle_key((1, False, 'dead_acute', 65105, 48, ['mod2'], False, True))
handle keycode unpressing  48: key 'dead_acute'
fake_key(48, False)
will try levels: [4, 0, 5, 1, 6, 2, 7, 3]
do_get_keycode(222, 'dead_acute', True, ['mod2'], 3)=48 (level=0, shift=False, mode=0, keysyms=['dead_acute', 'dead_grave', 'apostrophe', 'grave'])
switching group from 3 to 0

Since this is a compose key, you have to hit it twice to get the actual apostrophe alone.
But you can press it followed by another character and get things like é, ć and ó.
This seems to match the native behaviour in mswindows for me.

What is not behaving as you expected?

[totaam]

2021-01-20 21:24:30: TijZwa changed status from new to closed

[totaam]

2021-01-20 21:24:30: TijZwa set resolution to fixed

[totaam]

2021-01-20 21:24:30: TijZwa commented

---

Shame on me... It works as expected.
The only thing is; a long time ago I removed all thing like graves and accutes from the Keycodes.js file, so they were send to the server 'as-is'.

On the server I have Ibus (https://en.wikipedia.org/wiki/Intelligent_Input_Bus) running with input method 'Other - latn1-pre (m17n)', so Ibus handled these keys the right way.
That worked great for the HTML5 client, but with these changes, Ibus just ignores the dead_acute comming from the win32 client.
I've disable Ibus and everything is working fine.
I will re-enable the keycodes.js file and check if I need any tuning there.