Feature request: Listing mouse keys in defsrc

[GrewHair]

First off, thank you sooo much for Kanata - it literally saved my life. I've been in a desperate search for a Kmonad-style key remapping tool that would really care about Windows. I prefer Linux, but still have to use Windows for work - and the Kmonad's community really seems to be focused on Linux a lot more, leaving all kinds of bugs and weird behaviors of Windows unaddressed, which pretty much renders Kmonad unusable (at least for me) on Windows. Kanata really solves a lot of this for me, while bringing all sorts of new cool features as well!

The feature I'd like to request is being able to remap mouse keys - i.e. being able to list things like mlft and mrgt in the defsrc section.

Rationale: a lot of these fancy ergonomic keyboards (like Ergodox, Kinesis Advantage, etc.) offer a handy thumb cluster whereby you can activate the majority of the layers just by holding your thumb. This is great, because the thumb is the strongest finger and can handle straining action much better than any other finger. Now if you're tied to just your laptop keyboard for whatever reason - you're out of luck, and have to use something like home row mods. Which is nice, but has its disadvantages: it produces false positives - especially as you get sloppier during the work day, and it also requires you to hold down your index/middle/ring finger, and these get tired and start to sore way sooner (at least in my case). However, if you have a Thinkpad, or any laptop with a trackpoint, chances are it has some physical keys for left/right/maybe middle mouse click located atop the trackpad, right below the spacebar - a perfect place to land your thumbs on! So, potentially, if one would be able to tap-hold them (so they perform their normal functions when pressed alone and serve as modifiers/layer buttons when held), it would add 2 to 3 layer activation keys accessible by thumb!

Now I don't know nothing about Rust or how Windows handles mouse events, so I recognize it might be difficult/barely possible, and probably tap-holding the left mouse key would mess up click-and-drag behavior - but still, having this sort of ability even just for the right mouse key would do freaking wonders to my workflow - and hopefully to other people's workflows as well.

In any case, thank you for Kanata!

[jtroo]

I think what you're asking for could be possible using the Windows interception driver as well as on Linux. It certainly wouldn't fit too nicely into the current model, but at least those might be able to differentiate between and the physical buttons on a ThinkPad vs. trackpad clicks, as well as intercept and remap them. It depends how Lenovo implemented those interfaces though.

In the case that trackpad clicks vs. button clicks are differentiable, you could hack the kanata code to send different (unused) keys for the button device's L/R/M clicks and handle those appropriately. I happen to have a ThinkPad myself, so I can do some investigation on this. I don't have a design to easily generalize this functionality in mind right now though. Maybe I'll have a better idea after some testing.

[jtroo]

@GrewHair I forgot to say, thanks for the idea and for the kind words, it's much appreciated!

I've added some exploratory code in the mouse-intercept-investigation branch that works for my system. It should be noted that the code only adds a proof-of-concept on Windows using the interception driver.

The code remaps the right mouse button on my ThinkPad to be F13 instead. F13 can then be used in defsrc to have the desired behaviour. You can play around with the code, may need to print out the hwid for your system and replace LENOVO_BTN_HWID with the proper value.

For now, if you want the feature you'll have to work off of that branch and play around with the code. I likely won't be taking the concept and turning it into a fully supported feature for some time. Fully supporting different user customizations in the config file and also porting it to Linux would be a lot more work.

For anyone else seeing this issue, if you think you would use this feature, please leave a comment to help me gauge interest.

[Philipp-M]

Hey, this is actually a feature I'm looking for, in a different context though I guess:

I'm using a tiling window manager in Linux, and have remapped the tab key ((tap-hold-press 200 300 tab lmet))
to play as super key for all kinds of window-management related features, like moving/focusing windows etc.

One thing that bugs me currently a little bit, is that it's possible to hold the super key and move or resize windows by pressing a mouse button.

Currently this is limited by the hold timeout, so I have to wait 300 ms to be able to use the mouse for these actions.

It would be great to also have the mouse buttons trigger the hold key by using it as a second input device.
(Unfortunately kmonad also doesn't support this yet)

[jtroo]

@Philipp-M I've added some very-much untested code in the 108-linux-remap-mouse branch.

Listing the mouse buttons in defsrc is unfortunately not trivially workable with due to a limitation in the keyberon library and how kanata uses it, where indexes are u8 based.

The less user-friendly but easier code-wise workaround in the branch is to remap mouse buttons to other keys, e.g. unused F-keys (f22-f24). Then these unused F-keys would go in defsrc and be remapped in the deflayers. The F-keys could just be remapped to their original buttons, e.g. mlft, mrgt, mmid, but as you mentioned, the fact that they're getting processed by kanata means that they'll trigger the tap-hold-press.

I'll re-iterate that I haven't done any practical testing at all yet, but the code is there if you want to try it out and build off of it.

[Philipp-M]

Hey, thanks for investigating this.

Oh well... I started with this as well yesterday (including some refactoring), and as you said hit the u8 index issue in keyberon.

I have actually extended coord to be (u8, u16) everywhere (see jtroo/keyberon@master...Philipp-M:keyberon:coord-u8-to-u16), I'm not too much in the codebase yet, but I don't think this causes any issues, since coord is not used AFAIK to e.g. convert into a keyberon::KeyCode which I think is the real limitation as it's the USB HID Keyboard table (and thus shouldn't be extended or changed to u16 I guess).

I've extended the KanataLayers to support all OsCodes by increasing the size from 256 to OsCode::KeyMax (max value in OsCode (767)). This obviously increases the size of this table quite a bit, but I think it's still reasonable... For the future though it may enable all kinds of different OsCodes being mapped in defsrc.

Currently my implementation (main...Philipp-M:kanata:allow-every-oscode-in-defsrc) is working quite well so far (in Linux at least, but I guess it should(?) work in windows as well).

I can upstream this in a PR if you want to.

[jtroo]

Please do open PRs in the keyberon repo and here 🙂.

This surely won't work on Windows without a few more modifications since Windows doesn't treat mouse buttons as keyboard events, but that's fine. I or someone else can make the Windows changes at a later time.

[jokober]

+1 for this feature. It would be amazing to use the trackpad buttons as modifier/layer button!

[Philipp-M]

If you're using linux, this is already working, I use it as daily driver right now.

[jokober]

I see. However I don't get how to set it up correctly. Do you have an example configuration?

Looking through the code I found mappings in keys/mod.rs (e.g. 'mlft' to OsCode::BTN_LEFT ), but it looks like this is only a mapping relevant for Windows? Adding 'mft' to my defsrc didn't change anything. The buttons still have the same behavior like before.

Also I tried to add the trackpad devices to linux-dev. Is that necessary?

[Philipp-M]

Well I've configured it via nix:
The config: https://github.com/Philipp-M/nixos-config/blob/9d15c25ccf941b0d482241a75aeda4fb96e3c07f/configuration.nix#L190
and added the devices (including mouse, otherwise it doesn't intercept the mouse button events, in defcfg): https://github.com/Philipp-M/nixos-config/blob/9d15c25ccf941b0d482241a75aeda4fb96e3c07f/machines/zen/default.nix#L53

It could be that there are multiple mouse devices, for me there are two mouse input devices ...-event-mouse and ...-mouse, where *-event-mouse was the correct one.

You can also leave defcfg empty, and it automatically recognizes the devices that support keyboard inputs or mouse inputs (or maybe even don't put defcfg in there at all, I don't remember exactly anymore). But if there's a fancy input device (e.g. a joystick with mouse or keyboard support), it could pose problems, that's why I hardcoded the input devices (also because the nixos service automatically restarts kanata if it recognizes a reconnect of one of those devices).

[jokober]

I found a the reason why it didn't work. I had the package installed which does not include your changes yet. Building from source solved the problem. Thanks it works great!

[gerhard-h]

+1 for this feature. It would be amazing to use the trackpad buttons as modifier/layer button!
I also had this idea once, but no clue how to do it. ... meanwhile my new notebook don't have these buttons anymore

[jtroo]

If anyone's using Windows, with the Interception driver, with a laptop that has extra mouse buttons (wow that sounds pretty niche haha), the linked PR should hopefully implement this in Windows if you'd like to try. I don't think this is implementable without the driver for now, unfortunately, since there's no way to differentiate mouse devices in Windows hooks.

As usual, the standard warning about the Interception driver for potential new users:

This issue in the Interception driver exists: oblitum/Interception#25. This will affect you if you put your PC to sleep instead of shutting it down, or if you frequently plug/unplug USB devices.

[jtroo]

Leaving open for now for lack of sufficient testing

[GrewHair]

@jtroo awesome news, thanks soo much! Will try it later this week. Although I've avoided using the interception version so far because that bug bit me before on capsicain - but whatever, maybe I'll try cabled mouse instead of the bluetooth one, hopefully that helps.

FWIW IMHO differentiating devices - though definitely very useful - isn't always that essential. I can think of at least three use cases off the top of my head that could potentially do even better without it:

• this one was mentioned earlier here by @Philipp-M (say you want to use (tap-hold-release 200 200 spc lctl)on the spacebar, and also be able to zoom in the browser by holding the spacebar and scrolling the wheel, etc);
• just plain mouse button remapping (say you want mouse back/forward buttons to move through history in the terminal like the arrow keys do, or perform triple left click to instantly select paragraphs of text, etc, etc);
• defining Acme-style mouse chords. E.g. hold the forward button, then press and release the left click button, then press and release the right click button (together with the forward button), to perform some action.

Anyways, i don't wanna sound picky. What's already possible seems like a miracle in itself. It's more to emphasize the project's awesomeness and potential. So, thanks!

[GrewHair]

Noob question: how do I cargo build the wintercept variant?

[jtroo]

Noob question: how do I cargo build the wintercept variant?

You would pass in --features=interception_driver

Edit: or was it --features interception_driver... Maybe both? The justfile uses the latter

[GrewHair]

The immediate results are hilarious.

I've mapped mlft, mmid and mrgt to a, b, and c respectively, and what I get is:

• Clicking via trackpad itself (not the hardware buttons) works as intended, i.e. produces mouse clicks
• Clicking via either hardware trackpad buttons or an actual mouse produces letters
• No matter which button I click in this case (left, middle, or right), the letter's always c 😄
• The trackpad buttons also seem to have about half a second of "cooldown": once I press and release a button, c is printed, and then it needs half a second until I can do that again. If I press that button earlier then that, nothing happens. (UPD: Actual mice don't have that).

The hwid I get from Kanata's output is:

[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],

and it is the same for both the trackpad buttons and the usb mouse (guess it's just the way this laptop is built).
UPD: also the same for bluetooth mouse.

The laptop is HP EliteBook 840 G8.

I'll fiddle around with it more to see what I can squeeze out of it.

UPD: it seems that in any case, whichever base layer mapping (that corresponds to a mouse key) is the "rightmost"/closest to the end of the deflayer block, takes over all of the mouse keys in that layer (even _). If a mouse key isn't in the defsrc, it works normally.

[jtroo]

it seems that in any case, whichever base layer mapping (that corresponds to a mouse key) is the "rightmost"/closest to the end of the deflayer block

That weird behaviour is most likely because the mouse button OsCodes aren't mapped, so their numbers are overlapping all at 0, which is a bug of course.

kanata/src/keys/windows.rs

Line 496 in a248c3d

_ => 0,

The hwid I get from Kanata's output is: ...

Yea I found that was the case for a few of my mouse devices as well. My Lenovo Thinkpad does the right thing with this though. The problem if it's not set is that devices are no longer differentiable :(

[GrewHair]

That weird behaviour is most likely because the mouse button OsCodes aren't mapped, so their numbers are overlapping all at 0, which is a bug of course.

Sorry for a stupid question, a bug of what? Kanata? Windows? Interception? Firmware? Could you please elaborate?

It's not like it can't distinguish between them at all. E.g. if I put only mrgt in defsrc, it correctly remaps only the right click and leaves all others alone, so it can tell one from another in some sense.

[jtroo]

Sorry for a stupid question, a bug of what? Kanata?

All good, that's on me for not being explicit about which software is at fault; it's a bug in kanata at the code I linked.

[jtroo]

I think I've fixed the feasibly-fixable issues with the Windows implementation. Unfortunately issues like all devices showing as the same HW ID 0x0000 is not within the control of this project. Closing this issue.

[GrewHair]

Sure, you've totally crushed it :) Even with all that hardware-specific bs, it's still quite a usability boost at least for me. Thanks!