[Bug] KVM input not recognized by win-vind on remote machine

[qaptoR]

About Bug

The software does not recognize network inputs from KVM client software.

Steps to reproduce

1. I'm presently using InputDirector to manage a two computer setup
2. win-vind works correctly on remote computer with hardware keyboard, but KVM keyboard inputs from InputDirector are ignored.

Interestingly, if win-vind is in a blocking mode on the remote system], it will still block the KVM input

Environments

Environments
Windows Edition	10 Home
64 bit
most recent build

[pit-ray]

@qaptoR
I'm not very familiar with that virtual machine, but maybe the InputDirector uses SendInput to create keycode messages. The win-vind has not only keycode capture but also the emulation of input with external macros. And then, to avoid receiving own created messages and falling into the infinite dead lock, win-vind ignore injection with SendInput at here.

Therefore, every program to create key messages except for the physical keyboard driver, probably will not work well.

Thanks for using it.

[qaptoR]

@pit-ray
Thank you for letting me know.
I really like your software, I'm still trying to work it into my day-to-day routine.

I'll look into seeing what kind of messaging InputDirector uses, maybe I can create a patch?
Most likely would depend on whether or not we could detect where the input is coming from, and then maybe create a white-list for programs.

On a somewhat unrelated note, does the 'EasyClick' feature rely on .NET UI Automation library?
I noticed that applications which run their GUI with electron don't work with 'EasyClick'.
Also, neither does the Godot game engine, but that's for an entirely different reason (they use their own GUI library).

So I want to figure out how to hook into Electron, and what to expose in godot that will enable win-vind to hook in to that as well. (I develop games with VSCode, which uses electron, and Godot)

[pit-ray]

@qaptoR
Sorry for my late reply.

If you know the cause, please send me pull requests or issues.

And about EasyClick, it uses Win32 API C++ UI Automation, not .NET.
ref. https://github.com/pit-ray/win-vind/blob/master/src/util/uiwalker.cpp

However, it has been proven to work with VS Code on the latest Windows.

Thank you for using it. It may be undiscovered bugs, so I want more details.

Let me know what key-bindings do not work (e.g. gg, t). And, please check log files in ~/.win-vind/log/ after its behavior.

By the way, in my test environment, EasyClick works in VS Code.

VSCode: 1.54.3 (Stable), 1.55.0 (Insiders) Windows 10: Home 64bit 20H2 Build 19032.867
#11

Thanks!

[qaptoR]

@pit-ray Thank you!
my apologies. Yes, I have now gotten easyclick to work with vscode and microsoft edge.

as I'm trying to consider, from a purely mathematically logic pov how to handle KVM emulated input over a network, is it possible to create a 'mode' for win-vind that does not allow it to use external macros, and which also exposes it to emulated input.

this could be called something like "network limited device" mode, assuming that win-vind is being used as on a controlled network computer.

[pit-ray]

@qaptoR
Thanks for your many contributions. I'll reply all little by little since I'm busy this month.

About "network limited device" mode, we must detect the key message where come from.
If we enable the mode manually like set vmmode (virtual machine mode) and ignore all SendInput API of win-vind, we cannot use shortcut macros (e.g., Vim emulation mode uses <left> key or <right> key). In other words, the infinite loop problem for SendInput is not only external macros but also all key message generation functions.

Therefore, we must detect the type of application inside LowLevelKeyboardProc.

Win32API: https://learn.microsoft.com/en-us/windows/win32/winmsg/lowlevelkeyboardproc
win-vind implementation: https://github.com/pit-ray/win-vind/blob/master/src/core/inputgate.cpp#L137-L182

Thanks.

[qaptoR]

@pit-ray
Thankyou for the links! I will look into them as I have time as well.