Inject UTF-8 text #1426

rom1v · 2020-05-25T01:37:46Z

Since Android 7, we could use the PASTE keycode to inject UTF-8 text, instead of injecting every ASCII char sequentially. (ref: #786 (comment))

So I just implemented it.

If the device runs Android >= 7:

backup the clipboard
set the text to inject into the clipboard
inject the PASTE key code
restore the clipboard

It is very fast and allows to inject UTF-8 text, which is awesome for non-ASCII languages like Chinese.

Please test and report any problem.

Thank you for your feedbacks.

Here are binaries so that you can test even if you don't know how to build:

scrcpy.exe
SHA256: 4481d854630aaba9f31e4f3a19b46c4843c1918da03e1a486089bae3a3ebb09a
scrcpy-server
SHA256: 64cd4d4277040430c48d35677ad8472a6315337651fb9cfc5f62cf5abf2021b7

(on Windows, replace both files in the v1.13 release; on other platforms, use scrcpy-server and follow §Prebuilt server)

Only the main injection method was exposed on Device, the convenience methods were implemented in Controller. For consistency, move them all to the Device class.

Expose methods to inject key events and key codes with an additional parameter to specify the "mode" (async, wait for finish, wait for result).

On Android >= 7, inject text using the clipboard. This is faster and allows to inject UTF-8.

AndroidDeveloperLB · 2020-05-25T06:09:41Z

Where can I download it?

rom1v · 2020-05-25T07:40:49Z

I updated the post to include links to binaries.

AndroidDeveloperLB · 2020-05-25T07:55:00Z

Doesn't work well.
Each time I write a Hebrew letter, it also writes the English one .

Example:
I write: "ש"
And I get: "שa"

Not only that, but for some reason it doesn't work on all places. In the create-contact screen, it didn't work for some of the EditTexts...

And for some reason I can't run it without the console using "scrcpy-noconsole.exe" .

See here:
2020-05-25_10-52-45.zip

rom1v · 2020-05-25T08:09:42Z

Each time I write a Hebrew letter, it also writes the English one .

Example:
I write: "ש"
And I get: "שa"

What behavior do you get on v1.13?

What if you launch it with scrcpy --prefer-text?

Not only that, but for some reason it doesn't work on all places. In the create-contact screen, it didn't work for some of the EditTexts...

Oh, that's bad :(

And for some reason I can't run it without the console using "scrcpy-noconsole.exe" .

This is not a release, just a quick build so that you can test, I didn't post a new build of scrcpy-noconsole.exe.

AndroidDeveloperLB · 2020-05-25T08:22:50Z

By "v1.13", you mean the official one here:
https://github.com/Genymobile/scrcpy/releases
?
If so, instead of Hebrew it writes in English. So for the above example, when I type "ש" it puts "a" .

Using scrcpy --prefer-text seems to handle the English writing, but I've noticed an issue that is caused by the method itself of copying-to-clipboard:
It puts the texts that are copied into the clipboard history of Gboard ...

I still don't get how come the EditText issue exists. I can paste there...

rom1v · 2020-05-25T08:25:51Z

Not only that, but for some reason it doesn't work on all places. In the create-contact screen, it didn't work for some of the EditTexts...

It puts the texts that are copied into the clipboard history of Gboard ...

OK, thank you for your feedbacks. This method could not be used when typing text then.

However, I could probably use it to paste the text immediately on Ctrl+Shift+v (if Android >= 7), in addition to clipboard synchronization.

AndroidDeveloperLB · 2020-05-25T08:33:58Z

Can you please explain how could the EditText issue exist ? Pasting is allowed there...

Is it possible perhaps to create a (secured) app that uses root, to type characters via ScrCpy?
One that will allow only ScrCpy to type characters?
Or is even if it's possible, it could be very dangerous, as other apps might exploit it ?

rom1v · 2020-05-25T08:38:17Z

Can you please explain how could the EditText issue exist ? Pasting is allowed there...

I don't know.

Is it possible perhaps to create a (secured) app that uses root, to type characters via ScrCpy?
One that will allow only ScrCpy to type characters?
Or is even if it's possible, it could be very dangerous, as other apps might exploit it ?

Scrcpy should work without root. And indeed it looks dangerous.

AndroidDeveloperLB · 2020-05-25T08:52:34Z

I'm talking only about the non-English typing. I don't think there is any other solution, sadly.

Someone proposed a keyboard, but that's changing the way we use the device and it doesn't seem like a stable solution.

I already requested Google to add unicode support:
https://issuetracker.google.com/issues/143375921

It doesn't even work for emulator:
https://issuetracker.google.com/issues/153619173

But I don't think it's going to be added anytime soon.

quyleanh · 2020-09-03T04:23:34Z

@rom1v so could you investigate the work around based on their implementation?
As your comment, I think people cannot trust mirroid.io at all. Thank you for comment anyway.

AndroidDeveloperLB · 2020-09-03T06:44:27Z

@quyleanh Do you know perhaps of a different alternative to it, that does offer unicode?

quyleanh · 2020-09-03T08:50:28Z

@AndroidDeveloperLB no, I don't. I have tested QtScrcpy but it doesn't support unicode. (of course, it's based on scrcpy). Hope @rom1v can find out the way.

AndroidDeveloperLB · 2020-09-03T09:12:19Z

@rom1v I hope you sue them for violating.
But I also want to know how to avoid such an issue myself. I thought apache is quite free-to-use.

vip-liutao · 2020-10-21T01:25:02Z

接时替换键盘应用程

mirroid真强大,是我需要的
一样不延迟并且解决了输入法问题，这个在scrcpy困扰了我很久
这两款软件的音频都可以采用
蓝牙连接电脑
win10应用商店下载
Bluetooth audio receiver
使用

gaussandhisgun · 2021-01-28T21:46:51Z

can you add built linux binaries of this in here? my distro seems to have problems with meson as it provides only 0.45 and scrcpy needs 0.48 to be built

thedmd · 2021-02-18T02:24:23Z

Version from top post works for me like a charm. Polish characters are handled well. Thank you!

heweisheng · 2021-09-14T10:39:49Z

if check the message is utf8 char use paste?or not use ascii inject。maybe is a idea but i need check all char

AlexBurdu · 2021-11-10T15:25:40Z

I adapted this a bit in my fork and works great for my needs.
Thanks!

quyleanh · 2021-11-10T15:39:28Z

@AlexBurdu could you add some brief info?

AlexBurdu · 2021-11-10T20:53:08Z

So the implementation that you can see in my fork, which is a slightly modified version of the one in this PR works fine on my Pixel phone. Unfortunately it does not work well on my Samsung Galaxy S20 FE.
I believe Samsung may have customized the way clipboard works on their rom, as when I'm sending characters through clipboard I see them in Gboard keyboard clipboard history, but I don't see them in OS clipboard or Samsung Keyboard's clipboard.

I initially started poking at this to add Romanian diacritics support, but it seems that support for them is missing in the Android OS KeyCharacterMap class, and no events are returned for KeyCharacterMap.getEvents(chars)
For example, ACCENT_COMMA_BELOW is commented out in the source file. While ACCENT_BREVE is there, KeyCharacterMap.getEvents() still returns null for breve('A') / breve('a'). I didn't dig further to see why that happens, I'm assuming that some mapping for it is also missing in this function that is meant to be used for tests.

AlexBurdu · 2021-11-20T18:22:34Z

@rom1v would it be possible for scrcpy server to register an IME that would take care of text input, or is not possible?
Is scrcpy-server seen by the OS as a regular application or like "something else" (what else)?

Note: I have read DEVELOP.md, but it's still not clear to me and I'd like to contribute to this.

rom1v · 2021-11-20T20:50:24Z

Is scrcpy-server seen by the OS as a regular application or like "something else" (what else)?

No, it's not an Android app, just a Java executable accessing the Android APIs.

More details: https://blog.rom1v.com/2018/03/introducing-scrcpy/

About IME, also see #1751.

AlexBurdu · 2021-11-21T17:25:31Z

Thanks, the hack in #1751 is what I needed as a fallback to key inject!
It's a bit laggy sometimes, but not that annoying.

quyleanh · 2021-11-21T22:07:07Z

@AlexBurdu could you please give me the Windows build for your latest implementation? I would like to test with my inject text scenario.
Thank you.

domnpp · 2021-11-22T06:50:26Z

@quyleanh Here.
@AlexBurdu You did the hard work, I managed to build it for Windows and therefore I may as well share it with anyone else that's interested.
https://github.com/domenn/scrcpy/releases/tag/v1.19

AlexBurdu · 2021-11-22T07:29:53Z

Thanks, I just adapted the work others have done in the linked PRs.
Please note that you need to install ADBKeyBoard and select it as your keyboard while you use scrcpy for it to work:

Also, i have a few changes done to the mouse buttons bindings on my fork.

AlexBurdu · 2021-11-22T12:14:28Z

For Linux/Mac users, I use this trick to get the ADBKeyBoard set when I start scrcpy and then switch back to my previous one (GBoard) when I exit it. Should be easy to adapt it for Windows also.

export personal_phone=192.168.0.94:8888 && \
adb connect $personal_phone && \
adb -s $personal_phone shell ime set com.android.adbkeyboard/.AdbIME && \
scrcpy \
--turn-screen-off \
--stay-awake \
--window-x 2500 --window-y 0 \
--max-size 1024 \
-s $personal_phone; \
adb -s $personal_phone shell ime set com.google.android.inputmethod.latin/com.android.inputmethod.latin.LatinIME

gaussandhisgun · 2021-11-23T20:59:16Z

Thanks, I just adapted the work others have done in the linked PRs. Please note that you need to install ADBKeyBoard and select it as your keyboard while you use scrcpy for it to work:

Also, i have a few changes done to the mouse buttons bindings on my fork.

since i for can not submit issues on your fork built by @domenn, i'll put it right here. for whatever reason, on non-english layout, when pressing a russian key, both english and russian chars appear at the same time. this might be my clipboard manager or kde connect freaking out but they seem to copy only russian chars.

my devices are: redmi note 9s running pixelplusui on android 11 and a laptop running windows 10 21H1 x64, connection was done with usb, commandline is scrcpy.

upd. seems fine with scrcpy --prefer-text tho.

upd2. i also get a system accent colored screen if i start your build with -f to make it fullscreen by default

AlexBurdu · 2021-11-23T22:21:52Z

that sounds like a bug for https://github.com/senzhk/ADBKeyBoard, but I'm not sure.
I'm no longer using clipboard for this, but sending the chars through adb shell to ADBKeyBoard.

quyleanh · 2021-11-24T05:12:55Z

@domenn @AlexBurdu thank you very much. I have tried and it work well with my case.
Input Vietnamese text need to be careful to not mess with clipboard.

How can I disable inject with clipboard? I would like to use only with ADBKeyBoard.

AlexBurdu · 2021-11-24T09:52:49Z

We have diverged a lot from the initial PR discussion (sorry for that). in my fork, I don't have any code left for injecting through clipboard.

yilksd · 2022-01-05T13:28:04Z

that sounds like a bug for https://github.com/senzhk/ADBKeyBoard, but I'm not sure. I'm no longer using clipboard for this, but sending the chars through adb shell to ADBKeyBoard.

How to use your fork? I have just compiled your fork and launched it but I still cannot type Chinese character. I have the ADB keyboard app installed.

quyleanh · 2022-07-05T10:52:23Z

I have compiled the Windows builds with @AlexBurdu 's ADBKeyBoard merged.
Feel free to use in this release.
https://github.com/quyleanh/scrcpy/releases/tag/v1.24

electriquo · 2022-08-07T04:42:08Z

see #37 (comment) for probable solution

rom1v added 3 commits May 25, 2020 03:28

Move injection methods to Device

4bbabfb

Only the main injection method was exposed on Device, the convenience methods were implemented in Controller. For consistency, move them all to the Device class.

Add more convenience methods for injection

cc4e1e2

Expose methods to inject key events and key codes with an additional parameter to specify the "mode" (async, wait for finish, wait for result).

Add a new method for text injection

d613b10

On Android >= 7, inject text using the clipboard. This is faster and allows to inject UTF-8.

This was referenced May 25, 2020

The Swedish letter å can not be entered #1427

Closed

utf-8 is not supported #1428

Closed

rom1v mentioned this pull request Oct 3, 2020

Adding new option for falling back to clipboard pasting on character injection failures. #1805

Open

rom1v mentioned this pull request Feb 16, 2022

Add new keyboard input emulation mode based on clipboard #3027

Open

1 task

quyleanh referenced this pull request in AlexBurdu/scrcpy Jul 5, 2022

Merge branch 'Genymobile:master' into master

80f3e15

rom1v mentioned this pull request May 4, 2023

Why not use the copy and paste function to solve text input problems in windows? #3970

Closed

1 task

rom1v mentioned this pull request Jul 10, 2023

€ character cannot be entered #2511

Closed

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Inject UTF-8 text #1426

Inject UTF-8 text #1426

rom1v commented May 25, 2020 •

edited

Loading

AndroidDeveloperLB commented May 25, 2020

rom1v commented May 25, 2020 •

edited

Loading

AndroidDeveloperLB commented May 25, 2020

rom1v commented May 25, 2020

AndroidDeveloperLB commented May 25, 2020

rom1v commented May 25, 2020

AndroidDeveloperLB commented May 25, 2020

rom1v commented May 25, 2020

AndroidDeveloperLB commented May 25, 2020

quyleanh commented Sep 3, 2020

AndroidDeveloperLB commented Sep 3, 2020

quyleanh commented Sep 3, 2020

AndroidDeveloperLB commented Sep 3, 2020

vip-liutao commented Oct 21, 2020 •

edited

Loading

gaussandhisgun commented Jan 28, 2021

thedmd commented Feb 18, 2021

heweisheng commented Sep 14, 2021

AlexBurdu commented Nov 10, 2021 •

edited

Loading

quyleanh commented Nov 10, 2021

AlexBurdu commented Nov 10, 2021 •

edited

Loading

AlexBurdu commented Nov 20, 2021 •

edited

Loading

rom1v commented Nov 20, 2021

AlexBurdu commented Nov 21, 2021 •

edited

Loading

quyleanh commented Nov 21, 2021

domnpp commented Nov 22, 2021

AlexBurdu commented Nov 22, 2021 •

edited

Loading

AlexBurdu commented Nov 22, 2021 •

edited

Loading

gaussandhisgun commented Nov 23, 2021 •

edited

Loading

AlexBurdu commented Nov 23, 2021

quyleanh commented Nov 24, 2021 •

edited

Loading

AlexBurdu commented Nov 24, 2021 •

edited

Loading

yilksd commented Jan 5, 2022

quyleanh commented Jul 5, 2022

electriquo commented Aug 7, 2022

Inject UTF-8 text #1426

Inject UTF-8 text #1426

Conversation

rom1v commented May 25, 2020 • edited Loading

AndroidDeveloperLB commented May 25, 2020

rom1v commented May 25, 2020 • edited Loading

AndroidDeveloperLB commented May 25, 2020

rom1v commented May 25, 2020

AndroidDeveloperLB commented May 25, 2020

rom1v commented May 25, 2020

AndroidDeveloperLB commented May 25, 2020

rom1v commented May 25, 2020

AndroidDeveloperLB commented May 25, 2020

quyleanh commented Sep 3, 2020

AndroidDeveloperLB commented Sep 3, 2020

quyleanh commented Sep 3, 2020

AndroidDeveloperLB commented Sep 3, 2020

vip-liutao commented Oct 21, 2020 • edited Loading

gaussandhisgun commented Jan 28, 2021

thedmd commented Feb 18, 2021

heweisheng commented Sep 14, 2021

AlexBurdu commented Nov 10, 2021 • edited Loading

quyleanh commented Nov 10, 2021

AlexBurdu commented Nov 10, 2021 • edited Loading

AlexBurdu commented Nov 20, 2021 • edited Loading

rom1v commented Nov 20, 2021

AlexBurdu commented Nov 21, 2021 • edited Loading

quyleanh commented Nov 21, 2021

domnpp commented Nov 22, 2021

AlexBurdu commented Nov 22, 2021 • edited Loading

AlexBurdu commented Nov 22, 2021 • edited Loading

gaussandhisgun commented Nov 23, 2021 • edited Loading

AlexBurdu commented Nov 23, 2021

quyleanh commented Nov 24, 2021 • edited Loading

AlexBurdu commented Nov 24, 2021 • edited Loading

yilksd commented Jan 5, 2022

quyleanh commented Jul 5, 2022

electriquo commented Aug 7, 2022

rom1v commented May 25, 2020 •

edited

Loading

rom1v commented May 25, 2020 •

edited

Loading

vip-liutao commented Oct 21, 2020 •

edited

Loading

AlexBurdu commented Nov 10, 2021 •

edited

Loading

AlexBurdu commented Nov 10, 2021 •

edited

Loading

AlexBurdu commented Nov 20, 2021 •

edited

Loading

AlexBurdu commented Nov 21, 2021 •

edited

Loading

AlexBurdu commented Nov 22, 2021 •

edited

Loading

AlexBurdu commented Nov 22, 2021 •

edited

Loading

gaussandhisgun commented Nov 23, 2021 •

edited

Loading

quyleanh commented Nov 24, 2021 •

edited

Loading

AlexBurdu commented Nov 24, 2021 •

edited

Loading