How to open only the mouse control thread?

[yuyiqilin]

OS: Windows 11
Scrcpy version: 3.0
Installation method: Windows release

I am a developer, and I am using scrcpy to develop a keypress automation tool for controlling Android phones. It is a program that controls an Android phone by configuring 'click location information' on the computer. In version 2.7, I could keep only the mouse thread active while disabling both video and audio to save resources. However, I have found that in version 3.0, it is not possible to enable only the mouse thread. How should I proceed?

[rom1v]

The default mouse mode has changed from "uhid" to "disabled" when video playback is turned off: 2c25fd7.

So you must enable it explicitly:

scrcpy --no-video --no-audio -M

I fixed the documentation 1d2f16d.

[yuyiqilin]

The default mouse mode has changed from "uhid" to "disabled" when video playback is turned off: 2c25fd7.

So you must enable it explicitly:

scrcpy --no-video --no-audio -M

I fixed the documentation 1d2f16d.

You can control it using the -M parameter with the app tool you provided. How can I control it in my own developed tool, for example, by using the following adb command-line parameters:

adb CLASSPATH=/data/local/tmp/scrcpy-server.jar app_process / com.genymobile.scrcpy.Server 3.0 log_level=debug video_bit_rate=8000000 tunnel_forward=false control=True audio=False video=False display_id=0 show_touches=False stay_awake=False power_off_on_close=false downsize_on_error=true cleanup=true max_size=1080

[rom1v]

You would need to mimic the protocol between the client and the server: https://github.com/Genymobile/scrcpy/blob/master/doc/develop.md#protocol

[yuyiqilin]

You would need to mimic the protocol between the client and the server: https://github.com/Genymobile/scrcpy/blob/master/doc/develop.md#protocol

I have completed everything related to the protocol, and everything is working fine. What I am referring to is that scrcpy-server does not provide a usable parameter to configure the mouse mode as uhid, is that correct?

[rom1v]

Correct, this is a client configuration. UHID events are generated by the client (so that the HID code is shared between AOA and UHID, where AOA does not use a server, but sends the events directly over USB).

[yuyiqilin]

Correct, this is a client configuration. UHID events are generated by the client (so that the HID code is shared between AOA and UHID, where AOA does not use a server, but sends the events directly over USB).

When I was using scrcpy-server version 2.7, I could enable the mouse thread separately. Why is it no longer possible in version 3.0? Has scrcpy-server in version 3.0 disabled this feature? I only enabled the mouse thread and then used the Socket protocol you provided to control mouse clicks. In version 3.0, after sending the mouse click Socket data packet, an exception is thrown saying: 'The mouse Socket has been disconnected.

[yuyiqilin]

Correct, this is a client configuration. UHID events are generated by the client (so that the HID code is shared between AOA and UHID, where AOA does not use a server, but sends the events directly over USB).

Can you fix this issue? Otherwise, I will have to revert to scrcpy-server version 2.7, as version 2.7 allows me to enable only the mouse control thread.

[rom1v]

When I was using scrcpy-server version 2.7, I could enable the mouse thread separately.

What do you mean?

In version 3.0, after sending the mouse click Socket data packet, an exception is thrown saying: 'The mouse Socket has been disconnected'

I don't remember changing anything related to that. Check the git log between 2.7 and 3.0.

[yuyiqilin]

When I was using scrcpy-server version 2.7, I could enable the mouse thread separately.

What do you mean?

In version 3.0, after sending the mouse click Socket data packet, an exception is thrown saying: 'The mouse Socket has been disconnected'

I don't remember changing anything related to that. Check the git log between 2.7 and 3.0.

I developed my program using only your scrcpy-server file and using the socket protocol for controlled communication. In version 2.7 of scrcpy-server, I can only open the mouse control thread, but in the latest release version 3.0, after sending the click packet, an exception will be thrown, that is, the host you call closed the connection。

This means that the 3.0 version of scrcpy-server must have changed something, so that I can not only open the mouse thread. In addition, starting both video and mouse threads at the same time is normal in version 3.0.

[rom1v]

but in the latest release version 3.0, after sending the click packet, an exception will be thrown

Use git bisect between v2.7 and v3.0 to find out the commit which breaks for your program.

[yuyiqilin]

but in the latest release version 3.0, after sending the click packet, an exception will be thrown

Use git bisect between v2.7 and v3.0 to find out the commit which breaks for your program.

When I close the video and audio socket, after connecting to the scrcpy-server, it returns an info message: 'No video mirroring, SDK mouse disabled.' Is it because of this error message that prevents the use of the mouse socket for communication?I need this SDK mouse works!

[rom1v]

This is a client side message (see my previous comment: #5542 (comment)).

If you use scrcpy-server directly, this code is not called.

[yuyiqilin]

This is a client side message (see my previous comment: #5542 (comment)).

If you use scrcpy-server directly, this code is not called.

Oh, alright. This is indeed the result from using your client sample. But I still haven't found where the bug is.Sorry to bother you.

[rom1v]

But I still haven't found where the bug is.

Did you git bisect?

[yuyiqilin]

But I still haven't found where the bug is.

Did you git bisect?

Sorry, I'm not very good at using this.

[rom1v]

#1284 (comment)

[yuyiqilin]

#1284 (comment)

The information about this exception, is it helpful to you? When I sent the mouse move information, scrcpy-server threw this exception：

{[server] INFO: Device: [Xiaomi] Redmi 23013RK75C (Android 14)
[server] DEBUG: Controller stopped
[server] DEBUG: Device message sender stopped
[server] ERROR: Exception on thread Thread[control-recv,5,main]
java.lang.NullPointerException: Attempt to read from field 'com.genymobile.scrcpy.control.PositionMapper com.genymobile.scrcpy.control.Controller$DisplayData.positionMapper' on a null object reference in method 'com.genymobile.scrcpy.control.PositionMapper com.genymobile.scrcpy.control.Controller$DisplayData.access$300(com.genymobile.scrcpy.control.Controller$DisplayData)'
at com.genymobile.scrcpy.control.Controller$DisplayData.access$300(Controller.java:55)
at com.genymobile.scrcpy.control.Controller.injectTouch(Controller.java:361)
at com.genymobile.scrcpy.control.Controller.handleEvent(Controller.java:255)
at com.genymobile.scrcpy.control.Controller.control(Controller.java:194)
at com.genymobile.scrcpy.control.Controller.lambda$start$0$com-genymobile-scrcpy-control-Controller(Controller.java:202)
at com.genymobile.scrcpy.control.Controller$$ExternalSyntheticLambda0.run(D8$$SyntheticClass:0)
at java.lang.Thread.run(Thread.java:1012)

[ZhaoZuohong]

I use scrcpy-server only and communicate by socket connection. v2.7 works fine, but v3.0 doesn't.

This is the first time I try git bisect. The final output is:

> git bisect good                                                               
874eaec487369f7fcaa9ed8c5f85569659565d4f is the first bad commit
commit 874eaec487369f7fcaa9ed8c5f85569659565d4f
Author: Romain Vimont <[email protected]>
Date:   Sat Oct 12 09:23:31 2024 +0200

    Move screen-related features out of Device.java
    
    Move the code related to screen size and rotation/fold to ScreenCapture.
    
    For now, keep the ScreenInfo instance in the Device class to communicate
    with the Controller, but it will be removed by further commits.
    
    PR #5370 <https://github.com/Genymobile/scrcpy/pull/5370>

 server/src/main/java/com/genymobile/scrcpy/Server.java                 |   3 ++-
 server/src/main/java/com/genymobile/scrcpy/device/Device.java          | 127 +++++++++++++-------------------------------------------------------------------------------------------------
 server/src/main/java/com/genymobile/scrcpy/video/ScreenCapture.java    | 123 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------
 server/src/main/java/com/genymobile/scrcpy/video/SurfaceCapture.java   |   3 ++-
 server/src/main/java/com/genymobile/scrcpy/wrappers/WindowManager.java |  20 ++++++++++++++++--
 5 files changed, 135 insertions(+), 141 deletions(-)

I then tested against commit 874eaec487369f7fcaa9ed8c5f85569659565d4f and one before it (14e5439dee5486f870bda95add4102eaba39971c). Commit 14e5439d works fine. With commit 874eaec here is no effect after sending packets over the control socket.

Note that in v3.0 scrcpy just quits, which is different from commit 874eaec.

[rom1v]

Please test branch issue5542: 330264c

[ZhaoZuohong]

I've tested branch issue5542 and it works properly.

[rom1v]

Please also test this new branch: issue5542.3.

[ZhaoZuohong]

Branch issue5542.3 works for me.

[rom1v]

Merged into dev 🚀

[yuyiqilin]

Merged into dev 🚀

This version 3.0.2 still has some bugs. When only using scrcpy-server and enabling the mouse thread, the 'maxsize=xxxx' parameter does not work, and the mouse operation position is still relative to the original size. This bug did not exist in version 2.7, but it appears in version 3.0.2