3.0.2 still has some bugs for just opening the mouse control thread

[yuyiqilin]

OS: Windows 11
Scrcpy version: 3.0.2
Installation method: Windows release

This version 3.0.2 still has some bugs. When only using scrcpy-server and just opening the mouse thread, the 'maxsize=xxxx' parameter does not work, and the mouse operation position is still relative to the original size. This bug did not exist in version 2.7, but it appears in version 3.0.2。
This bug causes the click position to be different when the same maxsize value is configured, depending on whether the video thread is enabled or not.

[rom1v]

the 'maxsize=xxxx' parameter does not work, and the mouse operation position is still relative to the original size.

Refs #5542.

Yes, the architecture has changed in scrcpy 3.0 to support virtual displays: the events are now resolved depending on the actual virtual display (d193967) and possibly filters applied. Btw, the transform to apply depends on the API used (with SurfaceControl, the events are sent to the original display id with coordinates relative to the original display size, while with DisplayManager, the events are sent to the mirroring virtual display relative to the mirroring virtual display size, to fix #4598 and #5137).

If there is no video playback (i.e. no display), the events cannot be mapped. In the fix above, I just allowed to send events to absolute positions, but display transforms may not be applied (resize, angle, rotation, flip…). In the controller, there is no information about display when there is no display. I probably won't change that for use cases not relevant to scrcpy, sorry. I keep that in mind though, if it's possible to make it work without impacting scrcpy uses cases.

[yuyiqilin]

the 'maxsize=xxxx' parameter does not work, and the mouse operation position is still relative to the original size.

Refs #5542.

Yes, the architecture has changed in scrcpy 3.0 to support virtual displays: the events are now resolved depending on the actual virtual display (d193967) and possibly filters applied. Btw, the transform to apply depends on the API used (with SurfaceControl, the events are sent to the original display id with coordinates relative to the original display size, while with DisplayManager, the events are sent to the mirroring virtual display relative to the mirroring virtual display size, to fix #4598 and #5137).

If there is no video playback (i.e. no display), the events cannot be mapped. In the fix above, I just allowed to send events to absolute positions, but display transforms may not be applied (resize, angle, rotation, flip…). In the controller, there is no information about display when there is no display. I probably won't change that for use cases not relevant to scrcpy, sorry. I keep that in mind though, if it's possible to make it work without impacting scrcpy uses cases.

The socket protocol for the mouse includes parameters for position, such as width, height, and x, y coordinates. Is it possible to perform a simple conversion based on this?

[rom1v]

The socket protocol for the mouse includes parameters for position, such as width, height, and x, y coordinates. Is it possible to perform a simple conversion based on this?

It includes the coordinates, and the video size to which these coordinates are relative.

For injecting the events, these coordinates must be mapped to the display size, which is not known from the Controller anymore (the transformation to apply is now passed directly from the "screen capture" instance, which does not exist if the screen is not captured).

[yuyiqilin]

The socket protocol for the mouse includes parameters for position, such as width, height, and x, y coordinates. Is it possible to perform a simple conversion based on this?

It includes the coordinates, and the video size to which these coordinates are relative.

For injecting the events, these coordinates must be mapped to the display size, which is not known from the Controller anymore (the transformation to apply is now passed directly from the "screen capture" instance, which does not exist if the screen is not captured).

I see. Thank you for your patience

[rom1v]

Actually, I think it's not that hard if done differently than the fix I merged for #5542 (by providing a DisplayData when video is disabled).

I'll check that later.

[yuyiqilin]

Actually, I think it's not that hard if done differently than the fix I merged for #5542 (by providing a DisplayData when video is disabled).

I'll check that later.

It would be great if it could be fixed. I'm looking forward to the update.

[rom1v]

Actually, the current mouse events in the protocol are really bound to the current video stream (that's why there are both the absolute position and the video size in each event).

In #5542, I allowed to support absolute position only (semantically a bit equivalent as if you executed adb shell input …).

But applying display transforms (like -m or other) when there is no display does not make sense in the context of scrcpy. The display size is not even known (and can change on rotation, etc…), the controller is not aware of it.

So I will leave like it is, sorry.

[yuyiqilin]

Actually, the current mouse events in the protocol are really bound to the current video stream (that's why there are both the absolute position and the video size in each event).

In #5542, I allowed to support absolute position only (semantically a bit equivalent as if you executed adb shell input …).

But applying display transforms (like -m or other) when there is no display does not make sense in the context of scrcpy. The display size is not even known (and can change on rotation, etc…), the controller is not aware of it.

So I will leave like it is, sorry.

It's okay. It's not a big problem. Then I'll control it in the original size