Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve mouse events and add secondary bindings #5076

Merged
merged 8 commits into from
Jul 11, 2024
Merged

Improve mouse events and add secondary bindings #5076

merged 8 commits into from
Jul 11, 2024

Conversation

rom1v
Copy link
Collaborator

@rom1v rom1v commented Jul 9, 2024

On Android, a MotionEvent may use either a MOUSE or FINGER as source. Since scrcpy 1.0, it used a FINGER. But when --forward-all-clicks (now deprecated in favor of --mouse-bind=++++) was added, this caused a problem: it was not possible to press secondary clicks with a FINGER, so if this flag was set, it used a MOUSE (c7b1d0e):

if (pointerId == POINTER_ID_MOUSE || pointerId == POINTER_ID_VIRTUAL_MOUSE) {
// real mouse event (forced by the client when --forward-on-click)
pointerProperties[pointerIndex].toolType = MotionEvent.TOOL_TYPE_MOUSE;
source = InputDevice.SOURCE_MOUSE;
pointer.setUp(buttons == 0);
} else {
// POINTER_ID_GENERIC_FINGER, POINTER_ID_VIRTUAL_FINGER or real touch from device
pointerProperties[pointerIndex].toolType = MotionEvent.TOOL_TYPE_FINGER;
source = InputDevice.SOURCE_TOUCHSCREEN;
// Buttons must not be set for touch events
buttons = 0;
pointer.setUp(action == MotionEvent.ACTION_UP);
}

FINGER was still useful for simulating pinch-to-zoom, rotate and tilt, because a double mouse pointer does not always work on all devices.

The fact that the decision to use either MOUSE or FINGER depends on the specific --mouse-bind value is weird, and cause issues: #5055 (comment)

The introduction of "mouse hover" in scrcpy v2.5 also causes issues on some devices, because in theory a "mouse hover" event should be with a MOUSE source: #5067

To fix both issues, always use a MOUSE source (regardless of the mouse bindings), except when simulating a virtual finger (pinch-to-zoom, rotate and tilt).

Fixes #5067


Then, improve --mouse-bind to support secondary mouse bindings with Shift+click.

It is now possible to pass a sequence of secondary bindings:

--mouse-bind=xxxx:xxxx
             <--> <-->
         primary   secondary
        bindings   bindings

If the second sequence is omitted, then it is the same as the first one.

By default, for SDK mouse, primary bindings trigger shortcuts and secondary bindings forward all clicks.

For AOA and UHID, the default bindings are reversed: all clicks are forwarded by default, whereas pressing Shift+click trigger shortcuts.

scrcpy --mouse-bind=bhsn:++++  # default for SDK
scrcpy --mouse-bind=++++:bhsn  # default for AOA and UHID

Fixes #5055


Here is a binary for Windows:

Thank you for your feedback 😉

rom1v added 8 commits July 11, 2024 12:04
For consistency, never use top-level const for local variables.

PR #5076 <#5076>
(A && !B) || (!A && B) <==> A ^ B

PR #5076 <#5076>
The device source (MOUSE or FINGER) to use depended on whether a
secondary click was possible via mouse bindings.

As a first step, always use a mouse source to break this dependency.
Note that this change might cause regressions in some (unknown) cases
(refs f70359f), but hopefully not.

Further commits will restore a finger source in some specific use cases,
but independent of secondary clicks.

Refs #5055 <#5055>
Fixes #5067 <#5067>
PR #5076 <#5076>
For pinch-to-zoom, rotation and tilt simulation, always use a finger
source (instead of a mouse) for both pointers (the real one and the
simulated one).

A "virtual" mouse does not work on all devices (e.g. on Pixel 8).

PR #5076 <#5076>
The buttons state was tracked by SDL_GetMouseState(), and scrcpy applied
a mask to ignore buttons used for shortcuts.

Instead, track the buttons actually pressed (ignoring shortcuts)
manually, to prepare the introduction of more dynamic mouse shortcuts.

PR #5076 <#5076>
Add secondary bindings (Shift+click) for mouse buttons.

In addition to:

    --mouse-bind=xxxx

It is now possible to pass a sequence of secondary bindings:

    --mouse-bind=xxxx:xxxx
                 <--> <-->
             primary   secondary
            bindings   bindings

If the second sequence is omitted, then it is the same as the first one.

By default, for SDK mouse, primary bindings trigger shortcuts and
secondary bindings forward all clicks.

For AOA and UHID, the default bindings are reversed: all clicks are
forwarded by default, whereas pressing Shift+click trigger shortcuts.

    --mouse-bind=bhsn:++++  # default for SDK
    --mouse-bind=++++:bhsn  # default for AOA and UHID

Refs 035d60c
Refs f5e6b80
Fixes #5055 <#5055>
PR #5076 <#5076>
@rom1v rom1v merged commit 9989668 into dev Jul 11, 2024
FreedomBen pushed a commit to FreedomBen/scrcpy that referenced this pull request Aug 2, 2024
For consistency, never use top-level const for local variables.

PR Genymobile#5076 <Genymobile#5076>
FreedomBen pushed a commit to FreedomBen/scrcpy that referenced this pull request Aug 2, 2024
FreedomBen pushed a commit to FreedomBen/scrcpy that referenced this pull request Aug 2, 2024
FreedomBen pushed a commit to FreedomBen/scrcpy that referenced this pull request Aug 2, 2024
The device source (MOUSE or FINGER) to use depended on whether a
secondary click was possible via mouse bindings.

As a first step, always use a mouse source to break this dependency.
Note that this change might cause regressions in some (unknown) cases
(refs f70359f), but hopefully not.

Further commits will restore a finger source in some specific use cases,
but independent of secondary clicks.

Refs Genymobile#5055 <Genymobile#5055>
Fixes Genymobile#5067 <Genymobile#5067>
PR Genymobile#5076 <Genymobile#5076>
FreedomBen pushed a commit to FreedomBen/scrcpy that referenced this pull request Aug 2, 2024
For pinch-to-zoom, rotation and tilt simulation, always use a finger
source (instead of a mouse) for both pointers (the real one and the
simulated one).

A "virtual" mouse does not work on all devices (e.g. on Pixel 8).

PR Genymobile#5076 <Genymobile#5076>
FreedomBen pushed a commit to FreedomBen/scrcpy that referenced this pull request Aug 2, 2024
FreedomBen pushed a commit to FreedomBen/scrcpy that referenced this pull request Aug 2, 2024
The buttons state was tracked by SDL_GetMouseState(), and scrcpy applied
a mask to ignore buttons used for shortcuts.

Instead, track the buttons actually pressed (ignoring shortcuts)
manually, to prepare the introduction of more dynamic mouse shortcuts.

PR Genymobile#5076 <Genymobile#5076>
FreedomBen pushed a commit to FreedomBen/scrcpy that referenced this pull request Aug 2, 2024
Add secondary bindings (Shift+click) for mouse buttons.

In addition to:

    --mouse-bind=xxxx

It is now possible to pass a sequence of secondary bindings:

    --mouse-bind=xxxx:xxxx
                 <--> <-->
             primary   secondary
            bindings   bindings

If the second sequence is omitted, then it is the same as the first one.

By default, for SDK mouse, primary bindings trigger shortcuts and
secondary bindings forward all clicks.

For AOA and UHID, the default bindings are reversed: all clicks are
forwarded by default, whereas pressing Shift+click trigger shortcuts.

    --mouse-bind=bhsn:++++  # default for SDK
    --mouse-bind=++++:bhsn  # default for AOA and UHID

Refs 035d60c
Refs f5e6b80
Fixes Genymobile#5055 <Genymobile#5055>
PR Genymobile#5076 <Genymobile#5076>
Gottox pushed a commit to Gottox/scrcpy that referenced this pull request Sep 29, 2024
For consistency, never use top-level const for local variables.

PR Genymobile#5076 <Genymobile#5076>
Gottox pushed a commit to Gottox/scrcpy that referenced this pull request Sep 29, 2024
Gottox pushed a commit to Gottox/scrcpy that referenced this pull request Sep 29, 2024
Gottox pushed a commit to Gottox/scrcpy that referenced this pull request Sep 29, 2024
The device source (MOUSE or FINGER) to use depended on whether a
secondary click was possible via mouse bindings.

As a first step, always use a mouse source to break this dependency.
Note that this change might cause regressions in some (unknown) cases
(refs f70359f), but hopefully not.

Further commits will restore a finger source in some specific use cases,
but independent of secondary clicks.

Refs Genymobile#5055 <Genymobile#5055>
Fixes Genymobile#5067 <Genymobile#5067>
PR Genymobile#5076 <Genymobile#5076>
Gottox pushed a commit to Gottox/scrcpy that referenced this pull request Sep 29, 2024
For pinch-to-zoom, rotation and tilt simulation, always use a finger
source (instead of a mouse) for both pointers (the real one and the
simulated one).

A "virtual" mouse does not work on all devices (e.g. on Pixel 8).

PR Genymobile#5076 <Genymobile#5076>
Gottox pushed a commit to Gottox/scrcpy that referenced this pull request Sep 29, 2024
Gottox pushed a commit to Gottox/scrcpy that referenced this pull request Sep 29, 2024
The buttons state was tracked by SDL_GetMouseState(), and scrcpy applied
a mask to ignore buttons used for shortcuts.

Instead, track the buttons actually pressed (ignoring shortcuts)
manually, to prepare the introduction of more dynamic mouse shortcuts.

PR Genymobile#5076 <Genymobile#5076>
Gottox pushed a commit to Gottox/scrcpy that referenced this pull request Sep 29, 2024
Add secondary bindings (Shift+click) for mouse buttons.

In addition to:

    --mouse-bind=xxxx

It is now possible to pass a sequence of secondary bindings:

    --mouse-bind=xxxx:xxxx
                 <--> <-->
             primary   secondary
            bindings   bindings

If the second sequence is omitted, then it is the same as the first one.

By default, for SDK mouse, primary bindings trigger shortcuts and
secondary bindings forward all clicks.

For AOA and UHID, the default bindings are reversed: all clicks are
forwarded by default, whereas pressing Shift+click trigger shortcuts.

    --mouse-bind=bhsn:++++  # default for SDK
    --mouse-bind=++++:bhsn  # default for AOA and UHID

Refs 035d60c
Refs f5e6b80
Fixes Genymobile#5055 <Genymobile#5055>
PR Genymobile#5076 <Genymobile#5076>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
1 participant