Add gamepad support #5270
Add gamepad support #5270
Conversation
|
This would be very cool |
|
I don't have any games with controller support installed, so I tested using https://greggman.github.io/html5-gamepad-test/ and a Gamepad Tester app. OS: Arch Linux (up-to-date) What works:
What kinda works:
What doesn't work:
|
|
Thank you very much!
Oh, I wrote in the wrong endianness: diff --git a/app/src/hid/hid_gamepad.c b/app/src/hid/hid_gamepad.c
index ca4dc67be..f78cfec73 100644
--- a/app/src/hid/hid_gamepad.c
+++ b/app/src/hid/hid_gamepad.c
@@ -322,12 +322,12 @@ sc_hid_gamepad_event_from_slot(uint16_t hid_id,
uint8_t *data = hid_input->data;
// Buttons are individual bits in the report descriptor, so the resulting
// u16 must be written in little-endian
- sc_write16be(data, slot->axis_left_x);
- sc_write16be(data + 2, slot->axis_left_y);
- sc_write16be(data + 4, slot->axis_right_x);
- sc_write16be(data + 6, slot->axis_right_y);
- sc_write16be(data + 8, slot->axis_left_trigger);
- sc_write16be(data + 10, slot->axis_right_trigger);
+ sc_write16le(data, slot->axis_left_x);
+ sc_write16le(data + 2, slot->axis_left_y);
+ sc_write16le(data + 4, slot->axis_right_x);
+ sc_write16le(data + 6, slot->axis_right_y);
+ sc_write16le(data + 8, slot->axis_left_trigger);
+ sc_write16le(data + 10, slot->axis_right_trigger);
sc_write16le(data + 12, slot->buttons & SC_HID_BUTTONS_MASK);
data[14] = sc_hid_gamepad_get_dpad_value(slot->buttons);
}I wondered why it did not affect analog sticks, the reason is because the physical source gives a 8-bit value, so when converted to a 16-bit value by SDL it gives 2 identical 8-bit values, so big/little endian makes no difference. (with this change, it takes values between 50% and 100%, there is still a minor fix to do to make it work between 0% and 100%) EDIT: one more diff to fix the trigger values: diff --git a/app/src/hid/hid_gamepad.c b/app/src/hid/hid_gamepad.c
index 97596e5bd..3dc342741 100644
--- a/app/src/hid/hid_gamepad.c
+++ b/app/src/hid/hid_gamepad.c
@@ -76,8 +76,8 @@ static const uint8_t SC_HID_GAMEPAD_REPORT_DESC[] = {
0x09, 0xC4,
// Logical Minimum (0)
0x15, 0x00,
- // Logical Maximum (65535)
- 0x27, 0xFF, 0xFF, 0x00, 0x00,
+ // Logical Maximum (32767)
+ 0x26, 0xFF, 0x7F,
// Report Size (16)
0x75, 0x10,
// Report Count (2)
@@ -418,26 +418,28 @@ sc_hid_gamepad_generate_input_from_axis(struct sc_hid_gamepad *hid,
struct sc_hid_gamepad_slot *slot = &hid->slots[slot_idx];
- // [-32768 to 32767] -> [0 to 65535]
- uint16_t value = ((int32_t) event->value) + 0x8000;
+// [-32768 to 32767] -> [0 to 65535]
+#define AXIS_RESCALE(V) (uint16_t) (((int32_t) V) + 0x8000)
switch (event->axis) {
case SC_GAMEPAD_AXIS_LEFTX:
- slot->axis_left_x = value;
+ slot->axis_left_x = AXIS_RESCALE(event->value);
break;
case SC_GAMEPAD_AXIS_LEFTY:
- slot->axis_left_y = value;
+ slot->axis_left_y = AXIS_RESCALE(event->value);
break;
case SC_GAMEPAD_AXIS_RIGHTX:
- slot->axis_right_x = value;
+ slot->axis_right_x = AXIS_RESCALE(event->value);
break;
case SC_GAMEPAD_AXIS_RIGHTY:
- slot->axis_right_y = value;
+ slot->axis_right_y = AXIS_RESCALE(event->value);
break;
case SC_GAMEPAD_AXIS_LEFT_TRIGGER:
- slot->axis_left_trigger = value;
+ // Trigger is always positive between 0 and 32767
+ slot->axis_left_trigger = MAX(0, event->value);
break;
case SC_GAMEPAD_AXIS_RIGHT_TRIGGER:
- slot->axis_right_trigger = value;
+ // Trigger is always positive between 0 and 32767
+ slot->axis_right_trigger = MAX(0, event->value);
break;
default:
return false;
Hmm, I cannot reproduce. It looks like control message corruption. Please run Can you reproduce over AOA?
Does it crash like a segfault (on the client)? Or with an exception on the server? Please build and run with: meson setup x -Db_sanitize=address
ninja -Cx
./run x -G --no-audio |
|
Hi @rom1v, Thank you very much for your work you are doing, I tried the binary you released but when I try to start the session it gives me permission error: C:\Users\super\Downloads\scrcpy-win64-gamepad_draft1\scrcpy-win64-gamepad_draft1>scrcpy.exe -G
scrcpy 2.6.1 <https://github.com/Genymobile/scrcpy>
INFO: ADB device found:
INFO: --> (tcpip) youyeetoo-r1-v3:5555 device redroid13_arm64
C:\Users\super\Downloads\scrcpy-win64-gamepad_draft1\scrcp... file pushed, 0 skipped. 85.9 MB/s (71328 bytes in 0.001s)
[server] INFO: Device: [redroid] redroid redroid13_arm64 (Android 13)
INFO: Renderer: direct3d
INFO: Texture: 1080x1920
[server] ERROR: Controller error
java.io.IOException: android.system.ErrnoException: open failed: EACCES (Permission denied)
at com.genymobile.scrcpy.control.UhidManager.open(UhidManager.java:68)
at com.genymobile.scrcpy.control.Controller.handleEvent(Controller.java:213)
at com.genymobile.scrcpy.control.Controller.control(Controller.java:102)
at com.genymobile.scrcpy.control.Controller.lambda$start$0$com-genymobile-scrcpy-control-Controller(Controller.java:110)
at com.genymobile.scrcpy.control.Controller$$ExternalSyntheticLambda0.run(D8$$SyntheticClass:0)
at java.lang.Thread.run(Thread.java:1012)
Caused by: android.system.ErrnoException: open failed: EACCES (Permission denied)
at libcore.io.Linux.open(Native Method)
at libcore.io.ForwardingOs.open(ForwardingOs.java:563)
at libcore.io.BlockGuardOs.open(BlockGuardOs.java:274)
at android.system.Os.open(Os.java:494)
at com.genymobile.scrcpy.control.UhidManager.open(UhidManager.java:51)
... 5 more
WARN: Device disconnected
C:\Users\super\Downloads\scrcpy-win64-gamepad_draft1\scrcpy-win64-gamepad_draft1>How can fix it? |
|
Oh, you don't have the permissions to open /dev/uhid on your device. Typically, this happens on old Android versions, but you are on Android 13. I have no solution for UHID. However, maybe AOA works over USB ( |
|
Rebuilt with
When pressing either the touchpad or the mute button, I get:
The first two lines show up on press, the other two on release. So the crash seems to be due to trying to read from an empty buffer (just a guess, I didn't look at the code)? We should just ignore these.
No, AOA seems fine (other than the issue with the triggers). |
|
I reworked the PR to fix all known issues. I also updated the binaries for Windows in the first post (draft 2). Thank you for your feedback ❤️ |
Everything's working correctly for me with With
|
|
This is how my gamepad rumble currently works: diff --git a/app/src/scrcpy.c b/app/src/scrcpy.c
index c45f88ad..8509b32b 100644
--- a/app/src/scrcpy.c
+++ b/app/src/scrcpy.c
@@ -756,6 +756,7 @@ aoa_complete:
if (options->gamepad_input_mode == SC_GAMEPAD_INPUT_MODE_UHID) {
sc_gamepad_uhid_init(&s->gamepad_uhid, &s->controller);
gp = &s->gamepad_uhid.gamepad_processor;
+ uhid_gamepad = &s->gamepad_uhid;
}
struct sc_uhid_devices *uhid_devices = NULL;
diff --git a/app/src/uhid/gamepad_uhid.c b/app/src/uhid/gamepad_uhid.c
index 5f19a9d0..5a85816d 100644
--- a/app/src/uhid/gamepad_uhid.c
+++ b/app/src/uhid/gamepad_uhid.c
@@ -119,7 +119,15 @@ sc_gamepad_uhid_process_hid_output(struct sc_gamepad_uhid *gamepad,
LOGI("==== HID output [%" PRIu16 "]", hid_id);
}
- // TODO
+ struct sc_hid_gamepad *gp = &gamepad->hid;
+ struct sc_hid_gamepad_slot* slot = &gp->slots[hid_id - SC_HID_ID_GAMEPAD_FIRST];
+ SDL_GameController *gc = SDL_GameControllerFromInstanceID(slot->gamepad_id);
+ // | | HID Report | SDL Parameter |
+ // |-----------|------------|---------------|
+ // | Intensity | 0 - 100 | 0 - 65535 |
+ // | Duration | 0 - 255 | 0 - 1000 |
+ SDL_GameControllerRumble(gc, data[4] * 0xFFFF / 100, data[5] * 0xFFFF / 100, data[6] * 1000 / 0xFF);
}
void
diff --git a/server/src/main/java/com/genymobile/scrcpy/control/UhidManager.java b/server/src/main/java/com/genymobile/scrcpy/control/UhidManager.java
index 408dbf5d..2db043bc 100644
--- a/server/src/main/java/com/genymobile/scrcpy/control/UhidManager.java
+++ b/server/src/main/java/com/genymobile/scrcpy/control/UhidManager.java
@@ -25,6 +25,7 @@ public final class UhidManager {
private static final int UHID_INPUT2 = 12;
// Linux: include/uapi/linux/input.h
+ private static final short BUS_BLUETOOTH = 0x05;
private static final short BUS_VIRTUAL = 0x06;
private static final int SIZE_OF_UHID_EVENT = 4380; // sizeof(struct uhid_event)
@@ -174,9 +175,9 @@ public final class UhidManager {
buf.put("scrcpy".getBytes(StandardCharsets.US_ASCII));
buf.put(empty, 0, 256 - "scrcpy".length());
buf.putShort((short) reportDesc.length);
- buf.putShort(BUS_VIRTUAL);
- buf.putInt(0); // vendor id
- buf.putInt(0); // product id
+ buf.putShort(BUS_BLUETOOTH);
+ buf.putInt(0x045e); // product id
+ buf.putInt(0x02fd); // vendor id
buf.putInt(0); // version
buf.putInt(0); // country;
buf.put(reportDesc);
How it worksIt uses this Linux driver: https://github.com/torvalds/linux/blob/d1f2d51b711a3b7f1ae1b46701c769c1d580fa7f/drivers/hid/hid-microsoft.c#L450-L461, so the device bus must be Bluetooth and the vender ID/product ID must match one of the list items. I changed the server to always send those values, but this will break HID keyboards and mice, so it should be specified by the client. Output report formatThis driver hard coded the output report format: https://github.com/torvalds/linux/blob/d1f2d51b711a3b7f1ae1b46701c769c1d580fa7f/drivers/hid/hid-microsoft.c#L44-L57. So the report descriptor (the It doesn't support trigger rumbles (byte 2 and 3 in the output report) (notice the Change to back buttonOne side effect of using this driver is that the Back/Select button no longer works using button number 10, it must be set using the The patch above didn't include this change. Other implementationsI believe there are other methods to support rumbles, for example with proper HID Also I believe wired Xbox/PlayStation controllers uses other specific drivers (and packet formats) to read inputs and play rumbles, but those can't be simulated by UHID. |
|
I also tested two Xbox controllers on Windows and it works:
|
OK, I cannot reproduce with my device/gamepad, but we would need an input resampler to handle this case: #3088. |
|
@yume-chan Thank you very much! However, I fail to generate any rumble request (HID output) from Android. I tested for example with "Game Controller Tester" (
I tested from branch (Btw, rumble will not be possible over AOA, since HID output is not supported.) |
|
Im on A14, UNICA Rom (OneUI 6.1) Here's some feedback:
Both "Triggers" start points are 0.5. Left depresses LTRIGGER, Right presses LTRIGGER. ive had this problem before on OTG but reversed (so the Triggers would act as the right stick instead) and i fixed that by modifying the "Generic.kl" in /system/usr/keylayout (ive tried vendor specific .kl many times, it just wouldn't work) so im guessing that's why its doing this. this is the code in my modified Generic.kl if you need it:
Other than that, all buttons work fine on AOA and im pretty excited for this, thanks for dropping test binaries. and i hope my gibberish is useful to you lol |
Did you test draft1 or draft2? These two problems you report for AOA should be fixed in draft2, but the behavior should be the same for UHID (broken in draft1, fixed in draft2).
Cannot reproduce. Probably related to the previous errors in draft1.
Thank you, fixed 👍
Oh thank you, stupid mistake, I assigned the same HID_ID to the mouse and to the first controller (off-by-one error in the hardcoded values 😄). EDIT: I just published a draft3 binary. |
Is your device running Android 12+? Controller rumble support was added in Android 12. This app works for me on an Android 13 device, but it always vibrates both motors no matter which one I choose (it also does this when my Xbox controller is connected to the device directly over Bluetooth). Judging from the permission prompt it shows on connection, it must use Android USB API to directly talk to controllers, so it can support all motors on all Android versions. (I think it's not possible to emulate USB devices through software) I test my implementation using Chrome and this code I wrote: https://codepen.io/yume-chan/pen/qBzLGNd. Move the sticks to control vibration. On PC it supports all four motors, but on Android it only supports two. (again, same as when the controller is connected to the device over Bluetooth or USB). My code can vibrate left and right motors separately. I checked again, at least for Xbox controllers, neither |
Yes, Android 14. I just tested, in fact it works like you say on a Pixel 8 with Android 14, but not on a Samsung Galaxy Tab with Android 14. Btw, if I set the vid:pid to 02fd:045e, the device is shown as a
Given that:
I think I will not support rumble for now. Maybe in the future if things improve. Thank you very much for your research and documentation in this thread 👍 |
|
Do you know any (free) Android game which supports 2 or more gamepads (a two-player fighting game for example)? |
afaik "Vita Fighters" supports 2 controllers |
|
@Withoutruless Thank you, it works 👍 |
So i copied the .kl for the PDP Xbox One Controller (should be the same for X360 and the normal XOne too) and made a custom .kl called "Vendor_ffff_Product_ffff.kl" (which is what scrcpy uses on AOA) and changed the text from: To: so now i can use both AOA and UHID just fine. everything else i mentioned has been fixed so thank you for that haha. |
Where do you copy that file? Is there something to change in this PR so that it works the same "out-of-the-box"? |
i copy that file to "/system/usr/keylayout" and it changes the way android handles input for the device with that specific vendor & product id.
if its possible to change the vendor & product id so it matches something like an Xbox 360 Wired Controller then that'd make android handle the received input differently and you wouldnt have to accomodate for the generic.kl's bugged RT & RStick which im assuming you do by sending: LTRIGGER as axis 0x0a instead of 0x02 Z as axis 0x02 instead of 0x03 so you could try changing those values and the emulated controller vendor & product id and see if it works. also would it be possible to add an optional background capture for UHID & AOA so it captures input regardless if the window is in focus or not? |
That's what I did here: #5270 (comment) I might use the real vid:pid of the physical device, but that might also cause inconsistencies between controllers I guess (I pass a custom report descriptor).
The problem is the same as with mouse and keyboard: I can't receive input events from SDL without a focused window (AFAIK). |
Introduce a gamepad processor trait, similar to the keyboard processor and mouse processor traits. Handle gamepad events received from SDL, convert them to scrcpy-specific gamepad events, and forward them to the gamepad processor. Further commits will provide AOA and UHID implementations of the gamepad processor trait. PR #5270 <#5270> Co-authored-by: Luiz Henrique Laurini <[email protected]>
Trigger SDL_CONTROLLERDEVICEADDED for all gamepads already connected when scrcpy starts. We want to handle both the gamepads initially connected and the gamepads connected while scrcpy is running. This is not racy, because this event may not be trigged automatically until SDL events are "pumped" (SDL_PumpEvents/SDL_WaitEvent). PR #5270 <#5270>
There was a registration mechanism to listen to HID outputs with a specific HID id. However, the UHID gamepad processor handles several ids, so it cannot work. We could complexify the registration mechanism, but instead, directly dispatch to the expected processor based on the UHID id. Concretely, instead of passing a sc_uhid_devices instance to construct a sc_keyboard_uhid, so that it can register itself, construct the sc_uhid_devices with all the UHID instances (currently only sc_keyboard_uhid) so that it can dispatch HID outputs directly. PR #5270 <#5270>
The sc_uhid_devices instance is initialized only when there is a UHID keyboard. The device message receiver assumed that it could not receive HID output reports without a sc_uhid_devices instance (i.e. without a UHID keyboard), but in practice, a UHID driver implementation on the device may decide to send UHID output reports for mouse or for gamepads (and we must just ignore them). So remove the assert(). PR #5270 <#5270>
Capture the gamepads even when the window is not focused. Note: In theory, with this flag set, we could capture gamepad events even without a window (--no-window). In practice, scrcpy still requires a window, because --no-window implies --no-control, and the input manager is owned by the sc_screen instance, which does not exist if there is no window. Supporting this use case would require a lot of refactors. Refs <#5270 (comment)> PR #5270 <#5270> Suggested-by: Luiz Henrique Laurini <[email protected]>
UHID may not work on old Android versions due to permission errors. Mention it in UHID mouse and gamepad documentation (it was already mentioned for UHID keyboard). Refs #4473 comment <#4473 (comment)> PR #5270 <#5270>
|
Ref issue #5362. |
This PR adds support for game controllers (gamepads).
There was a PR (#2130) implementing gamepad support using uinput on the device. However, this required to add a native dependency or to add native code (cf #2130 (comment)).
This PR uses HID instead, and provides support for AOA and UHID (like for keyboard and mouse).
It does not add any dependency, and it also works in OTG mode (via AOA), so even without USB debugging.
To enable it:
It is not enabled by default because UHID is not supported on all devices (especially with old Android versions, I don't remember which version exactly, due to a permission error).
I tested with PS4 and PS5 controllers connected by USB on my laptop, with gamepad tester apps, it seems to work.
Rumble is not implemented, because I don't know how to make it work. It probably requires to add the correct bytes to the report descriptor, and to parse the received HID outputs (the mechanism to receive them is already there). @yume-chan (or others) your help is welcome 😉
@LHLaurini I marked you as co-author of the commit named
Handle SDL gamepad events(I don't give the SHA1 because it will change on every rebase), because your PR helped me to write this commit 👍TODO
RumbleFixes #99