libusb backend for Windows and macOS #319

mcuee · 2021-08-08T02:25:00Z

As discussed in #61 , it may make sense to create a new libusb backend to remove some limitations of the native HID APIs under Windows. The users will need to use WinUSB driver to replace the native HID driver in order to make use of this (if using libusb HID backend, then the limitation still applies).

I am not so sure if there is any benefit to extend this to macOS, but it can be done as well with the latest kernel driver detaching function of libusb under macOS.

mcuee · 2021-08-08T02:26:53Z

Reference discussion for macOS: libusb/libusb#911

mcuee · 2021-08-08T02:27:56Z

Quick hack for mac/hid.c
Codes from:
https://blog.albertarmea.com/post/47089939939/using-pthreadbarrier-on-mac-os-x

diff --git a/libusb/hid.c b/libusb/hid.c
index 0b8aa1e..cb4a1e2 100644
--- a/libusb/hid.c
+++ b/libusb/hid.c
@@ -112,6 +112,75 @@ static int pthread_barrier_wait(pthread_barrier_t *barrier)
 
 #endif
 
+#ifdef __APPLE__
+
+#ifndef PTHREAD_BARRIER_H_
+#define PTHREAD_BARRIER_H_
+
+#include <pthread.h>
+#include <errno.h>
+
+typedef int pthread_barrierattr_t;
+typedef struct
+{
+    pthread_mutex_t mutex;
+    pthread_cond_t cond;
+    int count;
+    int tripCount;
+} pthread_barrier_t;
+
+
+int pthread_barrier_init(pthread_barrier_t *barrier, const pthread_barrierattr_t *attr, unsigned int count)
+{
+    if(count == 0)
+    {
+        errno = EINVAL;
+        return -1;
+    }
+    if(pthread_mutex_init(&barrier->mutex, 0) < 0)
+    {
+        return -1;
+    }
+    if(pthread_cond_init(&barrier->cond, 0) < 0)
+    {
+        pthread_mutex_destroy(&barrier->mutex);
+        return -1;
+    }
+    barrier->tripCount = count;
+    barrier->count = 0;
+
+    return 0;
+}
+
+int pthread_barrier_destroy(pthread_barrier_t *barrier)
+{
+    pthread_cond_destroy(&barrier->cond);
+    pthread_mutex_destroy(&barrier->mutex);
+    return 0;
+}
+
+int pthread_barrier_wait(pthread_barrier_t *barrier)
+{
+    pthread_mutex_lock(&barrier->mutex);
+    ++(barrier->count);
+    if(barrier->count >= barrier->tripCount)
+    {
+        barrier->count = 0;
+        pthread_cond_broadcast(&barrier->cond);
+        pthread_mutex_unlock(&barrier->mutex);
+        return 1;
+    }
+    else
+    {
+        pthread_cond_wait(&barrier->cond, &(barrier->mutex));
+        pthread_mutex_unlock(&barrier->mutex);
+        return 0;
+    }
+}
+
+#endif // PTHREAD_BARRIER_H_
+#endif // __APPLE__
+
 #ifdef __cplusplus
 extern "C" {
 #endif

mcuee · 2021-08-08T02:28:44Z

Quick hack for Makefile.macOS

###########################################
# Simple Makefile for HIDAPI test program
#
# Alan Ott
# Signal 11 Software
# 2010-06-01
###########################################

all: hidtest-libusb libs

libs: libhidapi-libusb.dylib

CC       ?= gcc
CFLAGS   ?= -Wall -g -fpic

LDFLAGS  ?= -Wall -g

COBJS_LIBUSB = hid.o
COBJS = $(COBJS_LIBUSB) ../hidtest/test.o
OBJS      = $(COBJS)
LIBS_USB  = `pkg-config libusb-1.0 --libs` -liconv -lpthread
LIBS      = $(LIBS_USB)
INCLUDES ?= -I../hidapi `pkg-config libusb-1.0 --cflags`


# Console Test Program
hidtest-libusb: $(COBJS)
	$(CC) $(LDFLAGS) $^ $(LIBS_USB) -o $@

# Shared Libs
libhidapi-libusb.dylib: $(COBJS_LIBUSB)
	$(CC) $(LDFLAGS) $(LIBS_USB) -shared -fpic -Wl,-install_name,[email protected] $^ -o $@

# Objects
$(COBJS): %.o: %.c
	$(CC) $(CFLAGS) -c $(INCLUDES) $< -o $@


clean:
	rm -f $(OBJS) hidtest-libusb libhidapi-libusb.so ../hidtest/hidtest.o

.PHONY: clean libs

mcuee · 2021-08-08T05:31:54Z

Runlog for macOS with hidtest using the quick mod libusb backend.

hidapi_test/libusb on  master [!?] ❯ sudo ./hidtest-libusb 
hidapi test/example tool. Compiled with hidapi version 0.10.1, runtime version 0.10.1.
Compile-time version matches runtime version of hidapi.

Device Found
  type: 04d8 003f
  path: 0002:001e:00
  serial_number: (null)
  Manufacturer: Microchip Technology Inc.
  Product:      Simple HID Device Demo
  Release:      2
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c52b
  path: 0002:000a:00
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2410
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c52b
  path: 0002:000a:01
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2410
  Interface:    1
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c52b
  path: 0002:000a:02
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2410
  Interface:    2
  Usage (page): 0x0 (0x0)

Device Found
  type: 047f c025
  path: 0002:0008:03
  serial_number: CB13A3E40E8E47D6A40769C27E90A38E
  Manufacturer: Plantronics
  Product:      Plantronics C320-M
  Release:      135
  Interface:    3
  Usage (page): 0x0 (0x0)

Manufacturer String: Microchip Technology Inc.
Product String: Simple HID Device Demo
Serial Number String: (208) ?
Indexed String 1: Microchip Technology Inc.
Unable to send a feature report.
Unable to get a feature report.
hid_error is not implemented yetwaiting...
waiting...
waiting...
Unable to read()
Data read:

mcuee · 2021-08-09T00:18:12Z

Under Windows, here is the output, with a quick mod of the following file.
https://gitlab.com/CalcProgrammer1/OpenRGB/-/blob/master/dependencies/hidapi/hidapi.c

I choose to use usbdk backend so that I do not need to replace the HID driver with WinUSB. And it may be able to find more devices like USB keyboard and mouse if they are present.

modified files and Windows 64bit binaries are included in the following zip file.
hidapi_libusb.zip

C:\work\hid\hidapi\libusb [master ≡ +7 ~1 -0 !]> .\hidtest-libusb.exe
hidapi test/example tool. Compiled with hidapi version 0.11.0, runtime version 0.10.1.
Compile-time version is different than runtime version of hidapi.
]nDevice Found
  type: 046d c534
  path: 0001:0008:00
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2901
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c534
  path: 0001:0008:01
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2901
  Interface:    1
  Usage (page): 0x0 (0x0)

Device Found
  type: 047f c056
  path: 0001:001d:03
  serial_number: D1CEC32927974D5F9BD6B2AEBF2EA8E3
  Manufacturer: Plantronics
  Product:      Plantronics Blackwire 3220 Series
  Release:      210
  Interface:    3
  Usage (page): 0x0 (0x0)

Device Found
  type: 04d8 003f
  path: 0001:001c:00
  serial_number: (null)
  Manufacturer: Microchip Technology Inc.
  Product:      Simple HID Device Demo
  Release:      2
  Interface:    0
  Usage (page): 0x0 (0x0)

Manufacturer String: Microchip Technology Inc.
Product String: Simple HID Device Demo
Serial Number String: (1033)
Indexed String 1: Microchip Technology Inc.
Unable to send a feature report.
Unable to get a feature report.
(null)Unable to write()
Error: (null)
Unable to write() (2)
waiting...
waiting...
waiting...
waiting...


C:\work\hid\hidapi\libusb [master ≡ +7 ~1 -0 !]> .\hidtest-libusb.exe
hidapi test/example tool. Compiled with hidapi version 0.11.0, runtime version 0.10.1.
Compile-time version is different than runtime version of hidapi.
]nDevice Found
  type: 413c b06e
  path: 0003:0007:00
  serial_number: (null)
  Manufacturer: (null)
  Product:
  Release:      101
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c534
  path: 0001:0002:00
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2901
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c534
  path: 0001:0002:01
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2901
  Interface:    1
  Usage (page): 0x0 (0x0)

Device Found
  type: 047f c056
  path: 0003:001c:03
  serial_number: D1CEC32927974D5F9BD6B2AEBF2EA8E3
  Manufacturer: Plantronics
  Product:      Plantronics Blackwire 3220 Series
  Release:      210
  Interface:    3
  Usage (page): 0x0 (0x0)

Device Found
  type: 413c 2107
  path: 0003:000d:00
  serial_number: (null)
  Manufacturer: DELL
  Product:      Dell USB Entry Keyboard
  Release:      178
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 413c b06f
  path: 0003:000a:00
  serial_number: (null)
  Manufacturer: (null)
  Product:
  Release:      101
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c077
  path: 0003:000e:00
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Optical Mouse
  Release:      7200
  Interface:    0
  Usage (page): 0x0 (0x0)

unable to open device

mcuee · 2021-08-11T07:42:54Z

Of course, it is also possible not to use usbdk, just comment out the following line in hid.c.

		//libusb_set_option(NULL, LIBUSB_OPTION_USE_USBDK);

hidapi_libusb2.zip

Youw · 2021-08-12T08:37:02Z

Instead of using "hacks" to make it functional I'd rather suggest to refactor libusb backend of HIDAPI and switch to C11 instead of using POSIX/pthread API.
That way we will have a clean implementation compliant with the C Standard.

Thoughts?

mcuee · 2021-08-12T09:08:14Z

Yes, that will be great.

Haha, my hack is just a quick demo to show that it is possible.

mcuee · 2021-08-27T02:52:11Z

In order for your USB HID device to be found by the libusb backend (hidapi_libusb2.zip), there are two possibilities.

Take note this is only applicable to device listed under Windows Device Manager as "Human Interface Devices" --> "USB Input Device". Keyboard/mouse and Bluetooth/I2C/SPI HID devices are not supported.

keep using the default kernel HID driver, then it has same (actually more) limitation as HIDAPI with the normal Windows HID API backend. But it is not recommended to use libusb HID backend because of all the limitations. So we recommend you to use HIDAPI instead.

https://github.com/libusb/libusb/wiki/Windows#known-restrictions
HID keyboards and mice cannot be accessed using the native HID driver as Windows reserves exclusive access to them.
Multiple HID top level collections are currently not supported (only the first top level collection will be used).

change the USB HID driver to WinUSB. In that case, certain limitations will be lifted. For example, now you can use control transfer to get the HID report descriptors.

mcuee · 2021-08-27T02:58:19Z

hidapi_libusb.zip uses the USBDK backend of the libusb, so it may be able to find more devices than hidapi_libusb2.zip. For example, it may be able to find some USB mice/keyboards. But it requires the installation of usbdk. libusb usbdk backend may be less stable than using WinUSB backend.
https://github.com/daynix/UsbDk/releases

mcuee · 2021-08-31T01:46:05Z

64bit Windows binary with hidapitester here (not using usbdk). libusb-1.0.dll is a mod version of 1.0.24.11650 with pull request libusb/libusb#986 to fix the Windows HID backend issue.
https://github.com/mcuee/hidapitester/releases/tag/v8610dc6

mcuee · 2023-02-25T03:26:39Z

nstead of using "hacks" to make it functional I'd rather suggest to refactor libusb backend of HIDAPI and switch to C11 instead of using POSIX/pthread API. That way we will have a clean implementation compliant with the C Standard.

@Youw

Since I do not know much about pthread and C++11 threading, and since ChatGPT is now popular, I asked ChatGPT to do the job.

Unfortuantely ChatGPT is not able to finish the full conversion and always got stuck.

(Edit: delete the code as it is not good).

Youw · 2023-02-25T08:16:26Z

Loool :)
Writing it in C++11 would be even easier, for sure)

mcuee · 2023-02-25T10:13:31Z

Loool :) Writing it in C++11 would be even easier, for sure)

Glad to hear that.

Interestingly I ask ChatGPT to convert the simple libusb stress_mt.c pthread based test codes to C++11 threading and it has no problems. But the result codes do not work under WIndows but work under Linux. Maybe libusb Windows has some issues under stress.

https://github.com/libusb/libusb/blob/master/tests/stress_mt.c

Enhance libusb stress test libusb#1128 (comment)

However, if I ask it to convert into C11, the result codes do not even compile since MinGW does not support C11 <threads>.

Enhance libusb stress test libusb#1128 (comment)

Youw · 2023-02-25T11:57:33Z

MinGW does not support C11

That's a bummer :(
I never actually used C11 threads. I only assumed, that compilers that have C++11 threads would have C implementation as well...

Writing an entire backend in C++ has many proc and cons. One of the most important ones - dependency on C++ Standard library.
And I'm pretty sure there will be some part of the (C) community that will not support such huge change as switching from C to C++.

But maybe, having an experimental (additional, not replacement) C++ port of libusb backend of HIDAPI - should be a "fun project" at least.
And I'm not talking about "just replacing pthreads with C++11 threads". I'm talking of taking a full advantage of Modern C++ (yes - that is a "thing").

mcuee · 2023-02-27T00:12:22Z

FYI: Apple clang does not support C11 threading either.

Ref:
https://stackoverflow.com/questions/16244153/clang-c11-threads-h-not-found

mcuee · 2023-05-11T08:02:01Z

I think in the end, this is not an important work (not much real use in reality) and therefore I will close it for now.

mcuee · 2023-05-14T02:07:29Z

Re-open as a low priority item.

mcuee · 2024-10-25T14:19:29Z

From #706, comment by mandar1jn.

I'm working on a program which would benefit from using the libusb backend of hidapi due to needing the ability to send control transfers. I was wondering if the libusb backend is available on windows. Libusb works on windows, but the build instructions never mention how to or if it's even possible and I had no luck getting it to compile myself.

mandar1jn · 2024-10-25T14:33:26Z

I'll add some context to this: I'm working with skylanders portals which are HID devices, but they recieve commands over control transfers, something you wouldn't normally do for an HID device, but what can you do about it. My current code uses hidapi with a workaround to get the device handle from hidapi and manually runs the required win32 functions, but I'd prefer to use libusb (I can still extract the libusb handle from what I saw) and then use that functionality to send control transfers, and maybe utilize the hotplug functionality

Youw · 2024-11-21T12:36:41Z

Right now there is no official support of libusb backend for Windows or macOS, but there steps has been made towards that direction with the help of the community.
You may try in scope of your project (or even contribute if you feel like it) to define the HIDAPI thread abstraction for libusb and define the HIDAPI_THREAD_MODEL_INCLUDE pointing to that implementation.
See https://github.com/libusb/hidapi/pull/582/files for details.

mcuee added libusb Related to libusb backend macOS Related to macOS backend Windows Related to Windows backend labels Aug 8, 2021

mcuee mentioned this issue Aug 26, 2021

hid_enumerate method issue on windows #325

Closed

mcuee added the enhancement New feature or request label Sep 7, 2021

mcuee mentioned this issue Feb 25, 2023

Enhance libusb stress test libusb/libusb#1128

Closed

mcuee mentioned this issue Apr 15, 2023

hid_device_info->interface_number zero on macOS 13.3 #531

Closed

mcuee closed this as not planned Won't fix, can't repro, duplicate, stale May 11, 2023

mcuee changed the title ~~libusb backend for Windows and macOS~~ libusb backend for Windows and macOS (Low Priority) May 14, 2023

mcuee reopened this May 14, 2023

mcuee mentioned this issue May 14, 2023

HID reports of more than 64 bytes and libusb backend #274

Open

mcuee mentioned this issue May 27, 2023

Building libusb support on Windows and macOS #581

Closed

mcuee mentioned this issue Oct 25, 2024

libusb backend on windows #706

Closed

mcuee changed the title ~~libusb backend for Windows and macOS (Low Priority)~~ libusb backend for Windows and macOS Oct 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

libusb backend for Windows and macOS #319

libusb backend for Windows and macOS #319

mcuee commented Aug 8, 2021 •

edited

Loading

mcuee commented Aug 8, 2021

mcuee commented Aug 8, 2021 •

edited

Loading

mcuee commented Aug 8, 2021

mcuee commented Aug 8, 2021

mcuee commented Aug 9, 2021 •

edited

Loading

mcuee commented Aug 11, 2021

Youw commented Aug 12, 2021 •

edited

Loading

mcuee commented Aug 12, 2021

mcuee commented Aug 27, 2021

mcuee commented Aug 27, 2021

mcuee commented Aug 31, 2021 •

edited

Loading

mcuee commented Feb 25, 2023 •

edited

Loading

Youw commented Feb 25, 2023

mcuee commented Feb 25, 2023 •

edited

Loading

Youw commented Feb 25, 2023

mcuee commented Feb 27, 2023 •

edited

Loading

mcuee commented May 11, 2023

mcuee commented May 14, 2023

mcuee commented Oct 25, 2024

mandar1jn commented Oct 25, 2024

Youw commented Nov 21, 2024

libusb backend for Windows and macOS #319

libusb backend for Windows and macOS #319

Comments

mcuee commented Aug 8, 2021 • edited Loading

mcuee commented Aug 8, 2021

mcuee commented Aug 8, 2021 • edited Loading

mcuee commented Aug 8, 2021

mcuee commented Aug 8, 2021

mcuee commented Aug 9, 2021 • edited Loading

mcuee commented Aug 11, 2021

Youw commented Aug 12, 2021 • edited Loading

mcuee commented Aug 12, 2021

mcuee commented Aug 27, 2021

mcuee commented Aug 27, 2021

mcuee commented Aug 31, 2021 • edited Loading

mcuee commented Feb 25, 2023 • edited Loading

Youw commented Feb 25, 2023

mcuee commented Feb 25, 2023 • edited Loading

Youw commented Feb 25, 2023

mcuee commented Feb 27, 2023 • edited Loading

mcuee commented May 11, 2023

mcuee commented May 14, 2023

mcuee commented Oct 25, 2024

mandar1jn commented Oct 25, 2024

Youw commented Nov 21, 2024

mcuee commented Aug 8, 2021 •

edited

Loading

mcuee commented Aug 8, 2021 •

edited

Loading

mcuee commented Aug 9, 2021 •

edited

Loading

Youw commented Aug 12, 2021 •

edited

Loading

mcuee commented Aug 31, 2021 •

edited

Loading

mcuee commented Feb 25, 2023 •

edited

Loading

mcuee commented Feb 25, 2023 •

edited

Loading

mcuee commented Feb 27, 2023 •

edited

Loading