libusb backend for Windows and macOS

[mcuee]

As discussed in #61 , it may make sense to create a new libusb backend to remove some limitations of the native HID APIs under Windows. The users will need to use WinUSB driver to replace the native HID driver in order to make use of this (if using libusb HID backend, then the limitation still applies).

I am not so sure if there is any benefit to extend this to macOS, but it can be done as well with the latest kernel driver detaching function of libusb under macOS.

[mcuee]

Reference discussion for macOS: libusb/libusb#911

[mcuee]

Quick hack for mac/hid.c
Codes from:
https://blog.albertarmea.com/post/47089939939/using-pthreadbarrier-on-mac-os-x

diff --git a/libusb/hid.c b/libusb/hid.c
index 0b8aa1e..cb4a1e2 100644
--- a/libusb/hid.c
+++ b/libusb/hid.c
@@ -112,6 +112,75 @@ static int pthread_barrier_wait(pthread_barrier_t *barrier)
 
 #endif
 
+#ifdef __APPLE__
+
+#ifndef PTHREAD_BARRIER_H_
+#define PTHREAD_BARRIER_H_
+
+#include <pthread.h>
+#include <errno.h>
+
+typedef int pthread_barrierattr_t;
+typedef struct
+{
+    pthread_mutex_t mutex;
+    pthread_cond_t cond;
+    int count;
+    int tripCount;
+} pthread_barrier_t;
+
+
+int pthread_barrier_init(pthread_barrier_t *barrier, const pthread_barrierattr_t *attr, unsigned int count)
+{
+    if(count == 0)
+    {
+        errno = EINVAL;
+        return -1;
+    }
+    if(pthread_mutex_init(&barrier->mutex, 0) < 0)
+    {
+        return -1;
+    }
+    if(pthread_cond_init(&barrier->cond, 0) < 0)
+    {
+        pthread_mutex_destroy(&barrier->mutex);
+        return -1;
+    }
+    barrier->tripCount = count;
+    barrier->count = 0;
+
+    return 0;
+}
+
+int pthread_barrier_destroy(pthread_barrier_t *barrier)
+{
+    pthread_cond_destroy(&barrier->cond);
+    pthread_mutex_destroy(&barrier->mutex);
+    return 0;
+}
+
+int pthread_barrier_wait(pthread_barrier_t *barrier)
+{
+    pthread_mutex_lock(&barrier->mutex);
+    ++(barrier->count);
+    if(barrier->count >= barrier->tripCount)
+    {
+        barrier->count = 0;
+        pthread_cond_broadcast(&barrier->cond);
+        pthread_mutex_unlock(&barrier->mutex);
+        return 1;
+    }
+    else
+    {
+        pthread_cond_wait(&barrier->cond, &(barrier->mutex));
+        pthread_mutex_unlock(&barrier->mutex);
+        return 0;
+    }
+}
+
+#endif // PTHREAD_BARRIER_H_
+#endif // __APPLE__
+
 #ifdef __cplusplus
 extern "C" {
 #endif

[mcuee]

Quick hack for Makefile.macOS

###########################################
# Simple Makefile for HIDAPI test program
#
# Alan Ott
# Signal 11 Software
# 2010-06-01
###########################################

all: hidtest-libusb libs

libs: libhidapi-libusb.dylib

CC       ?= gcc
CFLAGS   ?= -Wall -g -fpic

LDFLAGS  ?= -Wall -g

COBJS_LIBUSB = hid.o
COBJS = $(COBJS_LIBUSB) ../hidtest/test.o
OBJS      = $(COBJS)
LIBS_USB  = `pkg-config libusb-1.0 --libs` -liconv -lpthread
LIBS      = $(LIBS_USB)
INCLUDES ?= -I../hidapi `pkg-config libusb-1.0 --cflags`


# Console Test Program
hidtest-libusb: $(COBJS)
	$(CC) $(LDFLAGS) $^ $(LIBS_USB) -o $@

# Shared Libs
libhidapi-libusb.dylib: $(COBJS_LIBUSB)
	$(CC) $(LDFLAGS) $(LIBS_USB) -shared -fpic -Wl,-install_name,$@.0 $^ -o $@

# Objects
$(COBJS): %.o: %.c
	$(CC) $(CFLAGS) -c $(INCLUDES) $< -o $@


clean:
	rm -f $(OBJS) hidtest-libusb libhidapi-libusb.so ../hidtest/hidtest.o

.PHONY: clean libs

[mcuee]

Runlog for macOS with hidtest using the quick mod libusb backend.

hidapi_test/libusb on  master [!?] ❯ sudo ./hidtest-libusb 
hidapi test/example tool. Compiled with hidapi version 0.10.1, runtime version 0.10.1.
Compile-time version matches runtime version of hidapi.

Device Found
  type: 04d8 003f
  path: 0002:001e:00
  serial_number: (null)
  Manufacturer: Microchip Technology Inc.
  Product:      Simple HID Device Demo
  Release:      2
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c52b
  path: 0002:000a:00
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2410
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c52b
  path: 0002:000a:01
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2410
  Interface:    1
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c52b
  path: 0002:000a:02
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2410
  Interface:    2
  Usage (page): 0x0 (0x0)

Device Found
  type: 047f c025
  path: 0002:0008:03
  serial_number: CB13A3E40E8E47D6A40769C27E90A38E
  Manufacturer: Plantronics
  Product:      Plantronics C320-M
  Release:      135
  Interface:    3
  Usage (page): 0x0 (0x0)

Manufacturer String: Microchip Technology Inc.
Product String: Simple HID Device Demo
Serial Number String: (208) ?
Indexed String 1: Microchip Technology Inc.
Unable to send a feature report.
Unable to get a feature report.
hid_error is not implemented yetwaiting...
waiting...
waiting...
Unable to read()
Data read:

[mcuee]

Under Windows, here is the output, with a quick mod of the following file.
https://gitlab.com/CalcProgrammer1/OpenRGB/-/blob/master/dependencies/hidapi/hidapi.c

I choose to use usbdk backend so that I do not need to replace the HID driver with WinUSB. And it may be able to find more devices like USB keyboard and mouse if they are present.

modified files and Windows 64bit binaries are included in the following zip file.
hidapi_libusb.zip

C:\work\hid\hidapi\libusb [master ≡ +7 ~1 -0 !]> .\hidtest-libusb.exe
hidapi test/example tool. Compiled with hidapi version 0.11.0, runtime version 0.10.1.
Compile-time version is different than runtime version of hidapi.
]nDevice Found
  type: 046d c534
  path: 0001:0008:00
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2901
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c534
  path: 0001:0008:01
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2901
  Interface:    1
  Usage (page): 0x0 (0x0)

Device Found
  type: 047f c056
  path: 0001:001d:03
  serial_number: D1CEC32927974D5F9BD6B2AEBF2EA8E3
  Manufacturer: Plantronics
  Product:      Plantronics Blackwire 3220 Series
  Release:      210
  Interface:    3
  Usage (page): 0x0 (0x0)

Device Found
  type: 04d8 003f
  path: 0001:001c:00
  serial_number: (null)
  Manufacturer: Microchip Technology Inc.
  Product:      Simple HID Device Demo
  Release:      2
  Interface:    0
  Usage (page): 0x0 (0x0)

Manufacturer String: Microchip Technology Inc.
Product String: Simple HID Device Demo
Serial Number String: (1033)
Indexed String 1: Microchip Technology Inc.
Unable to send a feature report.
Unable to get a feature report.
(null)Unable to write()
Error: (null)
Unable to write() (2)
waiting...
waiting...
waiting...
waiting...

C:\work\hid\hidapi\libusb [master ≡ +7 ~1 -0 !]> .\hidtest-libusb.exe
hidapi test/example tool. Compiled with hidapi version 0.11.0, runtime version 0.10.1.
Compile-time version is different than runtime version of hidapi.
]nDevice Found
  type: 413c b06e
  path: 0003:0007:00
  serial_number: (null)
  Manufacturer: (null)
  Product:
  Release:      101
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c534
  path: 0001:0002:00
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2901
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c534
  path: 0001:0002:01
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Receiver
  Release:      2901
  Interface:    1
  Usage (page): 0x0 (0x0)

Device Found
  type: 047f c056
  path: 0003:001c:03
  serial_number: D1CEC32927974D5F9BD6B2AEBF2EA8E3
  Manufacturer: Plantronics
  Product:      Plantronics Blackwire 3220 Series
  Release:      210
  Interface:    3
  Usage (page): 0x0 (0x0)

Device Found
  type: 413c 2107
  path: 0003:000d:00
  serial_number: (null)
  Manufacturer: DELL
  Product:      Dell USB Entry Keyboard
  Release:      178
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 413c b06f
  path: 0003:000a:00
  serial_number: (null)
  Manufacturer: (null)
  Product:
  Release:      101
  Interface:    0
  Usage (page): 0x0 (0x0)

Device Found
  type: 046d c077
  path: 0003:000e:00
  serial_number: (null)
  Manufacturer: Logitech
  Product:      USB Optical Mouse
  Release:      7200
  Interface:    0
  Usage (page): 0x0 (0x0)

unable to open device

[mcuee]

Of course, it is also possible not to use usbdk, just comment out the following line in hid.c.

		//libusb_set_option(NULL, LIBUSB_OPTION_USE_USBDK);

hidapi_libusb2.zip

[Youw]

Instead of using "hacks" to make it functional I'd rather suggest to refactor libusb backend of HIDAPI and switch to C11 instead of using POSIX/pthread API.
That way we will have a clean implementation compliant with the C Standard.

Thoughts?

[mcuee]

Yes, that will be great.

Haha, my hack is just a quick demo to show that it is possible.

[mcuee]

In order for your USB HID device to be found by the libusb backend (hidapi_libusb2.zip), there are two possibilities.

Take note this is only applicable to device listed under Windows Device Manager as "Human Interface Devices" --> "USB Input Device". Keyboard/mouse and Bluetooth/I2C/SPI HID devices are not supported.

1. keep using the default kernel HID driver, then it has same (actually more) limitation as HIDAPI with the normal Windows HID API backend. But it is not recommended to use libusb HID backend because of all the limitations. So we recommend you to use HIDAPI instead.

https://github.com/libusb/libusb/wiki/Windows#known-restrictions
HID keyboards and mice cannot be accessed using the native HID driver as Windows reserves exclusive access to them.
Multiple HID top level collections are currently not supported (only the first top level collection will be used).

2. change the USB HID driver to WinUSB. In that case, certain limitations will be lifted. For example, now you can use control transfer to get the HID report descriptors.

[mcuee]

hidapi_libusb.zip uses the USBDK backend of the libusb, so it may be able to find more devices than hidapi_libusb2.zip. For example, it may be able to find some USB mice/keyboards. But it requires the installation of usbdk. libusb usbdk backend may be less stable than using WinUSB backend.
https://github.com/daynix/UsbDk/releases

[mcuee]

64bit Windows binary with hidapitester here (not using usbdk). libusb-1.0.dll is a mod version of 1.0.24.11650 with pull request libusb/libusb#986 to fix the Windows HID backend issue.
https://github.com/mcuee/hidapitester/releases/tag/v8610dc6

[mcuee]

nstead of using "hacks" to make it functional I'd rather suggest to refactor libusb backend of HIDAPI and switch to C11 instead of using POSIX/pthread API. That way we will have a clean implementation compliant with the C Standard.

@Youw

Since I do not know much about pthread and C++11 threading, and since ChatGPT is now popular, I asked ChatGPT to do the job.

Unfortuantely ChatGPT is not able to finish the full conversion and always got stuck.

(Edit: delete the code as it is not good).

[Youw]

Loool :)
Writing it in C++11 would be even easier, for sure)

[mcuee]

Loool :) Writing it in C++11 would be even easier, for sure)

Glad to hear that.

Interestingly I ask ChatGPT to convert the simple libusb stress_mt.c pthread based test codes to C++11 threading and it has no problems. But the result codes do not work under WIndows but work under Linux. Maybe libusb Windows has some issues under stress.

https://github.com/libusb/libusb/blob/master/tests/stress_mt.c

• Enhance libusb stress test  libusb#1128 (comment)

However, if I ask it to convert into C11, the result codes do not even compile since MinGW does not support C11 <threads>.

• Enhance libusb stress test  libusb#1128 (comment)

[Youw]

MinGW does not support C11

That's a bummer :(
I never actually used C11 threads. I only assumed, that compilers that have C++11 threads would have C implementation as well...

Writing an entire backend in C++ has many proc and cons. One of the most important ones - dependency on C++ Standard library.
And I'm pretty sure there will be some part of the (C) community that will not support such huge change as switching from C to C++.

But maybe, having an experimental (additional, not replacement) C++ port of libusb backend of HIDAPI - should be a "fun project" at least.
And I'm not talking about "just replacing pthreads with C++11 threads". I'm talking of taking a full advantage of Modern C++ (yes - that is a "thing").

[mcuee]

FYI: Apple clang does not support C11 threading either.

Ref:
https://stackoverflow.com/questions/16244153/clang-c11-threads-h-not-found

[mcuee]

I think in the end, this is not an important work (not much real use in reality) and therefore I will close it for now.

[mcuee]

Re-open as a low priority item.

[mcuee]

From #706, comment by mandar1jn.

I'm working on a program which would benefit from using the libusb backend of hidapi due to needing the ability to send control transfers. I was wondering if the libusb backend is available on windows. Libusb works on windows, but the build instructions never mention how to or if it's even possible and I had no luck getting it to compile myself.

[mandar1jn]

I'll add some context to this: I'm working with skylanders portals which are HID devices, but they recieve commands over control transfers, something you wouldn't normally do for an HID device, but what can you do about it. My current code uses hidapi with a workaround to get the device handle from hidapi and manually runs the required win32 functions, but I'd prefer to use libusb (I can still extract the libusb handle from what I saw) and then use that functionality to send control transfers, and maybe utilize the hotplug functionality