Add hid_set_num_input_buffers() API

[auxcorelabs]

Exposes a new public function hid_set_num_input_buffers(dev, num_buffers) to resize the per-device input report queue.

High-throughput HID devices (medical telemetry, high-poll-rate gaming peripherals, data acquisition hardware) can emit bursts of input reports that exceed the current hardcoded queue sizes (30 on macOS and Linux/libusb, 64 on Windows). When a burst exceeds the queue, reports are silently dropped with no error indication to the caller. Observed on hardware emitting ~90 reports in 200 ms bursts: both the macOS and Linux/libusb backends silently drop ~20% of frames.

Per-backend behavior

• macOS: resizes the userspace input-report queue
• Linux libusb: resizes the userspace report queue
• Windows: wraps HidD_SetNumInputBuffers (parity with existing behavior)
• Linux hidraw / NetBSD: no-op (kernel manages buffering — documented as a @note on the API declaration in hidapi.h)

Safety

• Values outside [1, HID_API_MAX_NUM_INPUT_BUFFERS] return -1 and register a descriptive error on the device (via the backend's register_device_error helper where available)
• Thread-safe: setter uses the same lock (dev->mutex / dev->thread_state) as the respective report callback

Design notes

• Backwards compatibility: Purely additive public API. Defaults per backend are unchanged, so existing callers see no behavioral change. hid_device is opaque in hidapi.h, so struct field additions in backend-private files are not an ABI break.
• Why 1024 cap: Bounds per-device memory to roughly 1 MB on USB high-speed links, preventing memory exhaustion from a caller that passes INT_MAX (malicious or buggy). We've measured bursts up to ~90 reports deep in practice; 1024 leaves room for devices ~10× more demanding. Overridable at build time via -DHID_API_MAX_NUM_INPUT_BUFFERS=N for memory-constrained targets.
• No getter: hidraw and NetBSD back the queue in the kernel with sizes not portably exposed to userspace, so a cross-platform getter couldn't return a meaningful value there.
• Why not hid_get_input_report() instead? That API issues a host-initiated GET_REPORT request (via the control endpoint or OS equivalent). It does not drain the interrupt-endpoint queue that streaming devices fill, and in practice many devices either don't implement GET_REPORT for input reports or respond incorrectly. The two APIs use different USB transfer mechanisms (interrupt endpoint vs. control endpoint); this PR fixes the interrupt-streaming path.

Userspace queue: linked list → ring buffer

The latest commit replaces the linked list with a fixed-size ring:

• O(1) enqueue/dequeue. hid_report_callback / read_callback no longer scan a tail on every incoming report.
• Exact cap semantics. Setter calls hidapi_input_ring_drop_oldest on shrink, so dev->num_input_buffers is the precise steady-state queue length (matches Copilot's "limit corresponds precisely to num_input_buffers" request).
• Clean allocation-failure handling. The previous code had an unchecked allocation that would segfault on OOM. The new push returns -1 cleanly; libusb drops the report silently (no active error channel), mac calls register_device_error.
• ABI unchanged. hid_device is opaque; the struct field swap is invisible to callers.
• Implementation. hidapi_input_ring_* static-in-header helper, present as byte-identical copies in libusb/ and mac/.

References

• when I use hidapi, how to use 'HidD_SetNumInputBuffers' to set Buffer size ? #154 — this PR also addresses the prior request to expose HidD_SetNumInputBuffers functionality to callers

[Youw]

refer to comments, otherwise looks good

[auxcorelabs]

Thanks for reviewing. I have made the changes.

[JoergAtGithub]

I think the name of the Win32-API HidD_SetNumInputBuffers describes better what it does than hid_set_input_report_buffer_size, as it does change the number of buffers and not the size of the bufferin bytes.

[auxcorelabs]

Good callout @JoergAtGithub. Have renamed the function to hid_set_num_input_buffers() and made the associated changes.

@Youw - Please let me know if any further changes required.

Thanks both.

[mcuee]

@auxcorelabs
Just wondering if you can enhance the hidtest application to include this new API.

If not, do you have a simple test to share? Thanks.

[Youw]

So far so good.

[Copilot]

Pull request overview

Adds a new public HIDAPI entry point to let callers increase the per-device input report queue depth, addressing silent report drops during bursty/high-throughput input on backends that queue reports in userspace (macOS + Linux/libusb) and exposing the existing Windows kernel buffering control.

Changes:

• Introduces hid_set_num_input_buffers(dev, num_buffers) and documents it in hidapi.h (with a max-cap macro).
• Updates macOS and Linux/libusb backends to enforce queue limits via a per-device num_input_buffers value (default 30).
• Adds no-op stub implementations for Linux hidraw and NetBSD uhid; updates hidtest to exercise the new API when available.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
hidapi/hidapi.h	Declares and documents the new public API and its max-cap macro.
mac/hid.c	Adds per-device buffer cap field, uses it in the report callback, and provides a setter guarded by the device mutex.
libusb/hid.c	Adds per-device buffer cap field, uses it in the read callback, and provides a setter guarded by the thread-state mutex.
windows/hid.c	Implements the API by forwarding to HidD_SetNumInputBuffers.
linux/hid.c	Adds a validated no-op stub for the hidraw backend.
netbsd/hid.c	Adds a validated no-op stub for the uhid backend.
hidtest/test.c	Calls the new API under a version guard to ensure all backends export it.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The queue-length enforcement here becomes both expensive and semantically fuzzy now that callers can raise num_input_buffers up to 1024. Each report traverses the entire linked list to find the tail (O(n)), which can become a hot path at high report rates, and the current num_queued accounting means the steady-state queue length is not exactly dev->num_input_buffers. Consider tracking a tail pointer and an explicit queued-count (or switching to a ring buffer) so enqueue/drop is O(1) and the limit corresponds precisely to num_input_buffers.

[Youw]

Someone needs to double-check this, but I'm pretty sure Copilot is right here.
We might even need to consider one step further and switch from linked list to a ring buffer as an optimisation

[auxcorelabs]

Updated to ring buffer.

[Youw]

is this a known implementation for such a ring buffer?
I'd love to at least basic unit-testing for such kind implementation.

2. We're still having a dynamic allocation each time we try to push/pop, which kind of defeats the purpose of a ring buffer in a first place. Since we know the maximum size of an input report (from a HID Report descriptor), we can pre-allocate a single continues buffer to fit at most num_input_buffers.

3. Having two idential implementations copy-pasted in two subdirs is not great. Since we already have it in the separate file, lets place it at common place: ../core/hidapi_input_ring.h

[auxcorelabs]

(1a) Known implementation. The ring's core shape — fixed slot array of (pointer, length) entries with head/tail wraparound, per-push payload allocation is closer to Linux kernel's drivers/hid/hidraw.c: same struct hidraw_report { __u8 *value; int len } slot, per-push kmalloc (kmemdup = kmalloc + memcpy) in hidraw_report_event, per-pop kfree in hidraw_read. Substantive differences are context-driven: bitmask wraparound requires a power-of-2 cap vs ours is build-time configurable, spin_lock_irqsave is mandatory in kernel interrupt context vs we run in userspace and can use pthread_mutex, drop-new vs drop-old. Chose drop-oldest for this PR's streaming-device use case.

(1b) Unit testing - will work on this.

(2) Per-push: low idle memory that grows with actual use; small CPU cost per report VS Flat pre-alloc: more memory always, though reducible at build time via -DHID_API_MAX_NUM_INPUT_BUFFERS=N for constrained targets, allocation-free on the hot path (lower CPU cost). It's a design choice — happy to go with whichever you prefer.

(3) - Thanks for pointing to the common place. Will move to ../core/hidapi_input_ring.h

[Youw]

2 - I suggest to pre-allocate the actual number of buffers requested (or default), and not the MAX_NUM_INPUT_BUFFERS - lets have that, since we've already started.
It will make the implementation maybe a little less common, and will require more complicated logic on "resize" action (i.e. allocate new buffer, move/copy existing data from old buffer, then free old buffer), but that's pretty much how std::vectorstd::array would work in C++ (except in our implementation, the size of a report is also determined in runtime - once from the report descriptor.

[Youw]

This is an impressive amount of new code contributed in a short amount of time.
What AI tool has been used to generate it? I will not believe it was hand-written.

[auxcorelabs]

This is an impressive amount of new code contributed in a short amount of time. What AI tool has been used to generate it? I will not believe it was hand-written.

This was a relatively large change, developed using a structured workflow with human oversight & guidance. The design and approach/choices were planned and reviewed (via AI-assisted passes and manually) before any code change. Two AI models were then used in complementary roles—one for code drafting and the other for iterative review to refine the result. The output went through human review and feedback, and the final changes were manually validated, including building across available platforms to verify there were no errors before committing.

From my experience, the effectiveness comes primarily from the structured workflow and repeated AI review iterations, followed by human review & guidance.

[Youw]

the effectiveness comes primarily from the structured workflow and repeated AI review iterations, followed by human review & guidance

I generally don't mind as I'm using similar tools and aproaches myself in various other projects.

But since HIDAPI is open-source well-know public repo with many users and this is a first huge contribution to HIDAPI using AI-tools, I'd like to mention this explicitly in a form suggested by https://docs.kernel.org/process/coding-assistants.html, specifically the commit message/PR description attribution (at least I'm planning to include it in the final squash-merge commit into master), e.g.:
Assisted-by: AGENT_NAME:MODEL_VERSION [TOOL1] [TOOL2]

that's why I asked what AI tool/model was used to produce this.

---

I'll make a generic NOTE about this somewhere in the README a bit later.