Ndi-python leaves the GIL locked during all library calls

[JC3]

Problem

It appears that ndi-python leaves the GIL locked during all library calls, which halts all other Python threads while NDIlib calls are being executed.

Solution

The GIL should be released during long-running, blocking library calls.

See https://docs.python.org/3/c-api/init.html#releasing-the-gil-from-extension-code for information about threading compatibility in extension libraries.

It looks like somebody has already implemented this in #22, it may be worth reviewing and potentially merging that PR then releasing an update.

Example

For example, the following program creates a thread that repeatedly calls find_wait_for_sources while continuously printing messages from the main thread (Python 3.10):

import threading
import time
import NDIlib as ndi

def p (message: str) -> None:
    print(f"[{time.time():.3f}] {message}")

def threadproc () -> None:
    finder = ndi.find_create_v2()
    while True:
        p("find_wait_for_sources enter")
        ndi.find_wait_for_sources(finder, 3000)
        p("find_wait_for_sources leave")

ndi.initialize()
thread = threading.Thread(target=threadproc)
thread.start()
while True:
    p("ding")
    time.sleep(0.1)

The expected output of this program would be "ding" every 1/10th of a second, but the actual output (notice the timestamps) is:

[1727534088.330] find_wait_for_sources enter
[1727534091.330] ding
[1727534091.330] find_wait_for_sources leave
[1727534091.330] find_wait_for_sources enter
[1727534094.331] ding
[1727534094.331] find_wait_for_sources leave
[1727534094.331] find_wait_for_sources enter
[1727534097.336] ding
[1727534097.336] find_wait_for_sources leave
[1727534097.336] find_wait_for_sources enter
[1727534100.337] ding
[1727534100.337] find_wait_for_sources leave
[1727534100.337] find_wait_for_sources enter
[1727534103.341] ding
[1727534103.341] find_wait_for_sources leave
[1727534103.341] find_wait_for_sources enter
[1727534106.341] ding
[1727534106.341] find_wait_for_sources leave

This shows that the main thread is not executing while the library is inside the find_wait_for_sources call, which is indicative of the GIL not being released during the call.

Consequence

It is impossible to use ndi-python in a threaded context. For moderately complex applications this is a showstopper, especially when performance is critical.

The only workaround is to use ndi-python in its own process and build all the IPC architecture necessary to shuffle frames around between processes, which is very cumbersome.