Can't get input with pylibremidi

[Maboroshy]

I'm trying to get pylibremidi working. My basic input handler:

import time
import pylibremidi as lm


PORT_NAME = 'LPX MIDI'

config = lm.InputConfiguration()
config.ignore_sysex = False
config.on_message = print
config.on_raw_data = print
config.on_error = print
config.on_warning = print

lm_port = lm.MidiIn(config)
inputs = {port.port_name: port for port in lm.Observer().get_input_ports()}
lm_port.open_port(inputs[PORT_NAME])

print(lm_port.is_port_open())
print(lm_port.is_port_connected())

while True:
    time.sleep(1)

It prints True for .is_port_open() and .is_port_connected(), but nothing more. Other MIDI lib prints the input, so the hardware works.

What am I missing here?

[Maboroshy]

I get an input if I replace time.sleep(1) with lm_port.poll(), but only a single message. The script fails with exit code -1073740791 (0xC0000409) after that.

[jcelerier]

seems to be a crash in the "on_raw_data" handling in the python bindings, investigating. With your examples on my side on windows it works if I just set config.on_message = print, not config.on_raw_data

[jcelerier]

same thing works fine from C++ on windows so that really corners it on the python binding

[jcelerier]

ok, the error seems to be that nanobind has trouble passing std::span to python

[jcelerier]

as it's really when I invoke the std::function callback passed that it fails

[jcelerier]

should be fixed in master

[jcelerier]

(and indeed, you should use poll, not sleep

[jcelerier]

should be fixed in master :)

[Maboroshy]

Printing works for me. Moving my handler for another step:

import pylibremidi as lm


PORT_NAME = 'LPX MIDI'

def handle_msg(msg: lm.Message):
    print(msg.bytes)

def handle_raw(raw):
    print(raw)

config = lm.InputConfiguration()
config.ignore_sysex = False
config.on_message = handle_msg
config.on_raw_data = handle_raw
config.on_error = print
config.on_warning = print

lm_port = lm.MidiIn(config)
inputs = {port.port_name: port for port in lm.Observer().get_input_ports()}
lm_port.open_port(inputs[PORT_NAME])

print(lm_port.is_port_open())
print(lm_port.is_port_connected())

while True:
    lm_port.poll()

Both callbacks crash with exit code -1073740791 (0xC0000409).

[Maboroshy]

(and indeed, you should use poll, not sleep

As far as I understand I need to make a Python thread that will .poll() a port. I can make a single thread that will poll all the opened ports or a thread for each port. As far as I know Python threads are GIL bound, they perform worse than C-threads.

I suggest to consider moving mandatory polling to the C part for callback handling. That's why I used time.sleep(1) expecting C part do the job.

[jcelerier]

I'm not sure I follow - unless you use free-threaded python builds which are super uncommon, python does not support threading at all, everything always has to be in the main thread, or in separate processes with multiprocess.

Right now what happens is that the C++ library receives the callback in separate threads and then moves them back to the main thread explicitly for the python bindings so that Python can process the messages, as receiving a message from a separate thread to python would cause pretty immediate crashes.

[Maboroshy]

As far as I know Python threads are real OS threads, but GIL limits them to run one at a time. Python threads don't live in a single OS thread like asyncio does.

I have that lm_port.poll() that is called indefinitely as fast as it can. I presume that to make each call Python does it's lookups and stuff, all bound to GIL. Also each call will be waiting in line with other Python threads' jobs for GIL to serve it. I also presume that each call is translated to C library which is not free. And a thread in the C library would have none of those.

If I'm correct, all the time lm_port.poll() is waiting for GIL to serve it is an extra latency and jitter in the MIDI input chain.

[jcelerier]

ok, I didn't know about python threads!

Here is a PR that adds a "direct" property on configurations.
If it is set to True, then messages will be sent from whatever thread the host API sends them: #192