Repository navigation
Add support for dropping the GIL while performing COBS operations #8
Description
In my application, I decode sometimes very large (megabytes) COBS packets while also handling soft-realtime IO operations where a missed deadline means an application-level failure. I would be able to eliminate these failures were I was able to offload COBS operations to another thread, but currently the C extension holds the GIL while decoding. If it unlocked the GIL I would be able to get much lower latency in my application.
Activity
cmcqueen commentedon Jul 31, 2025
I would consider doing this.
I'm not familiar with the topic, so I'm unsure as to what the design considerations are.
I had mildly considered this in the past, however, I thought releasing the GIL is mostly advised for a scenario in which a processor is waiting on I/O, and less likely to be useful when the processor is doing something computationally intensive. I didn't think it would be likely that anyone would use COBS on such large packets that it would be beneficial to release the GIL.
But, it sounds as though yours is a case that breaks all my previous assumptions.
I assume for many/most users who operate on smaller packets, releasing and reacquiring the GIL could impose a performance penalty.
cmcqueen commentedon Jul 31, 2025
If the C extension were to release the GIL, what would ensure the source byte-string or other bytes-object would not be modified by any other Python thread during the COBS encode/decode operation?
whitequark commentedon Jul 31, 2025
In both of these cases you are officially encouraged to drop GIL (numpy is doing this for example).
In my application, I plan to use COBS as the primary framing mechanism used for all non-trivial communication with the hardware (for which I developed a high-performance COBS encoder/decoder as well). It is extremely important to me that it is as high-throughput and low-latency as possible. The packet sizes are essentially unbounded (well, bounded by RAM); I can envision receiving a 100 MB packet in some circumstances. Of course right now this would be pretty bad latency-wise since decoding isn't incremental, but we can cross that bridge when we have to.
In most modern implementations, taking an uncontended lock is essentially free unless you're doing it in a busy loop (which isn't the case if we're executing Python bytecode already), so this shouldn't affect single-threaded applications at all. I don't think there's going to be much of a hit to multithreaded applications, but that I would want to measure before being confident about it.
whitequark commentedon Jul 31, 2025
I would do something like...
bytearray, large enough to hold both source and destination; you are now its sole owner, so you can freely modify its data without risk of racescmcqueen commentedon Aug 1, 2025
Wouldn't the extra memcpy make the algorithm considerably slower?
I wrote a Rust crate for cobs. :)
whitequark commentedon Aug 1, 2025
A memcpy can be usually done at close to the memory bandwidth and it is vectorized. Since your algorithm is scalar there is no way it will have a comparable speed, and will take up the bulk of the runtime. This should not be difficult to measure. (Also, unless I'm missing something, you'd be making a copy either way?)
Re Rust: I've considered using https://crates.io/crates/cobs, but it raises questions about deployment (e.g. I have a Pyodide based version) that I'm not ready to deal with yet.