-
-
Notifications
You must be signed in to change notification settings - Fork 338
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for user-level flat combining #1612
Comments
Can you post an example of what your manual version looks like, versus what it might look like with library support? |
Sure, one example of usage is https://github.com/catern/rsyscall/blob/master/python/rsyscall/epoller.py#L160 which uses https://github.com/catern/rsyscall/blob/master/python/rsyscall/concurrency.py#L7 which briefly says:
I don't have a concrete suggestion for how to fit that into the trio API. |
Ah, in that case I think this is a duplicate of #266 |
Closing in favor of #266. |
Er, I didn't comment before, but this is not a duplicate. I said one possible way to implement it would be #266, but that's just one implementation strategy. There are other ways this could be done - it's a completely independent feature. |
@catern In that case, can you give more details about how this is different? |
Well, the paper coining (mentioned in my original post) the term describes an implementation based on a lock and a list of requesters:
That's a quite different implementation which could also be made easier by support in trio. |
Yeah, I clicked through to the paper :-). My trouble is that they're talking about a very different scenario: implementing low-level threadsafe data structures with minimal locking overhead. In Trio this case is trivial because of being single-threaded. OTOH, we have to deal with cancellation, which they don't. So it seems like our challenges are totally different, and I'm not quite seeing how to translate their ideas into something that makes sense in the Trio context. |
Flat combining is a name coined by the paper "Flat Combining and the Synchronization-Parallelism
Tradeoff" for a synchronization technique where a single thread dynamically takes on the role of handling requests from other threads to perform mutations on a datastructure. This can provide performance improvements, and simplifies implementation of the datastructure - no need for fine-grained concurrency.
In trio, threads (tasks) are cheap, so it might seem pointless - just spawn a dedicated thread to service requests to mutate the datastructure. But spawning a thread requires a nursery for that thread, and complicates management for objects. It's much simpler for an object which is concurrently accessed to just pick one of the threads accessing it to perform mutations, as in flat combining.
I've implemented a lot of flat-combining manually in libraries using trio. It would be nice if there was a built-in way as part of trio to do this. This might combine nicely with support for kill-safe synchronization abstractions which I've also done a lot in trio, painfully manually.
The text was updated successfully, but these errors were encountered: