-
-
Notifications
You must be signed in to change notification settings - Fork 6
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RFC 0002: MT Execution Contexts #2
base: main
Are you sure you want to change the base?
Conversation
The distinction between execution context and scheduler could need a bit refinement. There is definitely some overlap in functionality, just by comparing the API. I guess execution contexts might take over some features of the current scheduler? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Absolutely lovely, well-written proposal. I completely agree with the design intent here, and only have a few—mostly overlapping—comments about event loops and the default context. The vast majority of this design is exactly what I would like to see in crystal.
0002-execution-contexts.md
Outdated
- a scheduler to run the fibers (or many schedulers for a MT context); | ||
- an event loop (IO & timers): | ||
|
||
=> this might be complex: I don’t think we can share a libevent across event bases? we already need to have a “thread local” libevent object for IO objects as well as for PCRE2 (though this is an optimization). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are multiple event bases required? It doesn't seem obvious to me that they are.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I'm wondering about that. We currently have one libevent base per thread, and that overcomplexifies IO::Evented with lots of objects to allocate in the HEAP for each IO and thread.
I'm probably being naive (though Go seems to do that) but maybe a global event loop wouldn't behave so badly? Even with the potential contention on adding an event to the libevent base, especially on machines with too many cores to count (e.g. ARM Neoverse).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
On the other hand not having to synchronize everything and their mother make other things a lot less complex. It is quite liberating to not give a shit about what other threads are doing.
FWIW, having one dedicated ring per thread is also how the makers of io_uring recommend multi thread usage.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hum, implementing our own wrapper on top of epoll/kqueue becomes more and more compelling.
Since we reschedule the fiber, we could put events on the stack —this is possible with libevent but not recommended: the struct size may change, and not have to keep them somewhere to try and avoid reallocating events all the time.
Then we could have one or many event loops and not care about thread locals (especially in IO); we could keep Fiber#resume_event
and merely take care that it can only be in one event loop queue at a time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, storing it on the stack is a very useful technique for io_uring as well (to keep it alive during submission). I've made a lot of use of that and it helped a lot. There the completion event need to be handled somehow though - I stored it in the fiber itself until it was awakened and could process it, but there may be better techniques.
One good thing about the current implementation though is that it handles the thundering herd problem decently, as far as I've been able to see. That is, if multiple fibers are waiting for something only one of them will wake when something happens. With many listeners that may become something to keep track of.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We shall move the "refactor the event loop" into a proper issue on https://github.com/crystal-lang/crystal/issues
It doesn't need a RFC as it's mostly internal implementation detail.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It might be interesting to consider one event loop (EL) per execution context (EC) 🤔
With a single EL per EC, a starving scheduler will run the EL and enqueue every resumable fibers for the EC.
With one EL per scheduler, a starving scheduler will run its own EL and push the fraction of fibers it happened to have, possibly flooding the EC queues and delaying other schedulers from running their own EL, and delaying resumable fibers from being resumed.
That may be a reason why Go has a single EL: it could be unfair otherwise.
0002-execution-contexts.md
Outdated
|
||
## Default context configuration | ||
|
||
This proposal doesn’t solve the inherent problem of: how can applications configure the default context at runtime (e.g. number of MT schedulers) since we create the context before the application’s main can start. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This proposal supports creating multiple execution contexts, let the application configure it's own EC and start fibers in it if required. That allows all the complexity the app needs when configuring the context the application actually runs in, because it's initialized by the application. The root context does not have to be well-used by the application.
@RX14 Tell me if I'm wrong, the plan would be: Crystal 1
Crystal 2
Isolated context I think I see that context for UI loops only, and want to prevent blocking behaviors, but there's nothing wrong with doing blocking calls in other use cases, and using the event-loop normally is fine. Still, spawning a fiber without an explicit context should either raise or the default context should be configured (as you suggest): abstract class ExecutionContext
class Isolated < ExecutionContext
def initialize(name : String, @spawn_context : ExecutionContext? = nil, &)
@thread = Thread.new(name) { yield }
end
def spawn(**args, &) : Fiber
if ctx = @spawn_context
ctx.spawn(**args) { yield }
else
raise RuntimeError.new("Can't spawn in isolated context (need a spawn context)")
end
end
end
end
mt = ExecutionContext::MultiThreaded.new
ui = ExecutionContext::Isolated.new("GTK", spawn_context: mt) { Gtk.main } Instead of raising, the spawn context could be the default EC. |
@RX14 I applied your suggestions to the RFC. |
There's no such method
@ysbaddaden I think I envision the root execution context being MT or ST a moot point, because every well-architected app has a single If we all agree, maybe we can start on the other 90% of the RFC: bikeshedding naming. I like |
@RX14 The I wouldn't bikeshed the namings just yet. As I'm experimenting with the types, I feel that the difference is getting thinner and thinner. In fact, Kotlin only has a single scheduler implementation, and a couple constructors to start execution contexts with 1 (ST) or many threads (MT). I'm also struggling with the inheritance: |
This pull request has been mentioned on Crystal Forum. There might be relevant details there: https://forum.crystal-lang.org/t/crystal-multithreading-support/6622/8 |
I forgot again but MT:1 would break |
I think we can accept breakage with |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like this, I like this a lot.
0002-execution-contexts.md
Outdated
|
||
Such a group of fibers will never run in parallel. This can vastly simplify the synchronization logic since you don’t have to deal with parallelism anymore, only concurrency, which is much easier & faster to deal with. For example no need for costly atomic operations, you can simply access a value directly. Parallelism issues and their impact on the application performance is limited to the global communication. | ||
|
||
## Issues |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think data and especially execution locality could show up on the negative side as well, as the round robin take away a lot of programmatic control of data locality as well. It is .. possible.. to manually schedule fibers to dedicated threads but it really is not the way it currently is meant to be used.
0002-execution-contexts.md
Outdated
- a scheduler to run the fibers (or many schedulers for a MT context); | ||
- an event loop (IO & timers): | ||
|
||
=> this might be complex: I don’t think we can share a libevent across event bases? we already need to have a “thread local” libevent object for IO objects as well as for PCRE2 (though this is an optimization). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm in the 'let the event loop decide if it want to be instantiated on a thread, execution level or global level' camp. How that would look API-wise I'm less sure - especially not if dynamic amount of threads in a context is to be supported.
This might be complex: I don’t think we can share a libevent across event bases
From what I have gathered from libevent docs, it is possible, but would necessitate a lot more synchronization when io happens (*), so it is probably slower.
But yes, it is complex. Windows, and its weird file handles says hi. Each open file handle is specific for each instance of whatever it uses, so it needs to be only one global one event instance there.
- we already enable some structures for thread safety but then create separate bases for each thread anyhow, IIRC. It was quite a while since I looked at it. I think we can remove that enabling without danger - they should really only be used when actually reusing a libevent base between threads). We don't use the specialized mt safe functions libevent that make use of it.
0002-execution-contexts.md
Outdated
- configuration (e.g. number of threads, …); | ||
- methods to spawn, enqueue, yield and reschedule fibers within its premises; | ||
- a scheduler to run the fibers (or many schedulers for a MT context); | ||
- an event loop (IO & timers): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It probably needs to be mentioned that it needs to continue to work with channels and mutexes. This is somewhat straightforward today with how fibers are bound to a thread once executed, so the simple version is to just schedule it using the normal interfaces. But interfaces for scheduling are not necessarily the same between execution contexts as they are within it!
For example, a basic work stealing scheduler can default to just enqueue a fiber in the executing scheduler/thread context, and let distribution between threads happen in other ways. But that doesn't work if the thing to wake up doesn't live in the same context - then it needs to be communicated somehow. And then the question is what to communicate it to.
Also somewhere it should probably be explicitly defined what happens if channel interaction happens in an isolated context (as defined above).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is explicit in the Guide side:
Applications can create any number of execution contexts in parallel. These contexts are isolated but they shall still be capable to communicate together with the usual synchronization primitives (e.g. Channel, Mutex) that must be thread-safe.
Once spawned a fiber shouldn’t move to another execution context. For example on re-enqueue the fiber must be resumed into it’s execution context: context B enqueues waiting sender from context A. That being said, we could allow to send a fiber to another context.
It's not detailed in the Technical side, though.
0002-execution-contexts.md
Outdated
def initialize(@name : String, @minimum : Int32, @maximum : Int32) | ||
# todo: start @minimum threads | ||
end |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Allowing a dynamic amount of threads requires more synchronization and complexity than having a static amount. While it sounds nice to be able to adjust, it probably warrants its own separate class. Making certain all threads are in a waiting state before starting to actually queue stuff allows a bunch of simplifications with less mutexes and risks a lot fewer possible race conditions too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Enqueue doesn't have to need much sync. Go pushes to a bounded local queue (per scheduler) with overflow to a global queue; threads can be started at any time: when they reach the run loop they will grab a batch of fibers from the global queue or steal from another scheduler. Stopping ain't more complex, schedulers aren't tied to a specific thread, the thread detaches the scheduler and returns itself to the thread pool.
The complexity is more in when to start / stop a thread.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And that complexity pushes for it to become a "future evolution".
As someone who isn't familiar with this stuff at all, my random question is:
|
Co-authored-by: Linus Sellberg <sellberg@gmail.com>
@Blacksmoke16 From what I read specifying a thread priority can hint the OS to schedule the thread on a big (efficient) or little (power) core. We can also set a thread affinity to a given core, but we must detect the core type beforehand. |
Adds notes about wrapping an existing EC, and thread affinities (to pin a thread to a core) in addition to set priorities (still no API). Simplifies the EC API to remove `yield` and `sleep` that may not be needed (the `Fiber.yield` and `sleep` methods can create the resume events), but adds `spawn(same_thread)` to handle the transition.
Co-authored-by: Johannes Müller <straightshoota@gmail.com>
@RX14 I'm growing fond of |
This pull request has been mentioned on Crystal Forum. There might be relevant details there: https://forum.crystal-lang.org/t/fiber-usage-in-high-io-application/6689/2 |
After spending 2 days chasing after a bug, I realized I mixed up two different things into a single ExecutionContext class: the execution context (cross context) and the fiber scheduler (thread local) which led to a nasty bug (cross scheduler local enqueues, when only cross context enqueues should be allowed). I'm trying to separate these into 2 different types (e.g. |
This pull request has been mentioned on Crystal Forum. There might be relevant details there: https://forum.crystal-lang.org/t/glib-main-loop-and-fibers-integration/6156/11 |
1. the context itself (that may have many threads); 2. the fiber scheduling (on _one_ thread). The context itself is meant to create and maintain (or monitor) one or many threads (that each need a scheduler) and the object itself can be used for cross context communication, for example context A telling context B to spawn a fiber. Their usage is both external (public API) and internal (private API). The schedulers are meant to do the actual scheduling of fibers inside a single thread. Their usage is mostly internal (private API). The choice of modules is to be more versatile in the way we want to implement each execution context. We might want to implement both modules in a single type (e.g. single threaded context) or as distinct types.
Co-authored-by: Sijawusz Pur Rahnama <sija@sija.pl>
Thinking again about names:
I'm also thinking about simple constructors: io_workers = ExecutionContext.concurrent
cpu_workers = ExecutionContext.parallel(size: 8)
ui = ExecutionContext.isolate { UI.main_loop } Yet, I'm still struggling for a nice name to the single threaded context 😞 |
I'm not sure we need such convenience constructors. This isn't essential anyway and we can figure it out later. |
@straight-shoota yes, the convenience fonctions aren't needed, but while |
This pull request has been mentioned on Crystal Forum. There might be relevant details there: https://forum.crystal-lang.org/t/how-to-build-for-release/6808/5 |
Add `GC.stop_world` and `GC.start_world` methods to be able to stop and restart the world at will from within Crystal. - gc/boehm: delegates to `GC_stop_world_external` and `GC_start_world_external`; - gc/none: implements its own mechanism (tested on UNIX & Windows). My use case is a [perf-tools](https://github.com/crystal-lang/perf-tools) feature for [RFC 2](crystal-lang/rfcs#2) that must stop the world to print out runtime information of each ExecutionContext with their schedulers and fibers. See crystal-lang/perf-tools#18
This pull request has been mentioned on Crystal Forum. There might be relevant details there: https://forum.crystal-lang.org/t/timeline-for-multithreading-support/3604/21 |
This pull request has been mentioned on Crystal Forum. There might be relevant details there: https://forum.crystal-lang.org/t/upcoming-release-1-14-0/7199/1 |
This pull request has been mentioned on Crystal Forum. There might be relevant details there: https://forum.crystal-lang.org/t/new-event-loop-unix-call-for-reviews-tests/7207/6 |
No description provided.