I like the look of it. The design I was thinking uses slightly different terminology and interface, but essentially the same idea:

scheduler is a container for threads and tasks. domain is a less-familiar term, and thread or task_thread is fine for a thread that runs tasks.
A scheduler is first-class. Users can make new ones and inject tasks into them. spawn(f) effectively does task::get_current_scheduler().add_task(task::create(f))
A thread is not first-class, as their lifecycle tends to be extremely platform-specific and it's not generally safe to start or kill one any old time you like. Users can, however, set policy about how many threads are active in a given scheduler. Three policies exist:
- THREAD_PER_CORE makes 1 thread per core and maps tasks to threads M:N
- THREAD_PER_TASK makes 1 thread per task.
- MANUAL_THREADS makes exactly as many threads as a user asks for.
By default there is one THREAD_PER_CORE scheduler.
Every C call takes the thread "out of service" from running tasks; we will not switch-away from the calling task (even if we re-enter the runtime, or some task code, from a C callback) until the outermost C call completes. This means in a THREAD_PER_CORE scheduler, you should not call code that's going to block in C, if you want to avoid starving the scheduler of threads. Make a secondary scheduler in MANUAL_THREADS mode and put your blocking tasks in there.

Implement scheduler domains #1721

Description