Does tokio support multple reactor eventloop?

[taikulawo]

Is your feature request related to a problem? Please describe.
We build a high concurrent service to replace nginx. In high concurrent area, one acceptor thread is not enough when there are many connection want to establish.

Describe the solution you'd like
Tokio runtime should have a function set acceptor eventloop thread number

Describe alternatives you've considered
spawn many tokio current thread runtime to match os thread count. But It lost work strealing feature when there no much task to run

[Darksonn]

The only way to do this is several current thread runtimes.

[taikulawo]

OK, I got it. I'm want to know why tokio don't want to support it? :)

[Darksonn]

I'm not necessarily against supporting this, but I think it would take a lot of work to do.

[taikulawo]

But I think it's a key for high concurrent server :)

[Noah-Kennedy]

It's actually hard to make reactor/driver per core work well using readiness-based IO systems like epoll. This is because [1, 2] of fundamental problems with bpth the implementation of these systems in kernels and readiness-based IO in general.

I've been investigating this in the context of io_uring, where it is much easier because completion-based IO has a lot of inherent synergy with reactor-per-core strategies since you don't really have a spurious wake ups issue because you only wake on completion.

This being said, I'm actually not sure that multi-reactor support is what you actually need. I think that what you really need is the ability to run pinned tasks on workers. This would allow you to have a separate instance of each acceptor task running on each worker. This is not something that we currently support, but I do think that it would be potentially quite useful to have.

@Darksonn I believe that there have been conversations about pinned tasks in the past. Is this something that you think would be worth revisiting?

[Darksonn]

Pinned tasks have come up in the context of spawn_local, but this cases issues if the runtime flavor supports block_in_place. Pinned tasks that are Send could eliminate the block_in_place concerns.

[Noah-Kennedy]

I'm not convinced that the issues with block_in_place are even all that significant. I think that documenting that these tasks won't run if the worker that they are pinned to is in that state would be enough here.

IMO these tasks fill a very different role than normal tasks or even a LocalSet. Task pinning is useful as a way to effectively scale operations like accepts across workers or as per-worker background tasks to manage worker-specific state stored in thread-locals.