Repository navigation
Excessive per-connection memory cause by channel #4057
Description
Activity
Very good point. We really only need one or two slots on the http1 dispatch channel. (The http2 could need more, since its
SendRequestis clone). However, even if we setmpsc::channel(2), it would still allocate with the block size hard coded.Another option is to use
futures::mpsc, which only allocates a slot on demand as a linked list. The downside, and I think it was noticeable in benchmarks (such as end_to_end), is that means an allocation on every singlesend.Yeah I found the same dilemma. It seems there could be a theoretical channel that achieves exactly what we need but I didn't find one at least in mainstream crates (crossbeam included; it uses the same 32 chunks). Probably not worth trying to find a less mature crate that meets this (or implementing)
Somewhat related, just linking them up: #3309
I filed a PR in hyper that could help with this: tokio-rs/tokio#8210
Reacted by John Howard, 叶子, Virgiel, Jake and 0x676e67Reacted by Jake and 0x676e67Checking back in, the linked PR looks like it could work, but it really could use other eyes on it that are motivated enough to help with optimizing async channels, specifically adding an array backend if the bound is small enough.
This is not a bug or anything, or even really behaving wrong, just an optimization potential.
Each HTTP/1.1 channel allocates an obvious 8k from
INIT_BUFFER_SIZE: usize = 8192; this is fine and expected (although perhaps could use some configuration for "many connections with small payloads/less stringent performance requirements").However, there is another ~8kb allocated as well per connection that I don't think is desirable. Each connection gets an unbounded
dispatchchannel. This in turn creates a channel that has 32 blocks (https://github.com/tokio-rs/tokio/blob/8f81e0814bd121683ebdfa21b494d1b77fb36403/tokio/src/sync/mpsc/mod.rs#L140 https://github.com/tokio-rs/tokio/blob/8f81e0814bd121683ebdfa21b494d1b77fb36403/tokio/src/sync/mpsc/block.rs#L47).Each http request is ~250b, giving us another ~8kb for the channel here.
While in theory the SendRequest API does allow pushing a bunch of requests at once, practically speaking most are using this in front of a pooling client (hyper_util or otherwise) which is going to only send requests that are ready. So I would think we don't need for most cases unbounded, nor a 32 block buffer.
A theoretical nice option here, that allows the strange case of sending multiple requests but optimizes the common case, would be to just have the channel not use 32 blocks (which I imagine is for performance reasons not relevant to us?). However, tokio doesn't expose any such option.