HTTP2: New calls blocking on SETTINGS_MAX_CONCURRENT_STREAMS

[JamesNK]

If I start 100 duplex streaming calls using HttpClient on a server where SETTINGS_MAX_CONCURRENT_STREAMS = 100, the 101st will block until one is closed:

var calls = new List<AsyncDuplexStreamingCall<DataMessage, DataMessage>>();

for (int i = 1; i < 1000; i++)
{
    Logger.LogInformation($"ITERATION {i}");

    // The streaming call stays open forever
    var call = client.DuplexStreamingCall();

    // Blocks on 101st iteration
    await call.ResponseHeadersAsync.DefaultTimeout();

    calls.Add(call);
}

Right now a user has two options:

1. Simple solution is to configure Kestrel (or whatever other HTTP/2 server they are talking to) to have an unlimited SETTINGS_MAX_CONCURRENT_STREAMS.
2. If a user doesn't control the server then they either have to use a separate HttpClient instance for each call (1 connection = 1 stream), or try to count the number of calls in progress and manage a pool of HttpClient instances.

Neither seem like ideal solutions. What is the best fix? Should there be an option on HttpClientHandler to start a new connection when max concurrent streams is hit instead of blocking? Related: https://github.com/dotnet/corefx/issues/32303

// @stephentoub @geoffkizer @wfurt

[wfurt]

is this reproducible to servers other than Kestrel?

[JamesNK]

I haven't tried, but I found issues of it happening on other HTTP/2 servers with gRPC: envoyproxy/envoy#3076

And @stephentoub mentioned in an email that HttpClient waiting after hitting the maximum stream count is expected behavior.

[stephentoub]

What is the best fix? Should there be an option on HttpClientHandler to start a new connection when max concurrent streams is hit instead of blocking?

The server is telling the client the maximum number of requests it wants to serve from that client; trying to work around that by creating a new connection both violates that request and the HTTP/2 specification.

[JamesNK]

Ok. Is the correct fix for the server to increase SETTINGS_MAX_CONCURRENT_STREAMS on the server?

[stephentoub]

Is the correct fix for the server to increase SETTINGS_MAX_CONCURRENT_STREAMS on the server?

If the goal is to allow more concurrency between the client and the server, then yes. The client assumes it has no upper limit unless the server tells it otherwise, and then it respects what the server tells it.

[geoffkizer]

Note the RFC specifically states, https://tools.ietf.org/html/rfc7540#section-9.1:

   Clients SHOULD NOT open more than one HTTP/2 connection to a given
   host and port pair

It's up to the server to decide how many concurrent streams to allow.