SocketsHttpHandler: Improve handling of failed connection attempts #60654
Description
In 6.0 we changed the way that requests are assigned to connections. When no idle connections are available, we will enqueue the request, and possibly initiate a new connection attempt (depending on queue length and connection pool policy). When a connection becomes available (either new or existing), we will use this to satisfy the next request in the queue, if any. In general this works well and provides reduced request latency and improved connection utilization.
As part of this change, we also changed how connection failure is handled. When a connection attempt fails, we fail the request at the top of the request queue, which may be a different request than originally caused the connection attempt to be initiated.
This behavior is a bit weird. More importantly, it results in additional overall request failures on connection timeout failures, in cases where the request timeout is less than the connection timeout (as explicitly set by the user, or as effectively enforced by the OS). Suppose we create a new SocketsHttpHandler instance with a 15s connect timeout, and submit a request to it with a 10s request timeout. The request will be enqueued and will cause a new connection attempt to be initiated, which will eventually fail due to timeout after 15s. Before this happens, though, the request itself will timeout after 10s. Suppose we now submit another request, again with a timeout of 10s. After 5s, the original connection attempt will timeout and cause the second request to fail, since it is at the top of the request queue. Effectively, the connection timeout has now caused two requests to fail, whereas in 5.0 it would have only caused one.
In general, when a connection is initiated in order to try to satisfy a queued request, but the original request has already been handled (meaning, either was processed by another connection that became available, or was canceled) by the time the connection actually fails, then there is no reason to fail a different request. We can simply ignore the connection failure in this case.
Doing this would fix the issue described above, and in general would result in fewer connection failures being reported to users.