configurable HTTP/2 PING timeouts in HttpClient

[chrisdot]

Justification

Long-running but idle connections (which happen in gRPC and long poll scenarios) can result in firewalls/etc. dropping connections out from under us, without any reset notification. This causes long timeouts on a client that is only reading data.

These APIs will have HttpClient periodically send ping frames over HTTP/2, which would prevent firewalls from seeing the connection as idle and detect otherwise broken connections sooner than we can right now.

API Proposal

public class SocketsHttpHandler
{
    // if we haven't received data on a connection in X time, send a PING frame.
    public TimeSpan KeepAlivePingDelay { get; set; }

    // if server does not respond to the PING in X time, kill the connection on our side.
    public TimeSpan KeepAlivePingTimeout { get; set; }

    // ****new since last review****
    public HttpKeepAlivePingPolicy KeepAlivePingPolicy { get; set; }
}

// Consider using a bool instead. Proposing enum for better extensibility in future.
public enum HttpKeepAlivePingPolicy
{
    // Only send pings if there are active requests.
    WithActiveRequests,

    // Send pings even for idle connections.
    // Common gRPC servers have behavior that will kill connections that use this. ¯\_(ツ)_/¯
    Always
}

Original issue below

I've been evaluating the new gRPC-dotnet and particularly its streaming feature in a long lived connected scenario.
I needed the HTTP/2 PING feature to detect the zombie connections (basically to know: "Am I still having an up and running connection to the client?") on both server and client side.
It seems that this feature won't be implement-able on the client side as long as there's no support of an API in the HttpClient, as the client part of gRPC-dotnet relies on HttpClient.

It seems that HttpClient already responds to ping but can not send any on demand. Am I right?

So I would like to request a new method in HttpClient class to support sending Http/2 pings or even better a kind of keep alive ping timeout to do it automatically in case there's no traffic...

BTW: I have also created an issue for the same feature for kestrel (I don't know what kestrel is using underneath).

[Tratcher]

Adding an HttpClient API to send pings on demand would be really complicated, pings are per connection and HttpClient doesn't expose any connection level APIs.

Using a timer to send pings would be much easier. WebSockets has a similar feature already.

[scalablecory]

Can you give some more detail on your scenario? Once you determine if a connection is still open, what actions will you be taking?

[chrisdot]

Can you give some more detail on your scenario? Once you determine if a connection is still open, what actions will you be taking?

@scalablecory
My scenario is a client/server relation where both sides need to know if the connection is still up and running at any time (in case when connection fails, it should start to act in a degraded mode).

As I imagine it, a response to a PING should not imply any action at all. This is just normal operating. I consider the things just in the opposit way: if the PING does not get a response, then the connection should be closed (an by getting an exception?!)

[Tratcher]

Even the act of sending a ping would likely trigger an immediate exception for local disconnect scenarios. Not so much if there's a proxy involved.

The API for a full round trip ping and ack would be tricky, especially since you could have multiple outstanding pings.

WinHttp WebSockets has a feature where it will periodically send unsolicited ping acks (pongs) just to check if the connection is open. It starts with acks because it doesn't want a response, it's just trying to detect socket errors.

[chrisdot]

@Tratcher

The API for a full round trip ping and ack would be tricky, especially since you could have multiple outstanding pings.

I don't see how this would be a real world a scenario having multiple outstanding pings, at a given moment. A ping would be sent on a idle connection only. And there's no reason to send a second one before the first one was ACKed.
Furthermore, this may be solved by using the "8 octets of opaque data in the payload" of the PING frame that the ACK has to send back?

Maybe there should be 2 parameters:

• an inactivity timeout after which a ping would be sent to test the line
• a timeout after which the ack should have been done

[Tratcher]

A ping would be sent on a idle connection only. And there's no reason to send a second one before the first one was ACKed.

If it were an internal feature based on an idle timer then yes, but if Ping is a callable API then none of that is guaranteed.

[scalablecory]

in case when connection fails, it should start to act in a degraded mode

@chrisdot Thanks. Would you be using this as a way to minimize latency noticed by the user by opening alternative routes or offline alternatives prior to the next request?

Your app would still need to handle the case of a request failing, as even if PING is successful 1 second earlier, the connection could have been killed in the mean time. But if used as the above, I can see some usefulness here.

Furthermore, this may be solved by using the "8 octets of opaque data in the payload" of the PING frame that the ACK has to send back?

Indeed, I don't see this as being a particularly difficult thing to implement.

The one hiccup is that the API would only work for HTTP/2 (and in future, HTTP/3) connections. I don't see that as being a blocker, but I believe it'd be a first for HttpClient so we'd want to make sure we get the design right for such an API.

[chwarr]

As an concrete example of HTTP2 PING frames, consider gRPC--which was the impetus for @chrisdot opening this issue.

The gRPC HTTP2 protocol explicitly uses PING frames:

Both clients and servers can send a PING frame that the peer must respond to by precisely echoing what they received. This is used to assert that the connection is still live as well as providing a means to estimate end-to-end latency. If a server initiated PING does not receive a response within the deadline expected by the runtime all outstanding calls on the server will be closed with a CANCELLED status. An expired client initiated PING will cause all calls to be closed with an UNAVAILABLE status. Note that the frequency of PINGs is highly dependent on the network environment, implementations are free to adjust PING frequency based on network and application requirements.

The https://github.com/grpc/grpc-dotnet/ project is building a gRPC implementation completely in .NET. One of its goals is to be able to interoperate with the other gRPC implementations.

The C core library implementation of gRPC implements the PING part of the protocol and calls it keepalive. The Go implementation is similar.

[chrisdot]

If it were an internal feature based on an idle timer then yes, but if Ping is a callable API then none of that is guaranteed.

@Tratcher : I think we don't need a direct method/API, a configurable timer manner (property) should be enough I guess. In my case I don't see the point of being able to send at the application level the PINGs. I think this is a transport level stuff and if it is transparent but configurable should be enough??

Would you be using this as a way to minimize latency noticed by the user by opening alternative routes or offline alternatives prior to the next request?

@scalablecory : yes, that's it! I would be able to know that the active connection is down even if on the application level I have no payload for a given time. And that's how it is used in gRPC in the streaming call model... (where everything started from)

The C core library implementation of gRPC implements the PING part of the protocol and calls it keepalive. The Go implementation is similar.

@chwarr: this is exactly the root of my request: I would need to be able to set the pendant of the following ping parameters (taken from the grpc-core) to the grpc-dotnet:

• GRPC_ARG_KEEPALIVE_TIME_MS: This channel argument controls the period (in milliseconds) after which a keepalive ping is sent on the transport.
• GRPC_ARG_KEEPALIVE_TIMEOUT_MS: This channel argument controls the amount of time (in milliseconds) the sender of the keepalive ping waits for an acknowledgement. If it does not receive an acknowledgment within this time, it will close the connection.

[jtattermusch]

CC @jtattermusch

[karelz]

@JamesNK do you think this will be needed for gRPC? Did you hear other requests?

Triage: We could implement just timeout (based on fraction of existing timeouts like PooledConnectionIdleTimeout), or we could expose new timeout.
Overall we are not convinced this is super-valuable. If we don't have strong need from users, we would suggest to do nothing, or just fraction-timer based solution.

[chrisdot]

Triage: We could implement just timeout (based on fraction of existing timeouts like PooledConnectionIdleTimeout), or we could expose new timeout.
Overall we are not convinced this is super-valuable. If we don't have strong need from users, we would suggest to do nothing, or just fraction-timer based solution.

I don't know how difficult that is, and I don't fully understand your proposition based on fraction of existing timers.
Timeout after which the PING ack should be done, is not a big deal: using the same timeout mechanism for any regular frame with payload could definitely be a valid option.
But for the inactivity timeout after which a ping would be sent to test the line, I'm sceptical. It may depend of the context constraints (depending to the answer of the question: "how quick should the applevel know that the transport layer is down?) This depend to the context, so maybe better configurable... And this may be better configurable as both client and server might use the same granularity...?

Sorry to insist :-) , but I think there is a tiny hole in the HTTP/2 implementation, which would make the streaming capability of gRPC quite unusable, and not interoperable with other stacks. But I guess there might be other applications over HTTP/2 that would make usage of the keep alive? (can't figure out what for the moment...)

[jtattermusch]

As @chrisdot have mentioned, this feature is critical for gRPC long lived RPCs and streaming RPCs.
It is also important for general interoperability with other gRPC implemenations (that do support keepalive)
Here is the gRPC's keepalive spec: https://github.com/grpc/grpc/blob/master/doc/keepalive.md