Skip to content

Allow connection pools to stop reusing an HTTP/2 connection before its keep-alive timeout #4186

Description

@jiahaoliang

Is your feature request related to a problem? Please describe.

An HTTP/2 connection can remain locally ready while its network path is unavailable. If no earlier transport or protocol error is observed, a connection pool can keep assigning new requests to that connection until the keep-alive PING timeout closes it.

A concrete scenario is a network failover or NAT/firewall state change after which new connections succeed but an existing connection remains unusable. The replacement path may have lost the old connection's state: a new TCP handshake can establish fresh state, while packets belonging to the old connection are silently dropped. If the client receives no FIN, RST, or other definitive error, the existing H2 connection can remain eligible for reuse locally even though it cannot make progress. Continuing to assign requests to it delays recovery despite a working path for new connections.

There are documented examples of the underlying network behavior:

  • Google Cloud NAT documents that removing an external NAT IP without first draining it deletes the associated connection-tracking entries and drops all return traffic. With another usable NAT IP available, this provides a scenario where fresh connections can work while the old mapping no longer carries responses.

This does not assume that every network failover invalidates existing connections. A temporary interruption may also recover, which is why stopping reuse should not itself terminate existing streams.

There are two different policies involved: how long to keep existing streams alive during a recoverable interruption, and how long to keep assigning new work to that connection. Reducing the keep-alive timeout improves replacement latency, but also terminates existing streams earlier.

For example, an application may want to retain a 60-second PING timeout for existing streams while stopping reuse after an outstanding keep-alive PING has waited 5 seconds for an ACK. These are example settings, not proposed defaults.

Describe the solution you'd like

Would an optional reuse-timeout notification on the HTTP/2 client builder fit Hyper's API?

The proposal adds keep_alive_reuse_timeout(Option<Duration>) and a per-connection KeepAliveObserver. The observer would be exposed through hyper::client::conn::http2. A pool such as hyper-util's legacy Client could use the notification to retire its own connection entry.

  • The option defaults to disabled.
  • Its timer starts with the existing keep-alive PING timeout phase.
  • An ACK ready before notification cancels the soft wait.
  • On expiry, Hyper invokes a nonblocking observer once, outside the PING state lock.
  • The original hard deadline continues to run. Notification does not close IO, send GOAWAY, cancel streams, or change direct SendRequest readiness.
  • A late ACK does not generate a reversal notification; the consuming pool owns its retirement policy.

The implementation uses Hyper's Timer abstraction and does not require changes to h2 or its dependency constraints.

Describe alternatives you've considered

Shortening the hard timeout also reduces interruption tolerance for existing streams. Application-level health requests or rebuilding an entire Client can provide a separate signal, but add another probe and broader connection-management logic. Deriving the signal from the existing PING wait keeps it specific to one connection.

Additional context

This issue tracks both parts of the proposal, with one implementation PR per repository:

  • Hyper: expose the optional PING reuse-timeout notification through the HTTP/2 client API.
  • hyper-util: add http2_keep_alive_reuse_timeout(Option<Duration>) to the legacy Client builder and bind each H2 handshake to an observer holding only that connection's PoisonPill. Existing pool checks then retire the entry without canceling its active streams.

The hyper-util setting would default to disabled. Its retirement policy would be irreversible even after a late ACK. Requests already checked out can race with retirement; neither PR migrates or replays requests or establishes connections proactively.

The hyper-util PR depends on the Hyper API. It can be reviewed as a draft using a companion checkout for validation, but must use an actual released Hyper version containing the API before it is ready to merge. Neither PR needs an h2 version pin.

Activity

  1. jiahaoliang commented on Sep 10, 2026

    @jiahaoliang
    Author

    Use my own word to summary:
    In some scenario, especially ip/route changed, the existing H2 connections receive no response from the other end, but new connection may work. The current implementation relies on http2_keep_alive_timeout to close the connections entirely. The proposal add an http2_keep_alive_reuse_timeout, where if the H2 PING doesn't get PONG within given timeout, the legacy client pool will not reuse the connection for new request but create new connection instead.

    That way allows faster recovery without giving up interruption tolerance by reducing http2_keep_alive_timeout .

  2. added
    C-featureCategory: feature. This is adding a new feature.
    and removed
    C-featureCategory: feature. This is adding a new feature.
    on Oct 9, 2026
  3. seanmonstar commented on Oct 9, 2026

    @seanmonstar
    Member

    It sounds like you'd like to have a second set of behavior be possible when the keep alive ping timeout is triggered. Instead of killing the connection, just mark the SendRequest is dead. I guess then a second timeout would eventually make the Connection task give up entirely?

    At first think, this feels like a fair about of extra complexity to support a connection which sounds like it's in the process of dying. How common is this, really?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    C-featureCategory: feature. This is adding a new feature.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions