hyper-util pool: unnamable types are hard to work with

Hello! I hope this is the right place to post this sort of feedback.

Ive played around a lot with the new pool module in hyper_util and here are some thoughts from a still-noobish developer:

the lack of documentation and examples makes it hard to understand how these APIs are supposed to be used
i had a hard time dealing with all the returning types being unnamable

For my reverse proxy i wanted to create a simple connection pool, so my instinct was to build a type that wraps the new Cache service from hyper_util and implement Service for it

this was very challenging because of the "service that returns another service" behaviour and because Cache isnt directly namable, so my wrapper had to be generic too. This is what i came up with:

impl<Cache, Sender> Service<Request<Incoming>> for ProxyService<Cache>
where
    // the cache is a service which returns another impl service
    Cache: Service<PeerAddr, Response = Sender> + Send + 'static + Clone,
    Cache::Future: Send + 'static,
    Cache::Error: Into<BoxError>,
    // this is the sender the cache spits out, which itself is another server
    Sender: Service<Request<Incoming>, Response = Response<Body>> + Send + 'static,
    Sender::Future: Send + 'static,
    Sender::Error: Into<BoxError>,

i would have much rather written a very simple Service that takes a requests and gives a response.

Ok, maybe we can erase the type with .boxed_clone(), well that only half works because you lose the special methods like .retain() since you can only coerce them into a "regular" service types since Cache isnt namable

the workaround i came up with was an intermediate service that calls the cache on the requests behalf

    // wrapper service to avoide double service calls and because "Cache" isnt nameable
    let http1con = service_fn(move |req: Request<_>| {
        let mut cache = http1cache.clone();
        async move {
            debug!("calling connector");
            let peer_addr = req
                .extensions()
                .get()
                .cloned()
                .ok_or_else(|| anyhow!("PeerAddr extension not found, req: {req:?}"))?;

            let Ok(mut sender) = cache
                .ready() 
                .await?
                .call(peer_addr) // call pool
                .await
                .inspect_err(|e| tracing::error!(%e, "couldnt get sender"))
            else {
                return Ok::<Response<Body>, BoxError>(response(StatusCode::INTERNAL_SERVER_ERROR));
            };
            
            // send off request
            sender.ready().await?.call(req).await.or_else(|e| {
                tracing::error!(%e, "sending failed");
                Ok(response(StatusCode::INTERNAL_SERVER_ERROR))
            })
        }
    });
    http1con.boxed_clone()

It doesnt feel right but maybe this is the intended behaviour?
Regardless, i greatly appreciate the work that went into hyper so i hope this was helpful

Hi! I've been working with similar HTTP connection pooling patterns in my own observability tooling and ran into the same friction. I'd like to contribute docs and a working example for the pool module in hyper-util showing the intended usage pattern for Cache — would that be welcome? Happy to also document the current workaround for the unnameable type issue so users aren't left figuring it out from scratch.

@pd241008

Hi there! Thank you guys for wanting to make the code easier to write and read that is always the goal. @pd241008 the docs would be very welcome! What do you think about putting it here?

Yea, it does make the bounds big... What I'd been doing in my slow reqwest refactor was allowing the construction of several layers in the same function, so type inference was able to handle most of the nastiness, and then I type erase it with boxing after I'd set up a layer that uses the retain types.

I don't have a good answer there. Unnameable types provide us some flexibility to grow the generics, but it also makes things harder in other ways. :(

@Catwoman08

Proposal: Usage documentation for `hyper_util::client::pool::cache`

Related issue: [hyperium/hyper-util#4059](https://github.com/hyperium/hyper-util/issues/4059) — "pool: unnamable types are hard to work with"
Author: pd241008
Status: Draft for review (per @Catwoman08's request in the issue thread)

Problem

Cache<M, Dst, Ev> is explicitly documented as "normally unnameable" — it's exposed in rustdoc only so its public methods (retain, is_empty) are visible, not so it can be written out as a concrete type. This is by design (it lets the internals evolve without breaking callers), but the module has no guidance on how to actually hold one in your own code. New users naturally reach for a struct field or a generic wrapper, hit an unnameable type, and improvise — often reinventing the same workaround independently.
This was exactly the friction @Reza-Darius hit while building a reverse proxy around Cache.

This proposal adds a # Usage section to the module-level docs laying out
the two supported patterns, so the workaround becomes documented behavior
instead of tribal knowledge.

Proposed module documentation

Add the following to hyper_util::client::pool::cache (module doc comment, above the existing "A cache of services" summary). Each block below is shown as plain Rust for readability — in the actual source, every line gets a //! prefix as a doc comment.

`# Usage` (intro)

[Cache] is intentionally unnameable outside of its own type
parameters — you cannot write Cache<Foo, Bar, Baz> as a field type
unless your own type is generic over the same parameters. This is
deliberate: it lets the pool's internals evolve without becoming a
breaking change for every downstream caller. In exchange, there are two
supported ways to hold one.

`## Pattern 1 — stay generic`

If you're writing a wrapper Service around the cache, make your
wrapper generic over the inner make-service and thread the bounds
through. This is the zero-cost option: no boxing, and you keep access
to [Cache::retain] and [Cache::is_empty].

This example is a real doctest (no ignore), so CI checks it stays correct:

use hyper_util::client::pool::cache::Cache;
use tower_service::Service;

struct MyProxy<M, Dst, Ev> {
    cache: Cache<M, Dst, Ev>,
}

impl<M, Dst, Ev> MyProxy<M, Dst, Ev>
where
    M: Service<Dst>,
{
    fn evict_stale(&mut self) {
        self.cache.retain(|_response| true /* keep-predicate */);
    }
}

`## Pattern 2 — erase it behind` `service_fn`

If you don't need retain() / is_empty() at the call site — e.g. you
just want something implementing Service<Request<Incoming>> that you
can store as a trait object or pass around without generics — wrap the
call to the cache in a closure instead of trying to name the cache's
type. This trades away the cache's special methods for a plain,
nameable impl Service.

Marked ignore in the real doc comment, since it depends on external
make-service / request types not available in doc-test scope:

let cache = hyper_util::client::pool::cache::builder().build(my_make_service);

let svc = tower::service_fn(move |req: Request<Incoming>| {
    let mut cache = cache.clone();
    async move {
        let dst = extract_destination(&req)?;
        let mut sender = cache.ready().await?.call(dst).await?;
        sender.ready().await?.call(req).await
    }
});

// `svc` is now a plain, boxable, storable service.
let erased = BoxCloneService::new(svc);

If you need periodic eviction (retain) alongside this pattern, keep a
clone of the original cache handle from before erasure — those
methods are not reachable once the cache is wrapped inside
service_fn.

Exact source to paste into cache.rs (click to expand)

//! # Usage
//!
//! [`Cache`] is intentionally unnameable outside of its own type
//! parameters — you cannot write `Cache<Foo, Bar, Baz>` as a field type
//! unless your own type is generic over the same parameters. This is
//! deliberate: it lets the pool's internals evolve without becoming a
//! breaking change for every downstream caller. In exchange, there are two
//! supported ways to hold one.
//!
//! ## Pattern 1 — stay generic
//!
//! If you're writing a wrapper `Service` around the cache, make your
//! wrapper generic over the inner make-service and thread the bounds
//! through. This is the zero-cost option: no boxing, and you keep access
//! to [`Cache::retain`] and [`Cache::is_empty`].
//!
//! ```
//! use hyper_util::client::pool::cache::Cache;
//! use tower_service::Service;
//!
//! struct MyProxy<M, Dst, Ev> {
//!     cache: Cache<M, Dst, Ev>,
//! }
//!
//! impl<M, Dst, Ev> MyProxy<M, Dst, Ev>
//! where
//!     M: Service<Dst>,
//! {
//!     fn evict_stale(&mut self) {
//!         self.cache.retain(|_response| true /* keep-predicate */);
//!     }
//! }
//! ```
//!
//! ## Pattern 2 — erase it behind `service_fn`
//!
//! If you don't need `retain()` / `is_empty()` at the call site — e.g. you
//! just want something implementing `Service<Request<Incoming>>` that you
//! can store as a trait object or pass around without generics — wrap the
//! *call* to the cache in a closure instead of trying to name the cache's
//! type. This trades away the cache's special methods for a plain,
//! nameable `impl Service`.
//!
//! ```ignore
//! let cache = hyper_util::client::pool::cache::builder().build(my_make_service);
//!
//! let svc = tower::service_fn(move |req: Request<Incoming>| {
//!     let mut cache = cache.clone();
//!     async move {
//!         let dst = extract_destination(&req)?;
//!         let mut sender = cache.ready().await?.call(dst).await?;
//!         sender.ready().await?.call(req).await
//!     }
//! });
//!
//! // `svc` is now a plain, boxable, storable service.
//! let erased = BoxCloneService::new(svc);
//! ```
//!
//! If you need periodic eviction (`retain`) alongside this pattern, keep a
//! clone of the original `cache` handle from *before* erasure — those
//! methods are not reachable once the cache is wrapped inside
//! `service_fn`.

Rationale for structure

Two patterns, not one "correct" answer. As @seanmonstar noted in the
thread, there isn't a single clean fix for the unnameable-type
friction — generics-with-inference and boxing-after-construction are
both legitimate, situational answers. Documenting both lets users pick
based on whether they need retain/is_empty at the call site.
Doctested where possible. Pattern 1 is written as a real (non-ignore)
doctest so it's checked by CI and can't silently drift from the actual
API. Pattern 2 is marked ignore since it depends on an external
make-service and request type not available in the doc context — happy
to adjust if a self-contained example is preferred.
Cross-links Cache::retain / Cache::is_empty, so the two exposed
methods are discoverable directly from the usage guidance instead of
requiring a separate trip to the struct page.

Open questions for maintainers

Preferred location: module doc (as above) vs. a standalone examples/
file referenced from the doc — happy to do either.
Whether to also document the workaround as a named idiom (e.g. "cache
passthrough service") so future issues/PRs can reference it by name.
Should Cached's own unnameability get the same treatment on its
struct page, or is covering it via Cache's docs sufficient?

Prepared for the hyper-util maintainers following the discussion in
issue #4059.

My preference is for the docs we add to ideally be hand written. Compilers may be able to enforce quality on LLM code, but: https://seanmonstar.com/micro/20260729-i-want-your-own-words/

@seanmonstar

Ok sure I will rewrite the docs and give it @seanmonstar also I am Extremly sorry for the inconvenience

Navigation Menu

Repository navigation

hyper-util pool: unnamable types are hard to work with #4059

Proposal: Usage documentation for `hyper_util::client::pool::cache`

Problem

Proposed module documentation

`# Usage` (intro)

`## Pattern 1 — stay generic`

`## Pattern 2 — erase it behind` `service_fn`

Rationale for structure

Open questions for maintainers

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

hyper-util pool: unnamable types are hard to work with #4059

Description

Activity

pd241008 commented on Jun 17, 2026

Catwoman08 commented on Jul 31, 2026

seanmonstar commented on Jul 31, 2026

pd241008 commented on Aug 1, 2026

Proposal: Usage documentation for hyper_util::client::pool::cache

Problem

Proposed module documentation

# Usage (intro)

## Pattern 1 — stay generic

## Pattern 2 — erase it behind service_fn

Rationale for structure

Open questions for maintainers

seanmonstar commented on Sep 8, 2026

pd241008 commented on Sep 9, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Proposal: Usage documentation for `hyper_util::client::pool::cache`

`# Usage` (intro)

`## Pattern 1 — stay generic`

`## Pattern 2 — erase it behind` `service_fn`