A second abstract "buffered" HTTP client protocol

[czechboy0]

Let's discuss having a second protocol for fully buffered requests and responses, likely without trailer support, that many existing clients could conform to today, even without supporting streaming and trailers.

Of course, a concrete client could then implement both the full streaming client protocol and this buffered one, and perform optimizations for the buffered code path.

This is to ensure that clients incapable of implementing the base behavior of the streaming client still have a way to participate, instead of implementing a non-compliant "streaming" client that doesn't deliver trailers.

[czechboy0]

I expect many libraries that perform HTTP calls and just send JSON bodies back and forth would take an instance of this buffered client, rather than the streaming one. Unless you have massive JSON payloads, there'd be little benefit using the streaming one, and the buffered one might even be more performant as it always knows the size of the body etc.

[guoye-zhang]

I think the question comes down to "do we want to design a separate protocol for fetch" since it's way more constrained than other HTTP clients. I'm leaning no since an extra protocol creates more cognitive and maintenance overhead than the benefit it provides.

The risk is that a library that relies on request body streaming is ported to WASM, and the only available HTTP client fetch doesn't support request body streaming, so the memory usage balloons or the request stalls. I understand that this situation is not ideal but I doubt how likely it is:

1. WASM is a different enough environment where various other constraints already made it hard to support.
2. It's unlikely that the library author hasn't tested it with fetch before releasing the library on WASM.

I suggest that we document the limitations for fetch clearly, and provide ways to detect these limitations. Libraries may need to remove certain functionalities on WASM, but it shouldn't need to adopt a whole different protocol.

[czechboy0]

Well the problem is less the memory ballooning, and more that if they're doing something like JSON:RPC, the library won't work at all, and there's nothing the library can do to fix it.

And that's all known at compile time, so the question becomes why even let it compile instead of surfacing it at compile time instead of giving the application owner false hope when they pass the Wasm client to the library that expects a client capable of streaming.

Overall, API packages have one requirement: every implementation works in every context when you use the abstraction - that's how Swift Log and friends work. If the HTTP client doesn't work this way, and libraries still need to enumerate in their documentation which concrete HTTP implementations they work with, I struggle to understand the point of the protocol. The libraries should just offer APIs to pass those concrete clients and avoid the confusion and bugs altogether.

[FranzBusch]

Let me preface this with that I am very sympathetic to solve most things at compile where possible. I do think that there is a trade-off between compile-time verification and developer experience and maintenance. A second protocol does come with a potentially decreased developer experience since now developers have to pick the right one and it comes with additional overhead for us as maintainers. Just laying out the options that we have:

1. Introduce two or more protocols to express all the different capabilities similar to how we are planning to do it for request options
2. Only offer the full streaming with trailers protocol and:
   a. Expect that clients that can't fully offer those features don't conform
   b. Expect that clients that can't fully offer those features conform but throw a runtime error

I think all of them are in some form valid approaches and just offer different trade-offs.

Two protocol approach

I wanna quickly explore how a two protocol approach could look like to see how much maintenance overhead it would bring. I think it is important that we don't call it "Buffered" though since that has a negative connotation in my opinion.

public protocol SimpleHTTPClient<RequestOptions>: ~Copyable {
    associatedtype RequestOptions: HTTPClientCapability.RequestOptions

    func perform(
        request: HTTPRequest,
        body: consuming RawSpan,
        options: RequestOptions,
    ) async throws -> some Sequence<UInt8> // Need to think about exactly what return type we wanna use here
}

public protocol StreamingHTTPClient<RequestOptions>: ~Copyable, SimpleHTTPClient { // Inherits from SimpleHTTPClient
    associatedtype RequestOptions: HTTPClientCapability.RequestOptions
    associatedtype RequestWriter: AsyncWriter, ~Copyable, SendableMetatype where RequestWriter.WriteElement == UInt8
    associatedtype ResponseConcludingReader: ConcludingAsyncReader, ~Copyable, SendableMetatype where ResponseConcludingReader.Underlying.ReadElement == UInt8, ResponseConcludingReader.FinalElement == HTTPFields?

    func perform<Return: ~Copyable>(
        request: HTTPRequest,
        body: consuming HTTPClientRequestBody<RequestWriter>?,
        options: RequestOptions,
        responseHandler: (HTTPResponse, consuming ResponseConcludingReader) async throws -> Return
    ) async throws -> Return
}

extension StreamingHTTPClient {
    func perform(
        request: HTTPRequest,
        body: consuming RawSpan,
        options: RequestOptions,
    ) async throws -> some Sequence<UInt8> {
        // Default implementation of the SimpleHTTPClient method so that streaming clients don't have to implement it
    }
}

Personal opinion

I am currently leaning to go with 2a and subset out fetch based clients but I do expect somebody to implement this nevertheless with runtime errors. We can never prohibit anyone from doing that anyways.

[czechboy0]

I like the SimpleHTTPClient name, I agree leading with the positives (simplicity) is better than with a potentially negative adjective (buffering). You'll notice that the simple one also wouldn't support trailers - which I think makes sense, trailers are mostly useful when you're doing streaming (as otherwise you'd just send that metadata in the headers already).

Nit: in your SimpleHTTPClient example, you're not returning an HTTPResponse, so I presume the returned value would be a tuple of the response and the body buffer (similar to OpenAPI).

I am currently leaning to go with 2a and subset out fetch based clients but I do expect somebody to implement this nevertheless with runtime errors. We can never prohibit anyone from doing that anyways.

If options 2a and 2b were the only ones, I'd be fine with it. What worries me the most is:

2c: A client can't implement a capability and silently drops that data. For example, when a client doesn't support response trailers, it can't know it received them, so can't throw an error. So to a library adopter of this protocol, they literally can't tell whether the server didn't send trailers, or if they're using a non-compliant client implementation. And that will erode trust and make important libraries opt out of the protocol and pick a concrete client (reducing the supported platforms as a result), because correctness and reliability is often more important than platform coverage.

I'm personally fine with 2a temporarily, and we'd make sure to strongly discourage any non-compliant implementation, we wouldn't support it, etc. However, I think you're right - it'd be just a matter of time before someone still implements this protocol incorrectly for fetch, and that library becomes part of the broadly used ecosystem of packages, and libraries will still live with this erosion of correctness and reliability.

So personally, I believe 1 is going to happen anyway, but I think we can probably hold the line for a year or two of 2a, and we can work with the Wasm community on designing the Simple protocol in collaboration, communicating clearly to them that there'll be a solution for them.

[FranzBusch]

2c: A client can't implement a capability and silently drops that data. For example, when a client doesn't support response trailers, it can't know it received them, so can't throw an error. So to a library adopter of this protocol, they literally can't tell whether the server didn't send trailers, or if they're using a non-compliant client implementation. And that will erode trust and make important libraries opt out of the protocol and pick a concrete client (reducing the supported platforms as a result), because correctness and reliability is often more important than platform coverage.

I don't think this argument holds entirely since even with a proposed SimpleHTTPClient protocol the implementation won't know if there were trailers send. It will also just drop them on the floor.

[guoye-zhang]

I think the buffered variant has to still support response streaming (which is supported by fetch today and way more commonly used than request streaming), otherwise people who need that will still implement the streaming protocol.

[czechboy0]

I don't think this argument holds entirely since even with a proposed SimpleHTTPClient protocol the implementation won't know if there were trailers send. It will also just drop them on the floor.

I'm not sure I follow.

The Simple protocol doesn't support trailers, so a library using the abstract client can't send or receive them. The required functionality is just sending and receiving headers and buffered bodies.

The Streaming protocol does currently support trailers, so a library can send and receive them, and the implementation must support them, otherwise the library won't function correctly. The required functionality is sending and receiving headers and trailers, and streaming bodies (no unexpected buffering).

Each library chooses the right protocol and set of protocols that represent additional capabilities. And the application owner must provide a concrete implementation that fulfills those requirements. If they provide a concrete implementation that doesn't fulfill them, the code won't compile and the application owner chooses a different implementation, and the library doesn't need to handle being passed a non-compliant implementation.

The only problem happens when a concrete implementation doesn't correctly implement the required functionality of the protocol. Then the library will misbehave and won't be able to tell if data was dropped as a bug or if the server didn't send the data.

[guoye-zhang]

What about middleware? Do they need 2 variants even if they don't interact with bodies and only headers?

[czechboy0]

I think Middleware is still an open question even for streaming, so it's hard to say right now.

For example, it's unclear to me whether we'll have a MiddlewareClient that takes an arrays of middlewares and an upstream client, or if each middleware will itself be a client. My lean is the former as it allows better observability, but it's probably worth discussing in a dedicated issue.

[FranzBusch]

My current design has a MiddlewareClient that uses a result builder for building up a middleware and takes a client which is called at the end of the chain. To @guoye-zhang if we were to have two protocols then yes we will most likely need two of each middleware as well.

I think we should start with the bi-directional streaming and trailers protocol only for now. We can always introduce the other simple protocol in the future if we wanted to. Let's get some real usage of this package and feedback on the one protocol before we add another one. With the understanding that any recommended implementation must support everything.

[czechboy0]

Right, that means we need to remove the URLSession implementation of the protocol from this repo though.

[guoye-zhang]

We could change the shape of perform of the concrete client to remove trailer support and remove the conformance

[guoye-zhang]

I went through internal NSURLSession usage and these are the rough data. (I only searched in Objective-C .m or .mm files since it's hard to string-search Swift usage along with the many variants of the API added over the years.)

• All dataTask usage (/ession dataTask/): 1400
• Buffered convenience dataTask usage (/ession dataTask[^\]]*$/ or /ession dataTask.*completionHandler.*\]/): 949+72
• All uploadTask usage (/ession uploadTask/): 131
• Buffered convenience uploadTask usage (/ession uploadTask[^\]]*$/ or /ession uploadTask.*completionHandler.*\]/): 59+15
• All downloadTask usage (/ession downloadTask/): 278
• _trailers SPI usage: 2

Then convenience usage may be overestimated since I'm searching for any line without a closing ] but I went through a few pages and they all looked to be completion handler usage.

Buffered convenience of data task and upload tasks (1095 in total) can be translated to the usage of the buffered protocol. That's roughly 60% of the call sites. Download task cannot since it streams the response to a file, not buffered in memory.

The data suggests people using the streaming interface greatly outnumber people using trailers, and we cannot ship a replacement API without streaming.

[guoye-zhang]

A few more thoughts:

1. It does not make sense to me that a backend that does not support trailers also can't support streaming.
2. We cannot force people to give up automatic cookies / cache / authentication support in URLSession, all of them more important than trailers IMO.
3. I doubt the usefulness of the abstract protocol if it only models AHC.