Support HTTP/0.9 responses in the client

[nox]

There are servers out there that sometimes reply with HTTP/0.9 responses, even modern-ish stack (we saw such responses from MSFT servers for example).

I'm filing this here and not on the httparse repo because this is a substantial change worth discussing in the open, and because we need to think about how to support it. Firefox for example has some logic to content-sniff the start of an actual HTTP/1.* response in the first 4KB of received data on the first request made on the connection, and this is obviously not code that should live in httparse.

As a start, I feel like Hyper should check whether the response starts with "HTTP" at the beginning of the data it received, and if not, maybe it should pretend it saw HTTP/1.0 200 OK\r\n\r\n.

I say HTTP/1.0 and not HTTP/0.9 because it is literally impossible for httparse::Response to represent 0.9 in its version field.

[seanmonstar]

👀 This is the first time I've seen HTTP/0.9 support asked for. I would have assumed that since HTTP/1.0 came out (double checks) over 25 years ago, people would have moved on. Oh well.

If a client sends a request to a server that happens to not speak HTTP at all, what's the proposed way to notice that versus just assuming it's HTTP/0.9?

[nox]

Firefox does this: https://searchfox.org/mozilla-central/rev/f07a609a76136ef779c65185165ff5ac513cc172/netwerk/protocol/http/nsHttpTransaction.cpp#2014-2031 So more or less if it doesn't find some "HTTP/1|2|3" or "ICY " in the first 11 bytes, it will assume this is a head-less HTTP/0.9 response. Note that Firefox explicitly rejects HTTP/0.9 responses to PUT requests.

The logic for Chrome is here: https://chromium.googlesource.com/chromium/src/net/+/8cc942b38e929612f0d85a685f04e73b3203dbb1/http/http_stream_parser.cc#966 AFAICT it also tries to find "HTTP" near the start of the incoming data, and if it doesn't it assumes HTTP/0.9. Note that Chrome explicitly rejects HTTP/0.9 responses not made against default scheme ports, except if it find "ICY " (see this code).

[seanmonstar]

I suppose what we could do in hyper is add a builder option, default off. If enabled, then do as you propose: try to parse as HTTP/1, if it fails and the response doesn't look like HTTP/1.X, return it as HTTP/0.9. How does that sound?

[nox]

That's fine by me, whatever you think would be best. We have yet to see any response that didn't start immediately at the first byte we are looking at, so that plan would be good I think.