Getting Effective URI

[rolftimmermans]

I have a trivial server with these lines:

println!("{:?}", req.uri());
println!("{:?}", req.headers());

When I execute curl localhost:3000/foo, I get this output:

/foo
{"host": "localhost:3000", "user-agent": "curl/7.54.0", "accept": "*/*"}

When I execute curl --http2-prior-knowledge localhost:3000/foo, I get the following:

http://localhost:3000/foo
{"user-agent": "curl/7.54.0", "accept": "*/*"}

Why the difference? It would be nice to have HTTP1 and HTTP2 behave the same... I couldn't pinpoint where this is happening in the code or I'd have sent a PR.

[seanmonstar]

The difference here is that in HTTP/1.1, a literal host header is sent by curl, and in HTTP2, it is not. Here's a pseudo translation of the two versions:

GET /foo HTTP/1.1
host: localhost:3000

GET http://localhost:3000/foo HTTP/2

So, in HTTP2 requests, the info that used to be sent in a host header is now in the URI authority. That's why you see it when printing req.uri().authority().

[rolftimmermans]

Well TIL that HTTP2 does not use the Host header. It seems instead that curl is lying to me ;-)

curl -v --http2-prior-knowledge localhost:3000/foo
...
GET /foo HTTP/2
> Host: localhost:3000
> User-Agent: curl/7.54.0
> Accept: */*

Anyway, would it then make sense to also include the value of the HTTP1 Host header in req.uri().authority() for consistency? That we it is always possible to get the host via req.uri().

[bluetech]

I think you are asking to be able to get what the HTTP specification calls the Effective Request URI. It is calculated from configuration, the request target, and the Host header. This sounds useful to me; it looks easy to get it wrong (or incomplete), and every multihomed HTTP server needs to handle it.

I don't think overriding req.uri() is a good idea though, the actual URI that was used should still be accessible. It sounds like it should be a standalone function which takes the necessary inputs and provides the Effective Request URI as output.

[rolftimmermans]

That would be perfect. Any suggestions what the API would be? I wouldn't mind implementing it with a PR.

To avoid others falling in the same pitfall that I fell in it would make sense to me to make this API as obvious as possible, potentially as simple as just doing req.uri(). Can we have a req.effective_uri() or something?

[rolftimmermans]

Adding onto this, from the description you linked it seems this should ideally also take into account any X-Forwarded-Proto headers, is that right? But trusting X-Forwarded-* headers does require explicit opt-in to avoid security issues...

[lnicola]

This should be implemented in http, not hyper, right?

[seanmonstar]

Yea, as mentioned, the original values should be left in place, since you might actually want to know them.

So far this hasn't been provided by hyper because knowing the scheme and host in HTTP/1.1 isn't always possible, unless the server has some more knowledge up front, like whether TLS is used, or the specific hostname that the server should be handling. Without that info, it seems like an API would have to return perhaps an Option<Uri>, or perhaps the function could ask for the extra information as arguments to use if the request doesn't include it.

Does such an API seem useful to be part of hyper directly? I'm neither for nor against, but if it is included here, those questions need to be handled.

[jplatte]

This was just re-discovered up in the context of extracting the URI as part of a middleware for axum: https://discord.com/channels/500028886025895936/870760546109116496/881853967146041385