HttpRequestMessage does not respect TryAddWithoutValidation

[shravan2x]

I have an application that sends HTTP requests via HttpClient. I use TryAddWithoutValidation to add an accept-language header. Trying to read the AcceptLanguage field returns the correct value. However, any request sent has a wrong value with spaces added into it. Example:

HttpRequestMessage mes = new HttpRequestMessage();
mes.RequestUri = new Uri("https://www.example.com");
mes.Headers.TryAddWithoutValidation("accept-language", "en-US,en;q=0.9");
Console.WriteLine(mes.Headers.AcceptLanguage); // has the correct value
HttpClient cli = new HttpClient();
await cli.SendAsync(mes); // sends the wrong value

Inspecting the sent request using a debugging proxy shows the value of the accept-language header is en-US, en; q=0.9 (note the added spaces).

I'm unsure if there is similar broken behavior for other HTTP headers as well, but I believe this is an important issue to resolve. There are valid use-cases where such broken HTTP headers can cause issues:

• A server may not support spaces in the field causing requests to be rejected. Even if the HTTP protocol allows spaces in the field, servers may wrongly not support them. An HTTP framework should not change values like this especially when users use TryAddWithoutValidation.
• Chrome doesn't use spaces in the accept-language headers it sends. Since it has a significant market share, this is a good reason why embedded servers may simply not support other formats of the header.
• Unique characteristics like these allow servers to sniff user device information for fingerprinting, reducing privacy.

Apart from these reasons, I believe that HTTP frameworks should be flexible enough to allow users to make changes as they prefer.

Tagging subscribers to this area: @dotnet/ncl
Notify danmosemsft if you want to be subscribed.

[stephentoub]

Thanks. Have you tried with .NET 5 (any of the previews)? I believe this should be addressed.

[shravan2x]

You're right, I'm not able to repro on .NET 5. Closing.