Skip to content

Allow alternate schemes in Kestrel requests #34013

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 6, 2021
+146 −25
Merged

Allow alternate schemes in Kestrel requests #34013

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
18ec2b7
Allow alternate schemes in Kestrel requests #30532
Tratcher Jul 1, 2021
d652627
Comment improvements
Tratcher Jul 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 9 additions & 7 deletions src/Servers/Kestrel/Core/src/Internal/Http2/Http2Stream.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -240,17 +240,19 @@ private bool TryValidatePseudoHeaders()
// ":scheme" is not restricted to "http" and "https" schemed URIs. A
// proxy or gateway can translate requests for non - HTTP schemes,
// enabling the use of HTTP to interact with non - HTTP services.

// - That said, we shouldn't allow arbitrary values or use them to populate Request.Scheme, right?
// - For now we'll restrict it to http/s and require it match the transport.
// - We'll need to find some concrete scenarios to warrant unblocking this.
// A common example is TLS termination.
var headerScheme = HttpRequestHeaders.HeaderScheme.ToString();
if (!ReferenceEquals(headerScheme, Scheme) &&
!string.Equals(headerScheme, Scheme, StringComparison.OrdinalIgnoreCase))
{
ResetAndAbort(new ConnectionAbortedException(
CoreStrings.FormatHttp2StreamErrorSchemeMismatch(HttpRequestHeaders.HeaderScheme, Scheme)), Http2ErrorCode.PROTOCOL_ERROR);
return false;
if (!ServerOptions.AllowAlternateSchemes || !Uri.CheckSchemeName(headerScheme))
{
ResetAndAbort(new ConnectionAbortedException(
CoreStrings.FormatHttp2StreamErrorSchemeMismatch(HttpRequestHeaders.HeaderScheme, Scheme)), Http2ErrorCode.PROTOCOL_ERROR);
return false;
}

Scheme = headerScheme;
}

// :path (and query) - Required
Expand Down
19 changes: 11 additions & 8 deletions src/Servers/Kestrel/Core/src/Internal/Http3/Http3Stream.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -685,15 +685,18 @@ private bool TryValidatePseudoHeaders()
// ":scheme" is not restricted to "http" and "https" schemed URIs. A
// proxy or gateway can translate requests for non - HTTP schemes,
// enabling the use of HTTP to interact with non - HTTP services.

// - That said, we shouldn't allow arbitrary values or use them to populate Request.Scheme, right?
// - For now we'll restrict it to http/s and require it match the transport.
// - We'll need to find some concrete scenarios to warrant unblocking this.
if (!string.Equals(RequestHeaders[HeaderNames.Scheme], Scheme, StringComparison.OrdinalIgnoreCase))
var headerScheme = HttpRequestHeaders.HeaderScheme.ToString();
if (!ReferenceEquals(headerScheme, Scheme) &&
!string.Equals(headerScheme, Scheme, StringComparison.OrdinalIgnoreCase))
{
var str = CoreStrings.FormatHttp3StreamErrorSchemeMismatch(RequestHeaders[HeaderNames.Scheme], Scheme);
Abort(new ConnectionAbortedException(str), Http3ErrorCode.ProtocolError);
return false;
if (!ServerOptions.AllowAlternateSchemes || !Uri.CheckSchemeName(headerScheme))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CheckSchemeName is a new check, perf?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note this new check only happens after the standard match checks have failed and you've opted into AllowAlternateSchemes, there should be no impact on the normal scenario.

I tried using Http2ConnectionHeadersBenchmark to measure this but the results between runs are wildly inconsistent.

Http2ConnectionHeadersBenchmark.MakeRequest:
AllowAlternateSchemes = false (default)

Method HeadersCount HeadersChange Mean Error StdDev Median Op/s Gen 0 Gen 1 Gen 2 Allocated
MakeRequest 1 False 47.663 us 0.9468 us 2.3226 us 47.941 us 20,980.6 - - - 416 B
MakeRequest 1 True 10.594 us 0.6478 us 1.7950 us 10.339 us 94,390.8 - - - 416 B
MakeRequest 4 False 8.633 us 0.1712 us 0.1601 us 8.620 us 115,840.4 - - - 416 B
MakeRequest 4 True 8.509 us 0.1538 us 0.2206 us 8.510 us 117,517.1 - - - 416 B
MakeRequest 32 False 12.754 us 0.3164 us 0.9179 us 12.351 us 78,404.7 - - - 416 B
MakeRequest 32 True 14.856 us 0.2850 us 0.2526 us 14.779 us 67,312.7 - - - 416 B

2nd run

Method HeadersCount HeadersChange Mean Error StdDev Median Op/s Gen 0 Gen 1 Gen 2 Allocated
MakeRequest 1 False 36.428 us 2.2939 us 6.7635 us 35.743 us 27,451.4 - - - 426 B
MakeRequest 1 True 37.094 us 2.4211 us 7.1387 us 35.420 us 26,958.4 - - - 424 B
MakeRequest 4 False 32.664 us 3.2156 us 9.3291 us 35.267 us 30,614.4 - - - 416 B
MakeRequest 4 True 8.444 us 0.1659 us 0.2679 us 8.353 us 118,424.5 - - - 416 B
MakeRequest 32 False 11.609 us 0.2135 us 0.1893 us 11.601 us 86,139.0 - - - 416 B
MakeRequest 32 True 14.142 us 0.2702 us 0.2528 us 14.023 us 70,712.6 - - - 416 B

AllowAlternateSchemes = false and scheme == "https" (same as the gRPC TLS termination scenario)

Method HeadersCount HeadersChange Mean Error StdDev Op/s Gen 0 Gen 1 Gen 2 Allocated
MakeRequest 1 False 36.66 us 2.118 us 6.246 us 27,276.3 - - - 423 B
MakeRequest 1 True 34.40 us 2.340 us 6.900 us 29,070.2 - - - 425 B
MakeRequest 4 False 34.19 us 2.033 us 5.995 us 29,249.5 - - - 423 B
MakeRequest 4 True 38.31 us 1.672 us 4.929 us 26,106.1 - - - 422 B
MakeRequest 32 False 12.15 us 0.243 us 0.532 us 82,297.0 - - - 416 B
MakeRequest 32 True 14.02 us 0.267 us 0.307 us 71,317.8 - - - 416 B

2nd run

Method HeadersCount HeadersChange Mean Error StdDev Op/s Gen 0 Gen 1 Gen 2 Allocated
MakeRequest 1 False 7.744 us 0.1548 us 0.1372 us 129,124.5 - - - 416 B
MakeRequest 1 True 8.157 us 0.1626 us 0.3133 us 122,594.2 - - - 416 B
MakeRequest 4 False 37.638 us 1.3081 us 3.8364 us 26,568.7 - - - 427 B
MakeRequest 4 True 8.455 us 0.1690 us 0.3451 us 118,279.9 - - - 416 B
MakeRequest 32 False 11.770 us 0.2220 us 0.1968 us 84,961.1 - - - 416 B
MakeRequest 32 True 14.149 us 0.2154 us 0.1909 us 70,677.3 - - - 416 B

{
var str = CoreStrings.FormatHttp3StreamErrorSchemeMismatch(RequestHeaders[HeaderNames.Scheme], Scheme);
Abort(new ConnectionAbortedException(str), Http3ErrorCode.ProtocolError);
return false;
}

Scheme = headerScheme;
}

// :path (and query) - Required
Expand Down
14 changes: 14 additions & 0 deletions src/Servers/Kestrel/Core/src/KestrelServerOptions.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,20 @@ public class KestrelServerOptions
/// </remarks>
public bool AllowSynchronousIO { get; set; }

/// Gets or sets a value that controls how the `:scheme` field for HTTP/2 and HTTP/3 requests is validated.
/// <para>
/// If <c>false</c> then the `:scheme` field for HTTP/2 and HTTP/3 requests must exactly match the transport (e.g. https for TLS
/// connections, http for non-TLS). If <c>true</c> then the `:scheme` field for HTTP/2 and HTTP/3 requests can be set to alternate values
/// and this will be reflected by `HttpRequest.Scheme`. The Scheme must still be valid according to
/// https://datatracker.ietf.org/doc/html/rfc3986/#section-3.1. Only enable this when working with a trusted proxy. This can be used in
/// scenarios such as proxies converting from alternate protocols. See https://datatracker.ietf.org/doc/html/rfc7540#section-8.1.2.3.
/// Applications that enable this should validate an expected scheme is provided before using it.
/// </para>
/// <remarks>
/// Defaults to <c>false</c>.
/// </remarks>
public bool AllowAlternateSchemes { get; set; }

/// <summary>
/// Gets or sets a value that controls whether the string values materialized
/// will be reused across requests; if they match, or if the strings will always be reallocated.
Expand Down
2 changes: 2 additions & 0 deletions src/Servers/Kestrel/Core/src/PublicAPI.Unshipped.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,8 @@
*REMOVED*~static Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(this Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions listenOptions, string fileName, string password) -> Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions
*REMOVED*~static Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(this Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions listenOptions, string fileName, string password, System.Action<Microsoft.AspNetCore.Server.Kestrel.Https.HttpsConnectionAdapterOptions> configureOptions) -> Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions
*REMOVED*~static Microsoft.AspNetCore.Server.Kestrel.Https.CertificateLoader.LoadFromStoreCert(string subject, string storeName, System.Security.Cryptography.X509Certificates.StoreLocation storeLocation, bool allowInvalid) -> System.Security.Cryptography.X509Certificates.X509Certificate2
Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerOptions.AllowAlternateSchemes.get -> bool
Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServerOptions.AllowAlternateSchemes.set -> void
Microsoft.AspNetCore.Server.Kestrel.Https.ClientCertificateMode.DelayCertificate = 3 -> Microsoft.AspNetCore.Server.Kestrel.Https.ClientCertificateMode
static Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(this Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions! listenOptions, Microsoft.AspNetCore.Server.Kestrel.Https.HttpsConnectionAdapterOptions! httpsOptions) -> Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions!
static Microsoft.AspNetCore.Hosting.ListenOptionsHttpsExtensions.UseHttps(this Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions! listenOptions, System.Action<Microsoft.AspNetCore.Server.Kestrel.Https.HttpsConnectionAdapterOptions!>! configureOptions) -> Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions!
Expand Down
64 changes: 59 additions & 5 deletions src/Servers/Kestrel/test/InMemory.FunctionalTests/Http2/Http2StreamTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -404,19 +404,73 @@ public async Task HEADERS_Received_CONNECTMethod_WithSchemeOrPath_Reset(string h
await StopConnectionAsync(expectedLastStreamId: 1, ignoreNonGoAwayFrames: false);
}

[Fact]
public async Task HEADERS_Received_SchemeMismatch_Reset()
[Theory]
[InlineData("https")]
[InlineData("ftp")]
public async Task HEADERS_Received_SchemeMismatch_Reset(string scheme)
{
await InitializeConnectionAsync(_noopApplication);

// :path and :scheme are not allowed, :authority is optional
var headers = new[] { new KeyValuePair<string, string>(HeaderNames.Method, "GET"),
new KeyValuePair<string, string>(HeaderNames.Path, "/"),
new KeyValuePair<string, string>(HeaderNames.Scheme, "https") }; // Not the expected "http"
new KeyValuePair<string, string>(HeaderNames.Scheme, scheme) }; // Not the expected "http"
await SendHeadersAsync(1, Http2HeadersFrameFlags.END_HEADERS | Http2HeadersFrameFlags.END_STREAM, headers);

await WaitForStreamErrorAsync(expectedStreamId: 1, Http2ErrorCode.PROTOCOL_ERROR,
CoreStrings.FormatHttp2StreamErrorSchemeMismatch(scheme, "http"));

await StopConnectionAsync(expectedLastStreamId: 1, ignoreNonGoAwayFrames: false);
}

[Theory]
[InlineData("https")]
[InlineData("ftp")]
public async Task HEADERS_Received_SchemeMismatchAllowed_Processed(string scheme)
{
_serviceContext.ServerOptions.AllowAlternateSchemes = true;

await InitializeConnectionAsync(context =>
{
Assert.Equal(scheme, context.Request.Scheme);
return Task.CompletedTask;
});

var headers = new[] { new KeyValuePair<string, string>(HeaderNames.Method, "GET"),
new KeyValuePair<string, string>(HeaderNames.Path, "/"),
new KeyValuePair<string, string>(HeaderNames.Scheme, scheme) }; // Not the expected "http"
await SendHeadersAsync(1, Http2HeadersFrameFlags.END_HEADERS | Http2HeadersFrameFlags.END_STREAM, headers);

var headersFrame = await ExpectAsync(Http2FrameType.HEADERS,
withLength: 36,
withFlags: (byte)(Http2HeadersFrameFlags.END_HEADERS | Http2HeadersFrameFlags.END_STREAM),
withStreamId: 1);

await StopConnectionAsync(expectedLastStreamId: 1, ignoreNonGoAwayFrames: false);

_hpackDecoder.Decode(headersFrame.PayloadSequence, endHeaders: false, handler: this);

Assert.Equal(3, _decodedHeaders.Count);
Assert.Contains("date", _decodedHeaders.Keys, StringComparer.OrdinalIgnoreCase);
Assert.Equal("200", _decodedHeaders[HeaderNames.Status]);
Assert.Equal("0", _decodedHeaders[HeaderNames.ContentLength]);
}

[Theory]
[InlineData("https,http")]
[InlineData("http://fakehost/")]
public async Task HEADERS_Received_SchemeMismatchAllowed_InvalidScheme_Reset(string scheme)
{
_serviceContext.ServerOptions.AllowAlternateSchemes = true;

await InitializeConnectionAsync(_noopApplication);

var headers = new[] { new KeyValuePair<string, string>(HeaderNames.Method, "GET"),
new KeyValuePair<string, string>(HeaderNames.Path, "/"),
new KeyValuePair<string, string>(HeaderNames.Scheme, scheme) }; // Not the expected "http"
await SendHeadersAsync(1, Http2HeadersFrameFlags.END_HEADERS | Http2HeadersFrameFlags.END_STREAM, headers);

await WaitForStreamErrorAsync(expectedStreamId: 1, Http2ErrorCode.PROTOCOL_ERROR,
CoreStrings.FormatHttp2StreamErrorSchemeMismatch("https", "http"));
CoreStrings.FormatHttp2StreamErrorSchemeMismatch(scheme, "http"));

await StopConnectionAsync(expectedLastStreamId: 1, ignoreNonGoAwayFrames: false);
}
Expand Down
56 changes: 51 additions & 5 deletions src/Servers/Kestrel/test/InMemory.FunctionalTests/Http3/Http3StreamTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -261,19 +261,65 @@ public async Task ConnectMethod_WithSchemeOrPath_Reset(string headerName, string
await requestStream.WaitForStreamErrorAsync(Http3ErrorCode.ProtocolError, CoreStrings.Http3ErrorConnectMustNotSendSchemeOrPath);
}

[Fact]
public async Task SchemeMismatch_Reset()
[Theory]
[InlineData("https")]
[InlineData("ftp")]
public async Task SchemeMismatch_Reset(string scheme)
{
var requestStream = await InitializeConnectionAndStreamsAsync(_noopApplication);

// :path and :scheme are not allowed, :authority is optional
var headers = new[] { new KeyValuePair<string, string>(HeaderNames.Method, "GET"),
new KeyValuePair<string, string>(HeaderNames.Path, "/"),
new KeyValuePair<string, string>(HeaderNames.Scheme, "https") }; // Not the expected "http"
new KeyValuePair<string, string>(HeaderNames.Scheme, scheme) }; // Not the expected "http"

await requestStream.SendHeadersAsync(headers, endStream: true);

await requestStream.WaitForStreamErrorAsync(Http3ErrorCode.ProtocolError, CoreStrings.FormatHttp3StreamErrorSchemeMismatch(scheme, "http"));
}

[Theory]
[InlineData("https")]
[InlineData("ftp")]
public async Task SchemeMismatchAllowed_Processed(string scheme)
{
_serviceContext.ServerOptions.AllowAlternateSchemes = true;

var requestStream = await InitializeConnectionAndStreamsAsync(context =>
{
Assert.Equal(scheme, context.Request.Scheme);
return Task.CompletedTask;
});

var headers = new[] { new KeyValuePair<string, string>(HeaderNames.Method, "GET"),
new KeyValuePair<string, string>(HeaderNames.Path, "/"),
new KeyValuePair<string, string>(HeaderNames.Scheme, scheme) }; // Not the expected "http"

await requestStream.SendHeadersAsync(headers, endStream: true);

var responseHeaders = await requestStream.ExpectHeadersAsync();

Assert.Equal(3, responseHeaders.Count);
Assert.Contains("date", responseHeaders.Keys, StringComparer.OrdinalIgnoreCase);
Assert.Equal("200", responseHeaders[HeaderNames.Status]);
Assert.Equal("0", responseHeaders["content-length"]);
}

[Theory]
[InlineData("https,http")]
[InlineData("http://fakehost/")]
public async Task SchemeMismatchAllowed_InvalidScheme_Reset(string scheme)
{
_serviceContext.ServerOptions.AllowAlternateSchemes = true;

var requestStream = await InitializeConnectionAndStreamsAsync(_noopApplication);

var headers = new[] { new KeyValuePair<string, string>(HeaderNames.Method, "GET"),
new KeyValuePair<string, string>(HeaderNames.Path, "/"),
new KeyValuePair<string, string>(HeaderNames.Scheme, scheme) }; // Not the expected "http"

await requestStream.SendHeadersAsync(headers, endStream: true);

await requestStream.WaitForStreamErrorAsync(Http3ErrorCode.ProtocolError, CoreStrings.FormatHttp3StreamErrorSchemeMismatch("https", "http"));
await requestStream.WaitForStreamErrorAsync(Http3ErrorCode.ProtocolError, CoreStrings.FormatHttp3StreamErrorSchemeMismatch(scheme, "http"));
}

[Fact]
Expand Down