App fails to process request and returns 503/404 when URL has more than 1 slash

[K-Meow]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

App throws exception when processing request https://www.contoso.com///path/path/path

App returns 404 when processing request https://www.contoso.com/path//path/////path

Expected Behavior

https://www.contoso.com///path/path/path and https://www.contoso.com/path//path/////path should be normalized to https://www.contoso.com/path/path/path, or give a chance to app to normalize it

Steps To Reproduce

MoreThanOneSlash.csproj

<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>netcoreapp3.1</TargetFramework>
    <SelfContained>false</SelfContained>
  </PropertyGroup>
</Project>

Program.cs

namespace Contoso.App
{
    using Microsoft.AspNetCore;
    using Microsoft.AspNetCore.Hosting;

    public class Program
    {
        public static void Main(string[] args)
        {
            var webHost = WebHost.CreateDefaultBuilder(args)
                .UseHttpSys(o => { o.AllowSynchronousIO = true; })
                .UseUrls("https://+:1234/path")
                .UseStartup<Startup>()
                .Build();
            webHost.Run();
        }
    }
}

StartUp.cs

namespace Contoso.App
{
    using Microsoft.AspNetCore.Builder;

    public class Startup
    {
        public void Configure(IApplicationBuilder app)
        {
        }
    }
}

Exceptions (if any)

System.ArgumentException: The path in 'value' must start with '/'. (Parameter 'value')
at Microsoft.AspNetCore.Http.PathString..ctor(String value)
at Microsoft.AspNetCore.Http.DefaultHttpRequest.get_Path()
at Microsoft.AspNetCore.Hosting.HostingApplicationDiagnostics.RecordRequestStartEventLog(HttpContext httpContext)
at Microsoft.AspNetCore.Hosting.HostingApplication.CreateContext(IFeatureCollection contextFeatures)
at Microsoft.AspNetCore.Server.HttpSys.MessagePump.ApplicationWrapper`1.CreateContext(IFeatureCollection contextFeatures)
at Microsoft.AspNetCore.Server.HttpSys.MessagePump.ProcessRequestAsync(Object requestContextObj)
fail: Microsoft.AspNetCore.Server.HttpSys.MessagePump[0]
ProcessRequestAsync
System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.AspNetCore.Hosting.HostingApplication.DisposeContext(Context context, Exception exception)
at Microsoft.AspNetCore.Server.HttpSys.MessagePump.ProcessRequestAsync(Object requestContextObj)

.NET Version

No response

Anything else?

.NET SDK (reflecting any global.json):
Version: 6.0.302
Commit: c857713418

Runtime Environment:
OS Name: Windows
OS Version: 10.0.19044
OS Platform: Windows
RID: win10-x64
Base Path: C:\Program Files\dotnet\sdk\6.0.302\

Host:
Version: 6.0.7
Architecture: x64
Commit: 0ec02c8c96

.NET SDKs installed:
3.1.421 [C:\Program Files\dotnet\sdk]
5.0.214 [C:\Program Files\dotnet\sdk]
5.0.215 [C:\Program Files\dotnet\sdk]
6.0.100 [C:\Program Files\dotnet\sdk]
6.0.302 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
Microsoft.AspNetCore.App 3.1.26 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 3.1.27 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 6.0.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 3.1.26 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 3.1.27 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 6.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 3.1.26 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 3.1.27 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 5.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 6.0.7 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

[Tratcher]

A 404 is expected from the given application, that's the default when nothing generates a response.

We intentionally do not normalize multiple slashes, that's not part of the URI spec and can cause filter bypass exploits.

We'll need to investigate the exceptions.

[Tratcher]

I'm able to repro the ArgumentException issue.

Binding: http://localhost:port/path
Request: http://localhost:port///path/path/path
Resulting PathBase: ///pa
Resulting Path: th/path/path
PathString then throws when trying to wrap the Path field because it's missing the leading slash.

The first issue is that Http.Sys seems to normalize consecutive slashes when it is matching the request to the binding, so the request does reach the app. However, AspNetCore doesn't normalize them when processing the request. When it goes to split the path and path base the lenths are wrong and it splits in the wrong place.

aspnetcore/src/Servers/HttpSys/src/RequestProcessing/Request.cs

Lines 84 to 85 in 6e3e3de

	PathBase = originalPath.Substring(0, prefix.PathWithoutTrailingSlash.Length); // Preserve the user input casing
	Path = originalPath.Substring(prefix.PathWithoutTrailingSlash.Length);

Recommendation: On Http.Sys we'll have to normalize or handle consecutive slashes in the path base to match the native behavior.

Test:
src\Servers\HttpSys\test\FunctionalTests\Listener\RequestTests.cs

    [ConditionalTheory]
    // Overlong ASCII
    // [InlineData("/", "/", "/")]
    // [InlineData("/path", "/path", "")]
    [InlineData("/path", "///path/path/path", "/")]
    public async Task Request_WithPathBase(string pathBase, string requestPath, string expectedPath)
    {
        string root;
        using (var server = Utilities.CreateHttpServerReturnRoot(pathBase, out root))
        {
            var responseTask = SendSocketRequestAsync(root, requestPath);
            var context = await server.AcceptAsync(Utilities.DefaultTimeout).Before(responseTask);
            Assert.Equal(expectedPath, context.Request.Path);
            context.Dispose();

            var response = await responseTask;
            Assert.Equal("200", response.Substring(9));
        }
    }

[Tratcher]

@K-Meow can you summarize the impact this has had for you? Yes, consecutive slashes are buggy, but do you regularly encounter them in your application? I'm not aware of any legitimate reason for urls to use consecutive slashes, there's no defined behavior for it.

[Tratcher]

Workaround (.NET 6):

using Microsoft.AspNetCore.Http.Features;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddSingleton<IHttpContextFactory, FixupFactory>();

builder.WebHost.UseHttpSys();
builder.WebHost.UseUrls("http://localhost:8080/base");

var app = builder.Build();

app.MapGet("/{**catch-all}", context => context.Response.WriteAsync($"base: {context.Request.PathBase.Value}, path: {context.Request.Path.Value}"));

app.Run();

public class FixupFactory : IHttpContextFactory
{
    private readonly DefaultHttpContextFactory _defaultFactory;

    public FixupFactory(IServiceProvider serviceProvider)
    {
        _defaultFactory = new DefaultHttpContextFactory(serviceProvider);
    }

    public HttpContext Create(IFeatureCollection featureCollection)
    {
        var requestFeature = featureCollection.Get<IHttpRequestFeature>();
        var pathBase = requestFeature.PathBase;
        var path = requestFeature.Path;
        if (pathBase.Contains("//", StringComparison.Ordinal))
        {
            var fullPath = pathBase + path;
            var pathBaseLength = pathBase.Length;
            // The real pathBase is the same length as the given pathBase here, but this one has duplicate slashes in it.
            // Find the real end of the pathBase accounting for the extra slashes.
            for (var i = 0; i < pathBaseLength; i++)
            {
                if (fullPath[i] == '/' && fullPath[i + 1] == '/')
                {
                    // Duplicate
                    pathBaseLength++;
                }
            }
            requestFeature.Path = fullPath.Substring(pathBaseLength);
            requestFeature.PathBase = fullPath.Substring(0, pathBaseLength);
        }

        return _defaultFactory.Create(featureCollection);
    }

    public void Dispose(HttpContext httpContext)
    {
        _defaultFactory.Dispose(httpContext);
    }
}