Bug: Path matching fails when the URL contains encoded parts

[chrischu]

When setting up a path matcher with a string containing URL-encoded values (e.g. Request.Create().UsingGet().WithPath("/path/a%20b")) the path matching no longer works. When looking at the request log in the FluentMockServer one can see that the request found does have the unencoded request URL ("/path/a b" instead of "/path/a%20b") and therefore the match isn't successful.

This seems to be due to a problem with OWIN decoding the URL (see aspnet/AspNetKatana#208). This bug seems to no longer exist when using ASP.NET Core instead of OWIN, so switching to ASP.NET Core hosting would be a way to fix it, albeit a pretty expensive one I assume.

[StefH]

Currently OWIN is only used for net452 and net46. For netstandard1.3 and 2.0, Microsoft.AspNetCore is used.

Or do you mean something else?

[chrischu]

Mhh I created a repro sample with .NET 4.7.2 that reproduces the problem: In the program I call the URL "/A%2fB" but WireMock.Net logs the call as "/A/B". When looking at the loaded modules in Debug mode it is also plainly visible that OWIN is used:

Am I doing something wrong?

After some more trials: Interestingly enough even when using "/A%252fB" as URL (%25 = URL-encoded %) the recorded URL is "/A/B" so OWIN seems to double-unencode the string. When using "/A%25252fB" the recorded URL is "/A%2fB" so the unencoding seems to stop at twice.

WireMockRepro.zip

[StefH]

Currently OWIN is used for net452.

And because net 4.6.x and net 4.7.x fallback to this framework, the OWIN code is also used in your application.

I did a quick test with net 461, and it seems I can use the latest Microsoft.AspNetCore (version 2.x.x) there.

So I'll update WireMock.Net to use Microsoft.AspNetCore from 4.6.1 and up.

And for net452, maybe apply a hotfix / patch?

[StefH]

I fixed the code to use Microsoft.AspNetCore (for net461 and up) and I tried to reproduce your problem with these tests:

[Fact] //: TODO : this test fails???
public void Request_WithPath_EncodedSpaces()
{
    // Assign
    var spec = Request.Create().WithPath("/path/a b").UsingAnyMethod();

    // when
    var body = new BodyData();
    var request = new RequestMessage(new UrlDetails("http://localhost/path/a%2fb"), "GET", ClientIp, body);

    // then
    var requestMatchResult = new RequestMatchResult();
    Check.That(spec.GetMatchingScore(request, requestMatchResult)).IsEqualTo(1.0);
}

[Fact]
public void Request_WithPath_Spaces()
{
    // Assign
    var spec = Request.Create().WithPath("/path/a b").UsingAnyMethod();

    // when
    var body = new BodyData();
    var request = new RequestMessage(new UrlDetails("http://localhost/path/a b"), "GET", ClientIp, body);

    // then
    var requestMatchResult = new RequestMatchResult();
    Check.That(spec.GetMatchingScore(request, requestMatchResult)).IsEqualTo(1.0);
}

However I need some help from you to give me correct examples + expectations. Because now the Request_WithPath_EncodedSpaces still fails.

[chrischu]

Cool!

I think that Request_WithPath_EncodedSpaces fails is to be expected. I don't expect it to work. The only case I expect to work is when at both ends the space is encoded (or not encoded like in the other test), like this:

[Fact]
public void Request_WithPath_EncodedSpaces()
{
    // Assign
    var spec = Request.Create().WithPath("/path/a%2fb").UsingAnyMethod();

    // when
    var body = new BodyData();
    var request = new RequestMessage(new UrlDetails("http://localhost/path/a%2fb"), "GET", ClientIp, body);

    // then
    var requestMatchResult = new RequestMatchResult();
    Check.That(spec.GetMatchingScore(request, requestMatchResult)).IsEqualTo(1.0);
}

[StefH]

#185

[chrischu]

I tried updating, but as expected (according to the test), the underlying problem is not fixed.

According to aspnet/AspNetKatana#208 this is only a problem in OWIN, but apparently this problem persists. I also tried setting the configuration setting (system.net/settings/httpListener.unescapeRequestUrl) since apparently the decoding behavior automatically happens in HTTP.sys

I looked at a another project (https://github.com/hibri/HttpMock) to find out how they achieve the hosting of the HTTP server and they seem to be using Kayak (https://github.com/kayakhttp/kayak) which does not have that problem (I tried reproducing it, but couldn't).

Maybe it would be an option to also use Kayak in WireMock.Net?

[StefH]

Kayak is very old and not updated recently?

[StefH]

Closing