[Uri] System.Uri rejects otherwise valid strings with length >= 65520

[airbreather]

The System.Uri class seems to reject URI strings greater than a strangely short length (experimentally, 65520 or higher, so probably this constant). I'm not able to find any documentation of this fact at https://docs.microsoft.com/en-us/dotnet/api/system.uri?view=netcore-3.1, nor does RFC 3986 discuss length restrictions (though I went through those two sources very quickly, so this could be my error).

Most URIs I've seen are short enough, but data URIs can easily be imagined to exceed this limit.

CA1056:UriPropertiesShouldNotBeStrings and related rules have been guiding me towards using this class to represent URI strings for a while now; their documentation does not list "you want to support URI strings that exceed 65519 characters in length" as good reasons to suppress warnings.

In my perfect world, I would appreciate it if the System.Uri class could represent all URI values that are legal in RFC 3986 (or at least the subset that can be represented by System.String).

I totally understand why an artificial restriction might appear as a result of engineering trade-offs, so perhaps a decent first step might be to document it better?

Repro program

The following C# program writes out the longest URI I could manage.

Incrementing the commented line causes it to throw an exception:

Unhandled exception. System.UriFormatException: Invalid URI: The Uri string is too long.
   at System.Uri.CreateThis(String uri, Boolean dontEscape, UriKind uriKind)
   at System.Uri..ctor(String uriString)
   at Program.Main() in <<AIRBREATHER_REDACTED>>\Program.cs:line 10

Program.cs

using System;
using static System.Console;

static class Program
{
    static void Main()
    {
        const int TextLength = 65489; // any higher will throw an exception
        string txt = new string('a', TextLength);
        WriteLine(new Uri($"data:text/plain;charset=utf-8,{txt}"));
    }
}

ConsoleApp0.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.1</TargetFramework>
  </PropertyGroup>

</Project>

System Information

>dotnet --info
.NET Core SDK (reflecting any global.json):
 Version:   3.1.101
 Commit:    b377529961

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.18363
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\3.1.101\

Host (useful for support):
  Version: 3.1.1
  Commit:  a1388f194c

.NET Core SDKs installed:
  2.2.402 [C:\Program Files\dotnet\sdk]
  3.1.100 [C:\Program Files\dotnet\sdk]
  3.1.101 [C:\Program Files\dotnet\sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.14 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.2.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

To install additional .NET Core runtimes or SDKs:
  https://aka.ms/dotnet-download

[Wraith2]

The System.Uri class seems to reject URI strings greater than a strangely short length (experimentally, 65520 or higher

I'm really not sure you can consider that short.

[airbreather]

The System.Uri class seems to reject URI strings greater than a strangely short length (experimentally, 65520 or higher

I'm really not sure you can consider that short.

Relative to the majority of use cases, I suppose not. "Short" was relative to hard limits of the system, given that this is the recommended type to use for data URIs and generated query strings; i.e., I understand unstable behavior brought about by strings approaching 2 GiB in length, but this is an artificial (edit: undocumented (edit-edit: actually just poorly-documented)) limitation that triggers well before we get to that range.

It's really data URIs that I'm concerned about here, since base64-encoded data streams hit this limit before the payload reaches 48 KiB.

[scalablecory]

I believe we have an issue for this already, but I can't find it.

CC @MihaZupan

[MihaZupan]

This is a documented length limit for Uri

See https://docs.microsoft.com/en-us/dotnet/api/system.uri.-ctor?view=netcore-3.1#System_Uri__ctor_System_String_

One of the conditions listed for throwing UriFormatException:

The length of uriString exceeds 65519 characters.

I agree that we can add a comment about this limit to the main Uri page as rn it is only documented on constructors.

[MihaZupan]

Changing the length limit on Uri is not a trivial change.

If you require more info about specific sections of really long data Uris, I would recommend writing a small parser specific to data uris, as the format seems trivial to parse.

[airbreather]

If you require more info about specific sections of really long data Uris, I would recommend writing a small parser specific to data uris, as the format seems trivial to parse.

FWIW, the issue came up when a user of my library was trying to use a "dumb" record type that happens to contain a URI. I'm not directly using the services that System.Uri adds on top of System.String, other than validating that the string is a valid URI.

My data model uses System.Uri primarily because the following FxCop rules document that it is best-practice to strongly favor System.Uri over System.String for all API members that represent URIs:

1. CA1054:UriParametersShouldNotBeStrings
2. CA1055:UriReturnValuesShouldNotBeStrings
3. CA1056:UriPropertiesShouldNotBeStrings

Those pages all include explanations along the lines of:

A string representation of a URI is prone to parsing and encoding errors, and can lead to security vulnerabilities. The System.Uri class provides these services in a safe and secure manner.

I can, of course, work around this issue by exposing an alternative way to create an instance of the GpxWebLink class that supports pass-through for an overlong URI, but that means exposing the user to all of the bad things that the quoted text above warns about. I really like to incorporate concepts of "fail-fast" and "pit of success" into my API designs, and the workaround is the antithesis of both.

[GrabYourPitchforks]

Is it the intent of your data model that your System.Uri field be able to handle arbitrary file uploads? I realize URLs theoretically can be of arbitrary length, but in practice I imagine the majority of Uri's consumers thinking of it as "what goes into the address bar in the web browser?"

[airbreather]

Is it the intent of your data model that your System.Uri field be able to handle arbitrary file uploads?

Not really, no.

The library defines a .NET object model that matches the XML schema for GPX files. The type in question is the equivalent of linkType in the XML schema.

The idea is that the library can be used for either:

1. Parsing an existing file into the equivalent representation in the .NET object model, or
2. Writing the .NET representation of a GPX file to a new file

It's part of the NetTopologySuite project, so this object model representation isn't necessarily the beginning or the end of the chain (there are converters to go between this direct object model translation and the standard NetTopologySuite geometry types), but some people find enough value in just being able to interact with GPX objects more directly.

From what I gather from IsraelHikingMap/Site#1128 and NetTopologySuite/NetTopologySuite.IO.GPX#39:

• It looks like in the specific application that's using this library, users are allowed to create waypoint objects that represent points-of-interest with their own images, and then save that out as GPX files.
• Apparently, there's a way to allow embedding those images directly in the GPX file, which is where the users have started running into the error.
  • Perhaps some GPS device can decode them offline? I'm not immediately sure about specifically why it's being put there, to be honest, but it seemed reasonable enough for me to accept at face value since I've heard of smallish images being encoded in data URIs in other contexts. 50 KiB is a little on the big side, but I think it's not outlandish.

I realize URLs theoretically can be of arbitrary length, but in practice I imagine the majority of Uri's consumers thinking of it as "what goes into the address bar in the web browser?"

Of course -- I'm sure that's why I can't seem to actually find other widespread complaints about this from the fundamental System.Uri side of things, despite this type being around for so long (though I have found evidence of people running into issues that flow from this same constant, e.g., #14674).

[HarelM]

As explained by @airbreather the site uses NetTopologySuite.IO.GPX to serialize files.
We want to store the content of an image inside this GPX file by using a base64 string to represent the image.
Creating multiple files is not part of the GPX sceme and opening this file offline should present the image.
The solution we decided was to use the URI field to store this base64 data, which as far as I understand is a valid value.
Bottom line: URL="data:image/jpg;base64, dksljfhalskjdfh... is our use case.