"Invalid runtimeconfig.json" on Windows for paths precisely from 260 to 274 characters

[ForNeVeR]

Description

(The issue only affects Windows.)

1. dotnet new console -o net5console && cd net5console && dotnet build
2. Create several directories of various lengths. For this experiment, I'll use:
   • T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789 (272 characters, more than MAX_PATH)
   • T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\30characters_34567890123456789 (252 characters, less than MAX_PATH)
   • T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\21characters_34567890 (243 characters, less than MAX_PATH)
3. Copy all the contents from net5console\bin\Debug\net5.0 to the directories from step 2.
4. Run the following commands (via cmd, since pwsh doesn't work with certain long paths too well), and see the results:

   > T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\net5console.exe
   Hello World!
   
   > T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\30characters_34567890123456789\net5console.exe
   Hello World!
   
   > T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\21characters_34567890\net5console.exe
   Cannot use file stream for [T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\21characters_34567890\net5console.runtimeconfig.dev.json]: No such file or directory
   Cannot use file stream for [T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\21characters_34567890\net5console.runtimeconfig.json]: No such file or directory
   Invalid runtimeconfig.json [T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\21characters_34567890\net5console.runtimeconfig.json] [T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\21characters_34567890\net5console.runtimeconfig.dev.json]

So, according to what I could gather during the initial investigation of the issue, it depends on path length for the runtimeconfig.json file:

• less than 260 characters: works
• from 260 to 274 characters inclusively: doesn't work, throws "invalid runtimeconfig.json"
• 275 characters and more: works

It looks like the countermeasures were applied for it to work under long paths, but there's an arithmetic or logical error in the code that applies the workaround, so it breaks for certain path lengths.

I expect it to work in all three cases. Or, if it breaks due to the path length, I expect it to break in a predictable way, and not for paths in a certain length range.

Configuration

>dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.203
 Commit:    383637d63f

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19041
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.203\

Host (useful for support):
  Version: 5.0.6
  Commit:  478b2f8c0e

.NET SDKs installed:
  2.1.808 [C:\Program Files\dotnet\sdk]
  3.1.302 [C:\Program Files\dotnet\sdk]
  5.0.100 [C:\Program Files\dotnet\sdk]
  5.0.203 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Regression?

I don't think it is. AFAIK, it works the same in .NET Core 3.1.

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

Tagging subscribers to this area: @vitek-karas, @agocke, @VSadov
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

(The issue only affects Windows.)

1. dotnet new console -o net5console && cd net5console && dotnet build
2. Create several directories of various lengths. For this experiment, I'll use:
   • T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789 (272 characters, more than MAX_PATH)
   • T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\30characters_34567890123456789 (252 characters, less than MAX_PATH)
   • T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\21characters_34567890 (243 characters, less than MAX_PATH)
3. Copy all the contents from net5console\bin\Debug\net5.0 to the directories from step 2.
4. Run the following commands (via cmd, pwsh doesn't work with certain long paths too well), and see the results:

   > T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\net5console.exe
   Hello World!
   
   > T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\30characters_34567890123456789\net5console.exe
   Hello World!
   
   > T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\21characters_34567890\net5console.exe
   Cannot use file stream for [T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\21characters_34567890\net5console.runtimeconfig.dev.json]: No such file or directory
   Cannot use file stream for [T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\21characters_34567890\net5console.runtimeconfig.json]: No such file or directory
   Invalid runtimeconfig.json [T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\21characters_34567890\net5console.runtimeconfig.json] [T:\Temp\LongPaths\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\50characters_3456789012345678901234567890123456789\21characters_34567890\net5console.runtimeconfig.dev.json]

So, according to what I could gather during the initial investigation of the issue, it depends on path length for the runtimeconfig.json file:

• less than 260 characters: works
• from 260 to 274 characters inclusively: doesn't work, throws "invalid runtimeconfig.json"
• 275 characters and more: works

It looks like the countermeasures were applied for it to work under long paths, but there's an arithmetic or logical error in the code that applies the workaround, so it breaks for certain path lengths.

I expect it to work in all three cases. Or, if it breaks due to the path length, I expect it to break in a predictable way, and not for paths in a certain length range.

Configuration

>dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.203
 Commit:    383637d63f

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19041
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.203\

Host (useful for support):
  Version: 5.0.6
  Commit:  478b2f8c0e

.NET SDKs installed:
  2.1.808 [C:\Program Files\dotnet\sdk]
  3.1.302 [C:\Program Files\dotnet\sdk]
  5.0.100 [C:\Program Files\dotnet\sdk]
  5.0.203 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.All 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 2.1.28 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.15 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.6 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Regression?

I don't think it is. AFAIK, it works the same in .NET Core 3.1.

Author:	ForNeVeR
Assignees:	-
Labels:	area-Host, untriaged
Milestone:	-

[ForNeVeR]

The span of 14 characters (260–274) could be explained by the difference in lengths between the path to the exe file and the path to the .runtimeconfig.json, which is, too, exactly 14 characters.

[elinor-fung]

Thanks for the investigation, @ForNeVeR - really helpful.

Looks like the hosting layer normalizes the app path for long paths, then builds up the .runtimeconfig.json path using it, but then does not normalize that built-up path - hence that span of characters that you pointed out as the difference between the app path and the .runtimeconfig.json path.

I think this should be a matter of changing the runtimeconfig reading to update/store the normalized path. It is currently doing a file existence check using pal::file_exists which just calls pal::realpath on Windows, but throws away the normalized path output. The Unix version of pal::file_exists doesn't do the real path conversion though.

runtime/src/native/corehost/runtime_config.cpp

Line 403 in 62712ec

if (!bundle::info_t::config_t::probe(m_path) && !pal::file_exists(m_path))

runtime/src/native/corehost/runtime_config.cpp

Line 335 in 62712ec

if (!pal::file_exists(m_dev_path))

I wonder if this could actually be the underlying issue with dotnet/aspnetcore#27469 - cc @vitek-karas @jkotalik

[vitek-karas]

Re dotnet/aspnetcore#27469 connection - that is potentially possible - assuming the system uses symlinks to redirect to a much longer path location.

[elinor-fung]

I think there's a similar issue with .deps.json - except in that case, not having a .deps.json is not an error, so it would result in unintended behaviour instead of failure to run.

assuming the system uses symlinks to redirect to a much longer path location

Not sure how long the paths are, but it does use symlinks for D:\home: https://github.com/projectkudu/kudu/wiki/Azure-Web-App-sandbox#home-directory-access-dhome

[suwatch]

For App Service, you can tell what d:\home symbolic link to by browsing to Kudu Debug Console -> dir d:\local. The VirtualDirectory0 should show the network file share what d:\home sym-linked to. The length should be < 100 characters leaving the app 150 characters.