Use MemoryMappedFileAccess.Read when searching a memory mapped file

[0xced]

Since we're only interested in searching for a pattern inside a memory mapped file, explicitly specify MemoryMappedFileAccess.Read (instead of the default MemoryMappedFileAccess.ReadWrite) in order to avoid IO exceptions on Windows.

[ghost]

Tagging subscribers to this area: @vitek-karas, @agocke
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Since we're only interested in searching for a pattern inside a memory mapped file, explicitly specify MemoryMappedFileAccess.Read (instead of the default MemoryMappedFileAccess.ReadWrite) in order to avoid IO exceptions on Windows.

Author:	0xced
Assignees:	-
Labels:	area-HostModel, community-contribution
Milestone:	-

[vitek-karas]

@0xced Do you have an example of the IO failure you refer to?

@VSadov Could you please review the change?

[0xced]

Do you have an example of the IO failure you refer to?

Yes, here are reproduction steps.

SingleFileIsBundle.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net6.0</TargetFramework>
    <DebugType>embedded</DebugType>
    <PublishSingleFile>true</PublishSingleFile>
    <SelfContained>false</SelfContained>
    <UseCurrentRuntimeIdentifier>true</UseCurrentRuntimeIdentifier>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.HostModel" Version="3.1.16" />
  </ItemGroup>

</Project>

Program.cs

using System;
using Microsoft.NET.HostModel.AppHost;

try
{
    Console.WriteLine($"HostWriter.IsBundle({Environment.ProcessPath})");
    var isBundle = HostWriter.IsBundle(Environment.ProcessPath, out var bundleHeaderOffset);
    Console.WriteLine($"isBundle: {isBundle}");
    Console.WriteLine($"bundleHeaderOffset: {bundleHeaderOffset}");
}
catch (Exception exception)
{
    Console.Error.WriteLine(exception);
}

On Windows, run the following commands:

dotnet publish -c Release
.\bin\Release\net6.0\win-x64\publish\SingleFileIsBundle.exe

After 50+ seconds (500 × 100ms retries) the following error occurs:

System.IO.IOException: The process cannot access the file '[…]\SingleFileIsBundle\bin\Release\net6.0\win-x64\publish\SingleFileIsBundle.exe' because it is being used by another process.
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize)
   at System.IO.MemoryMappedFiles.MemoryMappedFile.CreateFromFile(String path, FileMode mode, String mapName, Int64 capacity, MemoryMappedFileAccess access)
   at Microsoft.NET.HostModel.AppHost.HostWriter.<>c__DisplayClass4_0.<IsBundle>g__FindBundleHeader|0()
   at Microsoft.NET.HostModel.RetryUtil.RetryOnIOError(Action func)
   at Microsoft.NET.HostModel.AppHost.HostWriter.IsBundle(String appHostFilePath, Int64& bundleHeaderOffset)
   at Program.<Main>$(String[] args) in […]\SingleFileIsBundle\Program.cs:line 7

On macOS, running the following commands works fine.

dotnet publish -c Release
./bin/Release/net6.0/osx-x64/publish/SingleFileIsBundle

This produces the following result:

HostWriter.IsBundle([…]/SingleFileIsBundle/bin/Release/net6.0/osx-x64/publish/SingleFileIsBundle)
isBundle: True
bundleHeaderOffset: 165456

[vitek-karas]

Thanks.

Side note: I'm curious about the scenario where you're trying to determine if the current app is single-file or not? What do you need the information for?
Also note that the Microsoft.NET.HostModel package is not intended as a public API, it's only intended use is from the SDK.

[0xced]

Side note: I'm curious about the scenario where you're trying to determine if the current app is single-file or not? What do you need the information for?

I want to get the default DependencyContext of a single-file application. This is currently not supported but can be achieved once the location (offset + size) of the bundled deps.json file within the app host is known. I'm currently working on it in the context of serilog/serilog-settings-configuration#304.

Also note that the Microsoft.NET.HostModel package is not intended as a public API, it's only intended use is from the SDK.

That's good to know! I think this should be mentioned in the NuGet package description, currently it only says Abstractions for modifying .net core host-binaries. This could easily lead people to think it's fine for public API consumption.

[VSadov]

If we want to avoid file sharing issues, I am not sure that this change is sufficient.
The default file sharing mode for MemoryMappedFile.CreateFromFile is FileShare.None, so in other places a different pattern is used -

runtime/src/libraries/System.Reflection.Metadata/src/System/Reflection/Metadata/MetadataReader.netstandard.cs

Line 57 in 6a889d2

fileStream = new FileStream(assemblyFile, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize: 1, useAsync: false);

[agocke]

@VSadov How do we know which APIs will be used for the bundle?

[danmoseley]

Looking at old PR's ... is this change a reasonable cleanup, even if not necessarily a complete fix for anything?

[VSadov]

@danmoseley - the idea of not locking the file for other readers, since we only read, makes sense.
The problem is however not in the access mode, that the PR tries to fix, but in the sharing mode. Confusingly, these are different things.

Even if we creating a readonly mapping, the underlying API maps with FileShare.None and noone else will be able to read. A different API needs to be used to achieve the desired result.

[danmoseley]

@VSadov which underlying API? MMF opens with FileShare.Read:

runtime/src/libraries/System.IO.MemoryMappedFiles/src/System/IO/MemoryMappedFiles/MemoryMappedFile.cs

Line 146 in 8293726

SafeFileHandle fileHandle = File.OpenHandle(path, mode, GetFileAccess(access), FileShare.Read, FileOptions.None);

This succeeds:

using (var memoryMappedFile = MemoryMappedFile.CreateFromFile(@"c:\proj\testmmf", FileMode.Open, null, 0, MemoryMappedFileAccess.Read))
using (var memoryMappedFile2 = MemoryMappedFile.CreateFromFile(@"c:\proj\testmmf", FileMode.Open, null, 0, MemoryMappedFileAccess.Read))
{
    Console.WriteLine("success");
    Console.Read();
}

[VSadov]

@danmoseley - looks like this was fixed quite a while ago in e0205a6

[VSadov]

we may want to undo use of patterns like

runtime/src/libraries/System.Reflection.Metadata/src/System/Reflection/Metadata/MetadataReader.netstandard.cs

Line 57 in 6a889d2

fileStream = new FileStream(assemblyFile, FileMode.Open, FileAccess.Read, FileShare.Read, bufferSize: 1, useAsync: false);

[VSadov]

LGTM, Thanks!!