Keep getting build error with Mono DryJob

[maxim-saplin]

Tried VS for Mac on Catalina and VS2019 on Windows 10. In both cases there's same CS1069 build error.
Tried running my code from inside the console app and command line. NET472 and Core 3.1 DryJobs worked fine on windows.

Here's the benchmark code:

    [DisassemblyDiagnoser]
    [KeepBenchmarkFiles]
    [DryJob(BenchmarkDotNet.Jobs.RuntimeMoniker.Mono)]
    public class Program
    {
        static void Main()
        {
            var config = DefaultConfig.Instance.With(ConfigOptions.DisableOptimizationsValidator);
            BenchmarkRunner.Run<Program>(config);
        }
...
        [Benchmark]
        public void RunFloat32()
        {
            var res = c.RunXops(1000000000, false, false);
        }

        [Benchmark]
        public void RunIntt32()
        {
            var res = c.RunXops(1000000000, false, false);
        }
}

And that's the output (just the top of the file).

// Validating benchmarks:
// ***** BenchmarkRunner: Start   *****
// ***** Found 2 benchmark(s) in total *****
// ***** Building 1 exe(s) in Parallel: Start   *****
BuildScript: /Users/user/Projects/Console/Console/bin/Debug/netcoreapp3.1/Dry-Mono.sh
// ***** Done, took 00:00:01 (1.32 sec)   *****
// Found 2 benchmarks:
//   Program.RunFloat32: Dry-Mono(Runtime=Mono, IterationCount=1, LaunchCount=1, RunStrategy=ColdStart, UnrollFactor=1, WarmupCount=1)
//   Program.RunIntt32: Dry-Mono(Runtime=Mono, IterationCount=1, LaunchCount=1, RunStrategy=ColdStart, UnrollFactor=1, WarmupCount=1)

// Build Error: The build has failed!
CS1069: The type name 'File' could not be found in the namespace 'System.IO'. This type has been forwarded to assembly 'System.IO.FileSystem, Version=4.1.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' Consider adding a reference to that assembly.


// Build Error: The build has failed!
CS1069: The type name 'File' could not be found in the namespace 'System.IO'. This type has been forwarded to assembly 'System.IO.FileSystem, Version=4.1.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' Consider adding a reference to that assembly.

[adamsitnik]

Hi @maxim-saplin

Could you please share your project file?

Thanks!

[maxim-saplin]

Here we go... On Windows TargetFrameworks also had net472

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFrameworks>netcoreapp3.1</TargetFrameworks>
  </PropertyGroup>
 <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
    <DebugType></DebugType>
    <PlatformTarget>anycpu</PlatformTarget>
  </PropertyGroup>
  <ItemGroup>
    <ProjectReference Include="..\Saplin.xOPS\Saplin.xOPS.csproj" />
  </ItemGroup>
  <ItemGroup>
    <Reference Include="System.IO.FileSystem" />
  </ItemGroup>
  <ItemGroup>
    <None Remove="Properties\PublishProfiles\FolderProfile.pubxml" />
  </ItemGroup>
  <ItemGroup>
    <PackageReference Include="BenchmarkDotNet" Version="0.12.0" />
  </ItemGroup>
</Project>

[maxim-saplin]

JIC, here's the .csproj file of referenced class library:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFrameworks>netstandard2.0</TargetFrameworks>
  </PropertyGroup>

  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
    <DebugType></DebugType>
    <Optimize>false</Optimize>
    <DefineConstants></DefineConstants>
    <DelaySign>false</DelaySign>
    <WarningsNotAsErrors></WarningsNotAsErrors>
    <Nullable></Nullable>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
    <DefineConstants>TRACE;DEBUG;NETSTANDARD;NETSTANDARD2_0</DefineConstants>
    <Optimize>false</Optimize>
  </PropertyGroup>

</Project>

[adamsitnik]

Thanks for sharing the project file.

To get Mono toolchain working on non-Windows OSes you need to target some net* moniker like net472 and compile and run the benchmark project using Mono.

Example:

<TargetFrameworks>netcoreapp3.1;net46</TargetFrameworks>

msbuild /p:Configuration=Release BenchmarkDotNet.sln
mono samples/BenchmarkDotNet.Samples/bin/Release/net46/BenchmarkDotNet.Samples.exe

To tell the long story short, when you run the host process as .NET Core, we don't know the localization of Mono dlls and hence we can't reference them properly and it fails with the unclear error that you got.

[maxim-saplin]

msbuild won't complie the project on macOS with net46 added:

error NU1202: Package BenchmarkDotNet 0.12.0 is not compatible with net46 (.NETFramework,Version=v4.6). Package BenchmarkDotNet 0.12.0 supports: netstandard2.0 (.NETStandard,Version=v2.0)

[adamsitnik]

Please try net461 or newer

[maxim-saplin]

That worked, thanks a lot!
One more clarification. On Windows if I have 3 DryJobs (Core, NET472 and Mono) in order to get mono working I can't just execute the produced console app's exe file, it has to be started via command line/mono - correct?

[adamsitnik]

If you want to run the benchmarks as Mono, the host process (what you run) must be either Mono or Full .NET Framework. This is because we need to be able to locate the framework assemblies to be able to compile the auto-generated project.

Our code for Mono and Full .NET Framework:

BenchmarkDotNet/src/BenchmarkDotNet/Toolchains/Roslyn/Builder.cs

Lines 116 to 129 in 2ba3033

	private static string[] GetFrameworkAssembliesPaths()
	{
	string frameworkAssembliesDirectory = Path.GetDirectoryName(typeof(object).Assembly.Location);
	if (frameworkAssembliesDirectory == null)
	return Array.Empty<string>();
	return new[]
	{
	Path.Combine(frameworkAssembliesDirectory, "mscorlib.dll"),
	Path.Combine(frameworkAssembliesDirectory, "System.dll"),
	Path.Combine(frameworkAssembliesDirectory, "System.Core.dll"),
	Path.Combine(frameworkAssembliesDirectory, "System.Runtime.dll")
	};
	}

BTW I encourage you to use BenchmarkSwitcher and control the target runtimes via console line arguments as descrbied in https://benchmarkdotnet.org/articles/configs/toolchains.html#multiple-frameworks-support

Whenever you want to run multiple tfms you just specify them via console line arguments without touching the source code.

class Program
{
    static void Main(string[] args) 
        => BenchmarkSwitcher
            .FromAssembly(typeof(Program).Assembly)
            .Run(args); // crucial to make it work
}

Sample command:

dotnet run -c Release -f net461 --job dry --runtimes net461 mono netcoreapp3.1 --filter '*'

[maxim-saplin]

Thanks!

[adamsitnik]

@maxim-saplin I am going to close the issue, please feel free to reopen it if something does not work or is not clear.