Console Tool throws System.Reflection.ReflectionTypeLoadException

[TwentyFourMinutes]

So when I am trying to run the following command in the console:

dotnet benchmark Benchmarks.dll --filter *

it throws the following error at me:

Unhandled Exception: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types.
Could not load file or assembly 'System.Runtime, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

with the following StackTrace:

at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.RuntimeModule.GetTypes()
   at System.Reflection.Assembly.GetTypes()
   at BenchmarkDotNet.Extensions.ReflectionExtensions.GetRunnableBenchmarks(Assembly assembly) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Extensions\ReflectionExtensions.cs:line 131
   at BenchmarkDotNet.Running.TypeFilter.GetTypesWithRunnableBenchmarks(IEnumerable`1 types, IEnumerable`1 assemblies, ILogger logger) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Running\TypeFilter.cs:line 34
   at BenchmarkDotNet.Running.BenchmarkSwitcher.RunWithDirtyAssemblyResolveHelper(String[] args, IConfig config) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkSwitcher.cs:line 92
   at BenchmarkDotNet.Running.BenchmarkSwitcher.Run(String[] args, IConfig config) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkSwitcher.cs:line 67
   at BenchmarkDotNet.Tool.Program.OnExecute() in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet.Tool\Program.cs:line 44
--- End of stack trace from previous location where exception was thrown ---
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.Invoke(MethodInfo method, Object instance, Object[] arguments)
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.OnExecute(ConventionContext context)
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.<>c__DisplayClass0_0.<<Apply>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.<>c__DisplayClass138_0.<OnExecute>b__0()
   at BenchmarkDotNet.Tool.Program.Main(String[] args) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet.Tool\Program.cs:line 26

---

C# version: 8
.Net version: .Net Preview 5 preview 3
Console Tool version: 0.12.1

[vsevolodp]

Same issue with dotnet 3.1.

Installed with dotnet tool install --framework netcoreapp3.1 -g BenchmarkDotNet.Tool

>dotnet --info
.NET SDK (reflecting any global.json):
 Version:   5.0.100-preview.4.20258.7
 Commit:    65f0fc2cad

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.19041
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   C:\Program Files\dotnet\sdk\5.0.100-preview.4.20258.7\

Host (useful for support):
  Version: 5.0.0-preview.4.20251.6
  Commit:  47ec733ba7

.NET SDKs installed:
  3.1.201 [C:\Program Files\dotnet\sdk]
  3.1.202 [C:\Program Files\dotnet\sdk]
  3.1.300 [C:\Program Files\dotnet\sdk]
  3.1.301 [C:\Program Files\dotnet\sdk]
  5.0.100-preview.4.20258.7 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.All 2.1.19 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.1.19 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 3.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 5.0.0-preview.4.20257.10 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.1.19 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 3.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 5.0.0-preview.4.20251.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 3.1.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 3.1.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 5.0.0-preview.4.20251.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

I'm using documented sample to create dot net project as described here
dotnet benchmark BenchmarkTest.dll --filter *

> dotnet benchmark BenchmarkTest.dll --filter *
Unhandled Exception: System.Reflection.ReflectionTypeLoadException: Unable to load one or more of the requested types.
Could not load file or assembly 'System.Runtime, Version=4.2.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
Could not load file or assembly 'System.Runtime, Version=4.2.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.
   at System.Reflection.RuntimeModule.GetTypes(RuntimeModule module)
   at System.Reflection.RuntimeModule.GetTypes()
   at System.Reflection.Assembly.GetTypes()
   at BenchmarkDotNet.Extensions.ReflectionExtensions.GetRunnableBenchmarks(Assembly assembly) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Extensions\ReflectionExtensions.cs:line 131
   at BenchmarkDotNet.Running.TypeFilter.GetTypesWithRunnableBenchmarks(IEnumerable`1 types, IEnumerable`1 assemblies, ILogger logger) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Running\TypeFilter.cs:line 34
   at BenchmarkDotNet.Running.BenchmarkSwitcher.RunWithDirtyAssemblyResolveHelper(String[] args, IConfig config) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkSwitcher.cs:line 92
   at BenchmarkDotNet.Running.BenchmarkSwitcher.Run(String[] args, IConfig config) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet\Running\BenchmarkSwitcher.cs:line 67
   at BenchmarkDotNet.Tool.Program.OnExecute() in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet.Tool\Program.cs:line 44
--- End of stack trace from previous location where exception was thrown ---
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.Invoke(MethodInfo method, Object instance, Object[] arguments)
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.OnExecute(ConventionContext context)
   at McMaster.Extensions.CommandLineUtils.Conventions.ExecuteMethodConvention.<>c__DisplayClass0_0.<<Apply>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at McMaster.Extensions.CommandLineUtils.CommandLineApplication.<>c__DisplayClass138_0.<OnExecute>b__0()
   at BenchmarkDotNet.Tool.Program.Main(String[] args) in W:\Work\BenchmarkDotNet\src\BenchmarkDotNet.Tool\Program.cs:line 26

[adamsitnik]

Hi!

I was never a fan of adding the benchmark global tool that loads everything dynamically and I am not suprised that it fails...

My suggestion is to stop using it and just create a console app, install BDN and use BenchmarkRunner or BenchmakSwitcher and regular dotnet run to run it:

using System;
using System.Security.Cryptography;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

namespace MyBenchmarks
{
    public class Md5VsSha256
    {
        private const int N = 10000;
        private readonly byte[] data;

        private readonly SHA256 sha256 = SHA256.Create();
        private readonly MD5 md5 = MD5.Create();

        public Md5VsSha256()
        {
            data = new byte[N];
            new Random(42).NextBytes(data);
        }

        [Benchmark]
        public byte[] Sha256() => sha256.ComputeHash(data);

        [Benchmark]
        public byte[] Md5() => md5.ComputeHash(data);
    }

    public class Program
    {
        public static void Main(string[] args)
        {
            var summary = BenchmarkRunner.Run(typeof(Program).Assembly);
        }
    }
}

dotnet run -c Release

[WojciechNagorski]

@CodeTherapist Maybe you have time to check it? I think you made the first version of the global tool?

@adamsitnik I agree with you that global tool has no added value and does not make work easier. But I believe this problem can be fixed. But to be honest, I'm not going to spend any private time on it, I can prepare PR to remove this tool 😄

[Andrioden]

For thoose that arrive here randomly, as documented here. You can achieve the same through cli doing:

using BenchmarkDotNet.Running;

namespace Your.Namespace.Benchmarks
{
    public class Program
    {
        static void Main(string[] args) => BenchmarkSwitcher.FromAssembly(typeof(Program).Assembly).Run(args);
    }
}

dotnet run -c Release --filter *YourBenchmarkClass*

Seems like the BenchmarkDotNet.Tool indeed are redundant.

[quixoticaxis]

Are there any plans to remove the page from the documentation?
Would such PR be welcome?