Allow ArgumentsSource to reference methods in other types

[JimmyCushnie]

According to the docs for ArgumentsSource:

The source must be within benchmarked type!

It would be nice to lift this restriction. The attribute constructor could have an overload that takes a Type parameter.

For example, NUnit's TestCaseSource does this.

A common pattern in my code is that I have one class which defines all the different cases that a method might encounter. I feed the results of this class into both my unit tests and my benchmarks, to test each case for both correctness and performance. Currently, my benchmark code is bloated with methods that just return immediately with the result of a method in the cases class, which I need to do because ArgumentsSource can't directly access the method I want to use for the arguments. I would love to be able to remove all those methods.

Example code as it currently is:

[Benchmark]
[ArgumentsSource(nameof(MethodACases))]
public MethodAResults BenchmarkMethodA(MethodACase @case)
    => CodeBeingBenchmarked.MethodA(@case.Param1, @case.Param2, @case.Param3);

public static IEnumerable<MethodACase> MethodACases() => DiagnosticsCases.MethodACases();

Example code as I propose it ought to be:

[Benchmark]
[ArgumentsSource(typeof(DiagnosticsCases), nameof(DiagnosticsCases.MethodACases))]
public MethodAResults BenchmarkMethodA(MethodACase @case)
    => CodeBeingBenchmarked.MethodA(@case.Param1, @case.Param2, @case.Param3);

[timcassell]

I think it's a reasonable request. The source method would probably have to be static if it's in a different type, though (the current implementation allows instance methods).

Accepting PRs.

[JimmyCushnie]

The source method would probably have to be static if it's in a different type, though

Looking at the code, I don't believe this is the case. The current implementation just creates a new instance of the target type, and we can do the same with a custom type. It's not like it's using the same instance that the benchmarks are being run on.

BenchmarkDotNet/src/BenchmarkDotNet/Running/BenchmarkConverter.cs

Lines 310 to 314 in 93d12c4

	if (paramsSourceMethod != default)
	return (paramsSourceMethod, ToArray(
	paramsSourceMethod.Invoke(paramsSourceMethod.IsStatic ? null : Activator.CreateInstance(parentType), null),
	paramsSourceMethod,
	parentType));

[timcassell]

Yes, I suppose it's possible, but it would require the type to have a public, parameterless ctor, which doesn't provide any more value than a static function. Fwiw, I don't like that we even support instance methods currently for the same reason (also, it's confusing that it doesn't use the same instance that the benchmarks are ran on).

[timcassell]

I'm not saying you need to go out of your way to not support instance methods, though, if it makes sense to re-use the implementation for both cases.

[JimmyCushnie]

Fwiw, I don't like that we even support instance methods currently for the same reason (also, it's confusing that it doesn't use the same instance that the benchmarks are ran on).

I completely agree, and maybe this is something to change in a followup PR.

Accepting PRs.

Thanks very much! I just opened one: #2748