BenchmarkDotNet as global tool

[CodeTherapist]

This resolves #213.

[adamsitnik]

Hi @CodeTherapist

big thanks for doing this!

How can this be tested and deployed as a dotnet global tool?

@AndreyAkinshin do you like the BenchmarkDotNet.Tool name? maybe BenchmarkDotNet.Runner` would be better?

[CodeTherapist]

How can this be tested and deployed as a dotnet global tool?

Testing
I could add a unit test that executes the .exe against a sample benchmark dll - that would at least prove that the tool is able to be executed for the most common scenario. On the other hand, it is a really thin layer around the well tested BenchmarkSwitcher.

Deployment
The distribution of the tools are nuget packages. That would mean the nuget package of the tool must be uploaded to nuget.org to be available for everyone. As far as I understood, the nuget package should be created by the cake build.

Naming
Maybe I'm too cautious. I used the name BenchmarkDotNet.Tool to avoid any confusion with the existing class named BenchmarkRunner.

[AndreyAkinshin]

I think BenchmarkDotNet.Tool is a good name because many other .NET global tools use the same convention (e.g., see Cake.Tool, GitVersion.Tool, NuGetUtils.Tool.Restore). We can find a big list of different tool names here: https://github.com/natemcmaster/dotnet-tools It seems that there is no the universal convention, but you can find many titles with Tools, GlobalTool, and so on. I also don't like the BenchmarkDotNet.Runner title because of the clash with existed runner infrastructure.

@CodeTherapist could you also add a documentation article about this tool?

[CodeTherapist]

@AndreyAkinshin Yes, sure! Since that is my first contribution to this project, I'm unsure where to put that in the current docs structure. Maybe in docs/articles/guides? Where do you think is it to be placed right?

[AndreyAkinshin]

Maybe in docs/articles/guides?

Yeah, it's a good place.

[adamsitnik]

@AndreyAkinshin I did not know about this convention, it sounds good to me now.

btw @CodeTherapist could you try to import some of the MsBuild properies from https://github.com/dotnet/BenchmarkDotNet/blob/master/build/common.props file?

We import it in following way in all our projects:

BenchmarkDotNet/src/BenchmarkDotNet/BenchmarkDotNet.csproj

Line 2 in f354659

<Import Project="..\..\build\common.props" />

[CodeTherapist]

@adamsitnik Yes, I will do.
What do you think about to use the built-in MSBuild extensibility feature (#1011)?

[CodeTherapist]

could you also add a documentation article about this tool?

@AndreyAkinshin Added documentation! Please let me know if that is enough for a first version.

[adamsitnik]

@CodeTherapist awesome! the code looks great to me! thanks for doing that!

[adamsitnik]

@AndreyAkinshin please take a look and merge if you are ok with the code. I am!

[adamsitnik]

@ViktorHofer so now we are going to have a runner tool ;)

[AndreyAkinshin]

@CodeTherapist @adamsitnik It would be nice to rename the command from "benchmarkdotnet" to just "benchmark". It will be easier to type dotnet benchmark in the command line. What do you think?

P.S. I need some time to test this tool in different environments. But I'm definitely going to merge it in v0.11.4.

[CodeTherapist]

@AndreyAkinshin I thought about both variations: benchmark and benchmarkdotnet. I decided to the last because it matches the product name - but I'm fine also with benchmark.

[adamsitnik]

@AndreyAkinshin I want to try to write cross platform disassembler as global tool. I created a new branch called "tools" and I am going to merge this PR to the "tools" branch and work from there