Skip to content

UX: dotnet-test filtering syntax is difficult to discover #334

Closed

Description

Given current design, I would expect many new dotnet users will struggle to figure out how to use dotnet-test arguments, such as the --filter syntax on command line. Even after reading docs, looking at examples, and fiddling with dotnet-test, it took a dozen tries to figure out how to get it right.

Here's a summary of what happened in the first 10 minutes I tried to use dotnet-test.

Setup

Install dotnet-cli 1.0.0-preview5-004478
dotnet new -t xunittest
dotnet restore

What happened next

dotnet test ✅ Yay, it worked! Now let's see if I can filter to just one method.

dotnet test -method Tests.Tests.Test1 ❌ Tried the syntax that worked in dotnet-test-xunit

MSBUILD : error MSB1001: Unknown switch.
Switch: -method
For switch syntax, type "MSBuild /help"

dotnet test /help

Wall of text. MSBuild spits out 237 lines of how to use MSBuild. Nothing in there is relevant to dotnet-test

dotnet test --help 👍 getting closer. NB: I had to guess this was right. Nothing in the previous lines of output suggested this was available.

(Correct dotnet-test help output).

(Skims help output, sees --filter is available. Here is where I made a mistake. If I had read all of the help output, I might have figured it out sooner. But I didn't. My eyes were drawn to the left side of the help output which shows args, not the right side which included some examples of how to use --filter.

Attempt to use --filter. ❌

dotnet test --filter Test1
dotnet test --filter Name=Test1
dotnet test --filter "Name=~Test1"
dotnet test -f Tests.Tests.Test1 # this one actually collides with --framework...
dotnet test --filter Tests.Tests.Test1
dotnet test --filter "Name~Tests.Tests.Test1"

Most of the time, dotnet-test show this

Starting test execution, please wait...
Warning: No test is available in /Users/namc/dev/msbuild-demos/netcore/2-test/bin/Debug/netcoreapp1.0/2-test.dll. Make sure that installed
test discoverers & executors, platform & framework version settings are appropriate and try again.

Revisit dotnet test --help again. Finally discover that the correct syntax to filter is this:
dotnet test --filter FullyQualifiedName=Tests.Tests.Test1

Next steps
Create a shell script to invoke this because typing FullyQualifiedName= repeatedly will be a pain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions