No failing test details with MTP and dotnet test support with xunit v3

[martincostello]

Describe the bug

I had a look through yours and xunit's documentation, but I couldn't see anything about this so I'm not sure if it's a bug or not.

---

Playing around with an existing xunit v3 project to have it use MTP, I don't seem to be able to get any useful terminal output when a test fails:

❯ dotnet test
Restore complete (0.6s)
  LondonTravel.Skill succeeded (2.4s) → C:\Coding\martincostello\alexa-london-travel\artifacts\bin\LondonTravel.Skill\debug\LondonTravel.Skill.dll
  LondonTravel.Skill.Tests succeeded (1.5s) → C:\Coding\martincostello\alexa-london-travel\artifacts\bin\LondonTravel.Skill.Tests\debug\LondonTravel.Skill.Tests.dll
  LondonTravel.Skill.Tests test failed with 1 error(s) (8.4s)
    C:\Coding\martincostello\alexa-london-travel\artifacts\bin\LondonTravel.Skill.Tests\debug\LondonTravel.Skill.Tests.dll : error run failed: Tests failed: 'C:\Coding\martincostello\alexa-london-travel\artifacts\bin\LondonTravel.Skill.Tests\debug\TestResults\LondonTravel.Skill.Tests_net9.0_x64.log' [net9.0|x64]

Test summary: total: 104, failed: 1, succeeded: 103, skipped: 0, duration: 8.2s
Build failed with 1 error(s) in 13.7s

I would expect to be able to see details about the failing test in the terminal like if TestingPlatformDotnetTestSupport=false.

I tried explicitly adding the following, but it didn't seem to make any difference:

<TestingPlatformCommandLineArguments>$(TestingPlatformCommandLineArguments) --output Detailed</TestingPlatformCommandLineArguments>

Steps To Reproduce

1. Clone martincostello/alexa-london-travel@0829037
2. Run dotnet test from the test\LondonTravel.Skill.Tests directory

Expected behavior

Information about which test has failed is shown in the terminal.

Actual behavior

A link to a log file containing the failure is shown.

[Youssef1313]

@martincostello Thanks for reaching out. Can you please try <TestingPlatformShowTestsFailure>true</TestingPlatformShowTestsFailure> or <TestingPlatformCaptureOutput>false</TestingPlatformCaptureOutput>. If that works out for you, I think we will want to document these properties in the page you linked. They are already documented in a different page though.

[martincostello]

Yep, that fixed it thanks:

❯ dotnet test
Restore complete (2.6s)
  LondonTravel.Skill succeeded (28.8s) → C:\Coding\martincostello\alexa-london-travel\artifacts\bin\LondonTravel.Skill\debug\LondonTravel.Skill.dll
  LondonTravel.Skill.Tests succeeded (5.1s) → C:\Coding\martincostello\alexa-london-travel\artifacts\bin\LondonTravel.Skill.Tests\debug\LondonTravel.Skill.Tests.dll
  LondonTravel.Skill.Tests test failed with 2 error(s) (9.5s)
    C:\Coding\martincostello\alexa-london-travel\test\LondonTravel.Skill.Tests\HelpTests.cs(27): error test failed:
      Can_Invoke_Function (1s 093ms): Shouldly.ShouldAssertException : response.Reprompt
          should not be null but was
      Stack Trace:
         at MartinCostello.LondonTravel.Skill.HelpTests.Can_Invoke_Function() in C:\Coding\martincostello\alexa-lon
      don-travel\test\LondonTravel.Skill.Tests\HelpTests.cs:line 27
         at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task, Configur
      eAwaitOptions options)
      --- End of stack trace from previous location ---
         at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
         at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
         at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task, Configur
      eAwaitOptions options)

    C:\Coding\martincostello\alexa-london-travel\artifacts\bin\LondonTravel.Skill.Tests\debug\LondonTravel.Skill.Tests.dll : error run failed: Tests failed: 'C:\Coding\martincostello\alexa-london-travel\artifacts\bin\LondonTravel.Skill.Tests\debug\TestResults\LondonTravel.Skill.Tests_net9.0_x64.log' [net9.0|x64]

Test summary: total: 104, failed: 1, succeeded: 103, skipped: 0, duration: 9.2s
Build failed with 2 error(s) in 51.7s

[Youssef1313]

Thanks for confirming @martincostello. I'm keeping this issue open to improve the documentation for the page you linked.

[Evangelink]

@martincostello come (back) to MSTest!!! ;)