Description
openedon Mar 5, 2021
Description
After upgrade to MSTest.TestAdapter version 2.2.1 from 2.1.2, our [DataTestMethod] tests changed behavior. TestContext.TestName now contains a badly serialized representation of the data row being passed in.
We are using test names to construct result file paths. This change breaks them on Windows due to Windows file system rules. The package didn't have a major version rev, so this behavior change is unexpected.
If this one intentional, how do we get the old behavior?
Steps to reproduce
What steps can reproduce the defect?
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System;
namespace TestProject
{
[TestClass]
public class UnitTest1
{
public TestContext TestContext { get; set; }
[TestMethod]
public void TestMethod1()
{
}
[DataTestMethod]
[DataRow("hello", "there")]
[DataRow("general", "kenobi")]
public void Foo(string one, string two)
{
Assert.Fail($"{this.TestContext.TestName} - {one} - {two}");
}
}
}Expected behavior
On 2.1.2, we see the following output from dotnet test:
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Failed Foo [36 ms]
Failed Foo (hello,there) [33 ms]
Error Message:
Assert.Fail failed. Foo - hello - there
Stack Trace:
at TestProject.UnitTest1.Foo(String one, String two) in C:\Users\majastrz\source\repos\SerializedGarbage\TestProject\UnitTest1.cs:line 21
Failed Foo (general,kenobi) [< 1 ms]
Error Message:
Assert.Fail failed. Foo - general - kenobi
Stack Trace:
at TestProject.UnitTest1.Foo(String one, String two) in C:\Users\majastrz\source\repos\SerializedGarbage\TestProject\UnitTest1.cs:line 21
Failed! - Failed: 2, Passed: 1, Skipped: 0, Total: 3, Duration: 37 ms - TestProject.dll (netcoreapp3.1)
The relevant part is this: Assert.Fail failed. Foo - hello - there
Actual behavior
On 2.2.1, we see the following output from dotnet test:
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Failed Foo [100 ms]
Failed Foo (hello,there) [97 ms]
Error Message:
Assert.Fail failed. Foo(System.String,System.String) - hello - there
Stack Trace:
at TestProject.UnitTest1.Foo(String one, String two) in C:\Users\majastrz\source\repos\SerializedGarbage\TestProject\UnitTest1.cs:line 21
Failed Foo (general,kenobi) [< 1 ms]
Error Message:
Assert.Fail failed. Foo(System.String,System.String) - general - kenobi
Stack Trace:
at TestProject.UnitTest1.Foo(String one, String two) in C:\Users\majastrz\source\repos\SerializedGarbage\TestProject\UnitTest1.cs:line 21
Failed! - Failed: 2, Passed: 1, Skipped: 0, Total: 3, Duration: 101 ms - TestProject.dll (netcoreapp3.1)
The relevant part is this: Assert.Fail failed. Foo(System.String,System.String) - hello - there
Environment
Repro project file:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp3.1</TargetFramework>
<IsPackable>false</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.9.1" />
<PackageReference Include="MSTest.TestAdapter" Version="2.1.2" />
<PackageReference Include="MSTest.TestFramework" Version="2.1.2" />
<PackageReference Include="coverlet.collector" Version="3.0.3">
<PrivateAssets>all</PrivateAssets>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
</PackageReference>
</ItemGroup>
</Project>
The behavior in VS after the upgrade is also strange. TestExplorer chokes on the project after upgrade and refuses to run tests. However, this is reproducible via dotnet test outside of VS as well.
