Data driven tests are incompatible with Azure Test Plans after moving to MSTest.TestAdapter 2.2.4+

[TroyWalshProf]

Description

MSTest.TestAdapter 2.2.4 (and above) break the Visual Studio Test task integration when it comes to data driven tests

Steps to reproduce

1. Open or create a Azure DevOps project
2. Create a test case work item
   
3. Create a new MSTest based test project, make sure you are referencing MSTest.TestAdapter version 2.2.3
4. Create a data driven test
   
5. Associate the the data driven test with a test case
   
6. Create a test plan and add the test case
   
7. Create a pipeline to build and run the test
8. Add the Visual Studio Testing Task to the pipeline. Use the testPlan testSelector link up the test plan, test suite, end test configuration from step 6.
9. Run the pipeline and note that the tests results appear in your build and test plan
   
   
10. Go back into your test code and upgrade MSTest.TestAdapter to 2.2.8, check in the change and try rerunning the tests
11. Note* The tests look like they run correctly, but on the first iteration of the data driven test is executed
    
    
12. Got back into your test code and add "[assembly: TestDataSourceDiscovery(TestDataSourceDiscoveryOption.DuringExecution)]" just below the using statements in your test class file.
    
13. Note* The build now shows the data driven test as other and Azure Test plans show the tests as In Progress.
    
    

Expected behavior

Data driven tests would work with Azure Test Plans

Actual behavior

If you don't use [assembly: TestDataSourceDiscovery(TestDataSourceDiscoveryOption.DuringExecution)] the data driven tests are not run.
If you use [assembly: TestDataSourceDiscovery(TestDataSourceDiscoveryOption.DuringExecution)] only the first iteration of the data driven tests is run.

Environment

Azure DevOps
MSTest.TestAdapter 2.2.4 and above
Sample project and code: https://dev.azure.com/MagenicOpen/

[TroyWalshProf]

@nohwnd - Any idea if this is something that will get fixed?

[TroyWalshProf]

@Haplois - do you know if this issue is on the team's radar?

[TroyWalshProf]

@AbhitejJohn
@jayaranigarg

• Any idea if this is something that will get fixed?

[Evangelink]

Hi @TroyWalshProf. I will move forward by closing this issue as it overlaps with #1026 and there are more discussions on the other ticket. Please comment on the other ticket.