There is a flaw in the file deployment process when using a DeploymentItem

[fredbleuzet]

Description

I ran into an issue and it seems the process works the way it is described here https://msdn.microsoft.com/en-us/library/ms182475.aspx
In a way it's working as designed, but the design has a flaw as all the files from a dependency are not copied to the out folder.

Steps to reproduce

1. I created the project ClassLibrary1, a library with 1 class and 1 method.

2. To the ClassLibrary1 project I added TextFile1.txt file where the "Copy to Output Directory" is set to "Copy if newer". The 1 method reads the TextFile1.txt file.
   Basically when I distribute ClassLibrary1, I need to provide both ClassLibrary1.dll and TextFile1.txt together.

3. I created a unit test project UnitTestProject1 with 1 test method that calls the 1 method.

3.a. If UnitTestProject1 doesn't have any deployment item, the test runs from the unit test's bin\Debug folder and all is well.
3.b When UnitTestProject1 has a deployment item, vstest copies to the Out folder the deployment item, UnitTestProject1.dll and ClassLibrary1.dll but not TextFile1.txt. In this case vstest copies a non-functional version of ClassLibrary1 because its whole package is not copied.

Expected behavior

If UnitTestProject1 depends on a dependent project, all the files defined as output to this dependent project should be copied to the Out folder, even non-dll files, because the dlls and non-dlls make a whole functional set of files.

This is very much what a compiler does on an application ConsoleApp1 with a dependency on the ClassLibrary1. When ConsoleApp1 compiles, the compiler brings into its bin\Debug both ClassLibrary1.dll and TextFile1.txt, because that makes a whole functional file set.

Actual behavior

Because non dll files are not copied, when I have a dependent project I also need to add as deployment items all the files that are not dlls. This is the flaw.
Compared to the console project, the compiler doesn't need me to add the non-dll files from the ClassLibrary1 project into ConsoleApp1.csproj.

Environment

Any, it works as described, it's easy to reproduce.

[jayaranigarg]

@fredbleuzet : We would recommend you not to use deployment item until and unless you absolutely need it. Can you explain your scenario as to why do you need to deploy your files to a separate location and can't run your tests from bin/debug?

[fredbleuzet]

Hi @jayaranigarg , it's a fair question.
Along the years we had many issues or surprises due to the mismatch between the bin\Debug and the Out folder.
But back to your question, I think the origin of our problems is that we have a good number of projects in an infrastructure that is database driven. All our unit tests run off of an MDF file as the database that is part of the unit test project. We can't run from the bin\Debug folder because that file is just not static. One DeploymentItem for the MDF file means deployment items for everything non assembly that is accessed. It's a bit overwhelming at time but we've managed to keep up. And VS offers a number of good features with the local database.
Because we have many libraries, people in their own project make references to dependencies and just trust that the dependencies work. This contract is a bit the base of componentizing software. So I work on my project, I reference some other project from a coworker, I create the unit tests for my functionality and my test fails because my dependency happens to use non assembly files as well.
Me myself adding deployment items in my unit tests for files that my dependency happens to need is a bit unfair and not scalable as the number of libraries increases, particularly when I know the unit test project already has all the information. After all the compilation brings everything together nicely.
We've been trying to keep up but our workarounds are becoming more and more gross when all we actually need is dependencies copied properly.

That is our particular case, I doubt it's an isolated case.
I understand that the unit test framework must be smart about the unit test file handling to minimize run time. But I hold the position that this smartness logic can't apply in the same way to dependencies.
If I use a dependent library I need to trust it works and right now the trust is broken at the unit test framework level.
You see what I mean?
I have my problems to solve so that I keep going on my side, I have solutions and they are gross so I don't want to share :)
But remains a flaw in how the non assembly files are handled.

[jayaranigarg]

@fredbleuzet : Thank you for the detailed explanation. We acknowledge this to be an issue and we are planning to fix this in our future releases.

[hamidshahid]

@jayaranigarg Is there any ETA on when the fix would happen. We have a huge test suite that we are looking to move to MSTest v2 and this is a problematic area for us.

[jayaranigarg]

@hamidshahid : We pushed a fix for this yesterday. You can consume the fix for now from https://dotnet.myget.org/feed/mstestv2/package/nuget/MSTest.TestAdapter/1.4.0-build-20181031-01
https://dotnet.myget.org/feed/mstestv2/package/nuget/MSTest.TestFramework/1.4.0-build-20181031-01.
Update both framework and adapter to these nuget packages. You will also need to add a node to your runsettings file like

<MSTestV2>
    <DeployTestSourceDependencies>false</DeployTestSourceDependencies>
</MSTestV2>

We will update the documentation and release these packages on nuget in a while.

[hamidshahid]

Thanks @jayaranigarg. I will take the new package and test it out. Many thanks

[jayaranigarg]

Fix has been released in latest(1.4.0) version of adapter. Please update your adapter and framework nuget packages to latest. Closing the thread for now. Please feel free to reopen if you hit this again.