Replace Newtonsoft.Json with System.Text.Json

[IanKemp]

(This was discussed in #2316 but deserves to be its own issue, as the work necessary to close this issue would be different to the work to close that one.)

I'm tired of typing JsonSerializer in my test projects and getting it autocompleted to Newtonsoft.Json.JsonSerializer because Microsoft.Net.Test.Sdk (via Microsoft.TestPlatform.TestHost) references (a very old version) of the now-essentially-deprecated Newtonsoft.Json, especially when my greenfields .NET Core app under test has zero references to Newtonsoft.Json.

This would also help people who do have explicit dependencies on Newtonsoft.Json and are being bitten by #2279.

[nohwnd]

We would love to do this, but we need to keep our dependencies compatible with net452 and System.Text.Json is not compatible with that.

[0xced]

It's actually much worse, the dependency on Newtonsoft.Json is not properly declared in the NuGet package.

I was trying to replace Newtonsoft.Json with System.Text.Json in Stryker.NET and the dependency on Newtonsoft.Json hit me hard. After some work to replace all JSON handling everywhere in Stryker.NET I was finally able to remove <PackageReference Include="Newtonsoft.Json" Version="13.0.1" /> from its csproj. But thanks to JetBrains Rider great feature where you can see transitive packages (Implicitly Installed Packages in Solution), I saw that Newtonsoft.Json was still referenced.

It turns out it was referenced through Buildalyzer/3.2.2 → Microsoft.Extensions.DependencyModel/2.1.0 → Newtonsoft.Json/9.0.1. So in order to completely get rid of Newtonsoft.Json I explicitly updated Microsoft.Extensions.DependencyModel to version 5.0.0 which has zero dependencies for the net5.0 target framework. I added <PackageReference Include="Microsoft.Extensions.DependencyModel" Version="5.0.0" /> to Stryker.Core.csproj and Newtonsoft.Json was definitely gone.

But then… this happened at runtime:

System.IO.FileNotFoundException: Could not load file or assembly 'Newtonsoft.Json, Version=9.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'. The system cannot find the file specified.

File name: 'Newtonsoft.Json, Version=9.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed'
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.JsonDataSerializer..ctor()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.JsonDataSerializer.get_Instance()
   at Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.SocketCommunicationManager..ctor()
   at Microsoft.TestPlatform.VsTestConsole.TranslationLayer.VsTestConsoleRequestSender..ctor()
   at Microsoft.TestPlatform.VsTestConsole.TranslationLayer.VsTestConsoleWrapper..ctor(String vstestConsolePath, ConsoleParameters consoleParameters)
   at Stryker.Core.TestRunners.VsTest.VsTestRunner.PrepareVsTestConsole()

This is because Microsoft.TestPlatform.TranslationLayer is lying about its dependencies. Its JsonDataSerializer class depends on Newtonsoft.Json on .NET Standard 2.0 but Newtonsoft.Json is not declared as a dependency.

Q.E.D.

[SimonCropp]

almost 2 years later. perhaps it is time to drop support for net452? support is ending in 2 months anyway

[michael-hawker]

I was just doing some refactoring of the base file new UWP Unit Test project to coordinate the multiple copies we have across our project to a single set of code for our custom test harness.

Doing this suddenly broke the UWP app from connecting to the vstest.console or VS runner. Took me a day, but finally saw a FileNotFoundException about Newtonsoft.Json followed by a SocketException.

Added a reference to Newtonsoft.Json directly to the UWP Unit Test csproj file and everything went back to normal... 🙁

Not sure exactly why what I did broke the dependency look up, but it seems we're using the SdkReference for TestPlatform.Universal, so I'm not exactly sure how that works. (Compared to our WinAppSDK head which continued to work fine after our refactor, but it's referencing Microsoft.TestPlatform.TestHost directly and that at least declares the old Newtonsoft dependency.

[nohwnd]

#2316 (comment) we are still researching how to get rid of that dependency altogether, unfortunately System.Text.Json does not support the data annotations we use, and we don't want to add additional dependency to object model for .NET Framework.

#3622

[SimonCropp]

@nohwnd can u point to where/how u use data annotations?

[nohwnd]

@SimonCropp Sure, for example here in ObjectModel which has no dependency on the serializer we have the TestCase object, and it is annotated with DataMember attribute.

https://github.com/microsoft/vstest/blob/main/src/Microsoft.TestPlatform.ObjectModel/TestCase.cs#L69

Theoretically we could work around this on the serializer / deserializer side by explicitly listing the properties to use for every object.

Another issue with System.Text.Json is that it is not built-in in .NET Framework, and serializers are missing for common types that we use.