Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run multiple target frameworks and architectures in single vstest.console #3412

Merged
merged 142 commits into from
May 23, 2022
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
142 commits
Select commit Hold shift + click to select a range
f70b70b
Add test
nohwnd Feb 22, 2022
91c2628
Merge branch 'main' into multi-tfm-run-in-vstestconsole
nohwnd Feb 22, 2022
a1c9028
Passthru options. Discover request needs to fix settings on criteria …
nohwnd Feb 23, 2022
409cca3
Discovery partially works, I don't update the testsettings in the req…
nohwnd Feb 23, 2022
8e5a289
Add tests for run and discovery with different TFMs.
nohwnd Feb 24, 2022
8ce64ae
First test rewrite.
nohwnd Feb 24, 2022
5d2f4ee
test sessions start
nohwnd Feb 24, 2022
8ec7a91
Manually test console mode a little bit
nohwnd Feb 25, 2022
21eee17
Add nested types and split up
nohwnd Feb 25, 2022
6549db9
Test sessions, and rework a bit
nohwnd Mar 1, 2022
3ca787d
Fix programmer tests
nohwnd Mar 1, 2022
2a85b51
Merge branch 'main' into multi-tfm-run-in-vstestconsole
nohwnd Mar 1, 2022
14e66d8
Skip source validation and enable checks
nohwnd Mar 1, 2022
a8f5da8
Fix settings check in test sessions, and remove rest of review points
nohwnd Mar 1, 2022
dbfdcd3
Fixing tests
nohwnd Mar 2, 2022
a487325
Fix some unit tests
nohwnd Mar 2, 2022
88ae4ad
Unit tests fixed
nohwnd Mar 2, 2022
d7c059c
Build
nohwnd Mar 2, 2022
4d8054d
Merge branch 'main' into multi-tfm-run-in-vstestconsole
nohwnd Mar 2, 2022
9e1f93a
Apply suggestions from code review
nohwnd Mar 3, 2022
1d35dc5
Update src/Microsoft.TestPlatform.CrossPlatEngine/Utilities/SourceDet…
nohwnd Mar 3, 2022
66283a0
Merge branch 'main' into multi-tfm-run-in-vstestconsole
nohwnd Mar 3, 2022
dfb2e81
Fix acceptance tests.
nohwnd Mar 4, 2022
aff2f4a
Apply suggestions from code review
nohwnd Mar 4, 2022
83a9fc6
Remove more workspace
nohwnd Mar 4, 2022
07e4782
Merge branch 'fix-acceptance' of https://github.com/nohwnd/vstest int…
nohwnd Mar 4, 2022
7536588
Revert non-optmize on Release
nohwnd Mar 4, 2022
a94b1bd
More fixes
nohwnd Mar 4, 2022
ef668f0
Update test/Microsoft.TestPlatform.TestUtilities/IntegrationTestBase.cs
nohwnd Mar 4, 2022
4fb3b26
Merge branch 'main' into fix-acceptance
nohwnd Mar 4, 2022
8b04531
Fix unit test
nohwnd Mar 4, 2022
e57fe1d
Missing null operator
nohwnd Mar 4, 2022
98fa6b9
Revert incorrect rename
nohwnd Mar 4, 2022
6342ee4
Fix review remarks
nohwnd Mar 4, 2022
4f95d81
Build failures
nohwnd Mar 4, 2022
0f53078
Merge branch 'main' into multi-tfm-run-in-vstestconsole
nohwnd Mar 4, 2022
b62a26d
Upgrade
nohwnd Mar 4, 2022
c7de6ad
Merge branch 'fix-acceptance' into net6-tests
nohwnd Mar 4, 2022
f15ade9
Upgrade tests
nohwnd Mar 4, 2022
115b842
Merge branch 'main' into net6-tests
nohwnd Mar 4, 2022
100b4dd
Skip web tests locally
nohwnd Mar 4, 2022
cdaf3d0
Apply suggestions from code review
nohwnd Mar 4, 2022
390d80b
Skip in non-ci
nohwnd Mar 4, 2022
543c21c
Change order of tests to run from fastest
nohwnd Mar 4, 2022
76021d2
Newer approach to reference assemblies on MacOS and Linux
nohwnd Mar 7, 2022
4a79dc0
Merge branch 'newer-reference-assemblies' into net6-tests
nohwnd Mar 7, 2022
de7aefc
Revert "Change order of tests to run from fastest"
nohwnd Mar 7, 2022
1bddbc8
Acceptance for net48
nohwnd Mar 7, 2022
c4e7ff1
Fix filter
nohwnd Mar 7, 2022
bbd4473
Apply suggestions from code review
nohwnd Mar 7, 2022
7c6d946
Update scripts/build/TestPlatform.Settings.targets
nohwnd Mar 7, 2022
f38616a
Remove taking target framework and runtime from env variables because…
nohwnd Mar 7, 2022
c9dbf4e
Matrix for mstest wip
nohwnd Mar 9, 2022
79117c9
Merge branch 'main' into test-matrix
nohwnd Mar 9, 2022
ecdb18e
MSTest adapter tests
nohwnd Mar 10, 2022
a65c00a
Add vstest console data source, even though it is names translation l…
nohwnd Mar 10, 2022
0fc39cf
Second test
nohwnd Mar 11, 2022
3b14bb4
Merge branch 'main' into test-matrix
nohwnd Mar 11, 2022
aaecf7f
Remove unused field
nohwnd Mar 11, 2022
82cb607
Translation layer x vstest.console
nohwnd Mar 11, 2022
07cdf1c
Fixup console run
nohwnd Mar 14, 2022
1b0202a
Runs tests from the big data source, but test sdk is not used.
nohwnd Mar 14, 2022
d556525
Compose path to testhost + adapter.
nohwnd Mar 14, 2022
d243b59
One big unified data source. Still no version filtering.
nohwnd Mar 15, 2022
1d22cd9
Remove quote path trimming.
nohwnd Mar 16, 2022
82ad320
Some tests run.
nohwnd Mar 16, 2022
5216498
More fun with sources.
nohwnd Mar 16, 2022
0f087c0
Tests tests.
nohwnd Mar 17, 2022
63e6275
Merge branch 'main' into test-matrix
nohwnd Mar 21, 2022
1a45042
merge
nohwnd Mar 21, 2022
6e84158
Revert unnecessary renames
nohwnd Mar 21, 2022
2b3e039
Ignore strong naming in tests
nohwnd Mar 21, 2022
5987391
Add tags
nohwnd Mar 21, 2022
0e4ce5a
Fix console path
nohwnd Mar 21, 2022
969c2b3
fix
nohwnd Mar 22, 2022
8423de3
merge main
nohwnd Mar 22, 2022
9894fa3
CI cleanup always
nohwnd Mar 22, 2022
5af7437
Print free disk space
nohwnd Mar 23, 2022
81c0b1b
Whitespace and so on
nohwnd Mar 23, 2022
eff69de
Missing temp dir
nohwnd Mar 23, 2022
874f1c2
Merge branch 'main' into test-matrix
nohwnd Mar 23, 2022
5f3f5c2
Fix env variables
nohwnd Mar 23, 2022
a1a461c
Fixes I feel like I did before
nohwnd Mar 23, 2022
5a19d19
Touch
nohwnd Mar 23, 2022
df6a0f2
Merged
nohwnd Mar 23, 2022
0075b0b
Remove comment, and revert back to using TempDirectory
nohwnd Mar 24, 2022
85085dc
Fix quoting
nohwnd Mar 24, 2022
efc810f
Use Nuget.Versioning rather than semver, and revert strong name warni…
nohwnd Mar 24, 2022
02c7d41
Fix tests
nohwnd Mar 28, 2022
6bb4103
Merge branch 'test-matrix' into multi-tfm-run-in-vstestconsole
nohwnd Mar 28, 2022
57be8db
Fix test
nohwnd Mar 28, 2022
ab3093f
Fix
nohwnd Mar 28, 2022
b1e02f5
Fix
nohwnd Mar 28, 2022
3e637fc
Merge branch 'test-matrix' of https://github.com/nohwnd/vstest into t…
nohwnd Mar 28, 2022
a828bbc
Merge branch 'test-matrix' into multi-tfm-run-in-vstestconsole
nohwnd Mar 28, 2022
d33a3f2
Fix build
nohwnd Mar 28, 2022
2ac14cb
Free space
nohwnd Mar 28, 2022
020abf0
Free space
nohwnd Mar 28, 2022
55e8714
Add results directory to blame
nohwnd Mar 28, 2022
ebb15a1
Fix
nohwnd Mar 28, 2022
8c0cc0c
Add tools and fix gitingore
nohwnd Mar 29, 2022
23b5f66
Fix build
nohwnd Mar 28, 2022
1d9003c
Add tools and fix gitingore
nohwnd Mar 29, 2022
a8e8476
Merge branch 'main' into test-matrix
nohwnd Mar 29, 2022
25c5802
Make Cancel Discovery faster and more reliable
nohwnd Mar 29, 2022
0e063a4
Merge branch 'discovery-cancel-faster-better-more-reliable' into test…
nohwnd Mar 29, 2022
4dd10bc
Add fluent assertions
nohwnd Mar 29, 2022
fd235c1
Cancellation
nohwnd Mar 29, 2022
68c4e4a
Merge
nohwnd Mar 29, 2022
70f4e81
Merge branch 'test-matrix' into multi-tfm-run-in-vstestconsole
nohwnd Mar 29, 2022
b346a3e
Renames and fixes
nohwnd Mar 29, 2022
5e81462
Merge branch 'main' into multi-tfm-run-in-vstestconsole
nohwnd Mar 29, 2022
a24d1b1
Add debugger info
nohwnd May 12, 2022
6e0ef0a
Merged
nohwnd May 12, 2022
6933548
Half merged but is missing some test fixes
nohwnd May 12, 2022
65e1e0f
Programmer tests build and run, I think I left some not passing befor…
nohwnd May 12, 2022
0b25dff
Try with test sessions
nohwnd May 12, 2022
0f1e8bd
Fix programmer tests and add DISABLE_MULTI_TFM
nohwnd May 12, 2022
f7b8e94
Re-enable warnings for multtfm when there are settings forcing framew…
nohwnd May 12, 2022
095345d
Merge branch 'main' into multi-tfm-run-in-vstestconsole
nohwnd May 13, 2022
08d86c4
Smaller PR comments addressed
nohwnd May 13, 2022
3c22472
Rewrite ParallelOperationManagerTests because that class fully changed
nohwnd May 13, 2022
d624583
Fix build warnings
nohwnd May 13, 2022
da6150c
Remove test checking that we set custom launcher on default runtime p…
nohwnd May 13, 2022
f997480
Fix NOMERGE unit tests
nohwnd May 13, 2022
8734b3a
Fix runsettings tests
nohwnd May 13, 2022
2980a29
More unit tests
nohwnd May 13, 2022
94bc0f1
Fix more unit tests
nohwnd May 13, 2022
2f6c67a
Fix unit test
nohwnd May 13, 2022
08506a8
Update resources
nohwnd May 16, 2022
750c8bb
Fix collection modified exception
nohwnd May 16, 2022
fe4218e
Don't return true where you should not
nohwnd May 16, 2022
a6dbcf2
Fix assignment
nohwnd May 17, 2022
dbb04f6
Fix tests
nohwnd May 17, 2022
d6d47a5
up versions for matrix
nohwnd May 17, 2022
902e435
More PR feedbacks
nohwnd May 17, 2022
5e2f9b0
up internal version and remove debug attribute
nohwnd May 17, 2022
287b4e6
yah for the extra parent
nohwnd May 17, 2022
7dcf780
Fix testhost startup performance and choosing x86 for mixed runs.
nohwnd May 18, 2022
da0cbdf
Clean up HandlePartialRunComplete, and fix unit tests
nohwnd May 19, 2022
6e49ebe
Fix acceptance tests
nohwnd May 19, 2022
6f5bedb
Merge branch 'main' into multi-tfm-run-in-vstestconsole
nohwnd May 19, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Skip source validation and enable checks
  • Loading branch information
nohwnd committed Mar 1, 2022
commit 14e66d8d5904568d67aa22a7292fde16f33a2b9f
1 change: 1 addition & 0 deletions test/Intent/Runner.cs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

using System.Diagnostics;
using System.Reflection;

namespace Intent;
Expand Down
27 changes: 9 additions & 18 deletions test/vstest.ProgrammerTests/MultiTFMRunAndDiscovery.cs
Original file line number Diff line number Diff line change
@@ -1,21 +1,17 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full license information.

namespace vstest.ProgrammerTests;

using Intent;
using FluentAssertions;
using vstest.ProgrammerTests.Fakes;

using Microsoft.VisualStudio.TestPlatform.CommunicationUtilities.ObjectModel;
using Microsoft.VisualStudio.TestPlatform.ObjectModel;
using Microsoft.VisualStudio.TestPlatform.ObjectModel.Client;

# if DEBUG
#endif

using vstest.ProgrammerTests.Fakes;
using Microsoft.VisualStudio.TestPlatform.Utilities;
using Intent;
using Microsoft.VisualStudio.TestPlatform.ObjectModel.Client.Payloads;

namespace vstest.ProgrammerTests;
nohwnd marked this conversation as resolved.
Show resolved Hide resolved
public class MultiTFM
{
public class MultiTFMDiscovery
Expand Down Expand Up @@ -108,8 +104,7 @@ public async Task A()
// -- assert
fixture.AssertNoErrors();
// We figure out the framework for each assembly so there should be no incompatibility warnings
// REVIEW: This should be uncommented. Commenting it now, because it is helpful to see those warnings.
// fixture.TestRunEventsRegistrar.LoggedWarnings.Should().NotContainMatch("Test run detected DLL(s) which were built for different framework*");
fixture.TestRunEventsRegistrar.LoggedWarnings.Should().NotContainMatch("Test run detected DLL(s) which were built for different framework*");

fixture.ProcessHelper.Processes.Where(p => p.Started).Should().HaveCount(2);
var startWithSources1 = testhost1.FakeCommunicationChannel.ProcessedMessages.Single(m => m.Request.MessageType == MessageType.StartDiscovery);
Expand Down Expand Up @@ -217,8 +212,7 @@ public async Task B()
// -- assert
fixture.AssertNoErrors();
// We figure out the framework for each assembly so there should be no incompatibility warnings
// REVIEW: This should be uncommented. Commenting it now, because it is helpful to see those warnings.
// fixture.TestRunEventsRegistrar.LoggedWarnings.Should().NotContainMatch("Test run detected DLL(s) which were built for different framework*");
fixture.TestRunEventsRegistrar.LoggedWarnings.Should().NotContainMatch("Test run detected DLL(s) which were built for different framework*");

fixture.ProcessHelper.Processes.Where(p => p.Started).Should().HaveCount(2);
var startWithSources1 = testhost1.FakeCommunicationChannel.ProcessedMessages.Single(m => m.Request.MessageType == MessageType.StartDiscovery);
Expand Down Expand Up @@ -326,8 +320,7 @@ public async Task C()
// -- assert
fixture.AssertNoErrors();
// We figure out the framework for each assembly so there should be no incompatibility warnings
// REVIEW: This should be uncommented. Commenting it now, because it is helpful to see those warnings.
// fixture.TestRunEventsRegistrar.LoggedWarnings.Should().NotContainMatch("Test run detected DLL(s) which were built for different framework*");
fixture.TestRunEventsRegistrar.LoggedWarnings.Should().NotContainMatch("Test run detected DLL(s) which were built for different framework*");

fixture.ProcessHelper.Processes.Where(p => p.Started).Should().HaveCount(2);
var startWithSources1 = testhost1.FakeCommunicationChannel.ProcessedMessages.Single(m => m.Request.MessageType == MessageType.StartTestExecutionWithSources);
Expand Down Expand Up @@ -435,8 +428,7 @@ public async Task D()
// -- assert
fixture.AssertNoErrors();
// We figure out the framework for each assembly so there should be no incompatibility warnings
// REVIEW: This should be uncommented. Commenting it now, because it is helpful to see those warnings.
// fixture.TestRunEventsRegistrar.LoggedWarnings.Should().NotContainMatch("Test run detected DLL(s) which were built for different framework*");
fixture.TestRunEventsRegistrar.LoggedWarnings.Should().NotContainMatch("Test run detected DLL(s) which were built for different framework*");

fixture.ProcessHelper.Processes.Where(p => p.Started).Should().HaveCount(2);
var startWithSources1 = testhost1.FakeCommunicationChannel.ProcessedMessages.Single(m => m.Request.MessageType == MessageType.StartTestExecutionWithSources);
Expand Down Expand Up @@ -561,8 +553,7 @@ public async Task E()
// -- assert
fixture.AssertNoErrors();
// We figure out the framework for each assembly so there should be no incompatibility warnings
// REVIEW: This should be uncommented. Commenting it now, because it is helpful to see those warnings.
// fixture.TestRunEventsRegistrar.LoggedWarnings.Should().NotContainMatch("Test run detected DLL(s) which were built for different framework*");
fixture.TestRunEventsRegistrar.LoggedWarnings.Should().NotContainMatch("Test run detected DLL(s) which were built for different framework*");

fixture.ProcessHelper.Processes.Where(p => p.Started).Should().HaveCount(2);
var startWithSources1 = testhost1.FakeCommunicationChannel.ProcessedMessages.Single(m => m.Request.MessageType == MessageType.StartTestExecutionWithSources);
Expand Down