Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port BasicAddMissingReference.InvokeSomeFixesInVisualBasicThenVerifyReferences to the new test framework #57626

Merged
merged 6 commits into from
Nov 12, 2021

Conversation

sharwell
Copy link
Member

@sharwell sharwell commented Nov 8, 2021

Roslyn's integration tests use a custom harness, where test code executes in a separate process (testhost.x86.exe or similar) and use .NET Remoting to invoke methods on objects within a Visual Studio instance. This pull request creates a second integration test project using the new test harness from microsoft/vs-extension-testing. In addition to moving maintenance of the test harness infrastructure to a common repository, this change significantly reduces integration test reliance on .NET Remoting in hopes of improving test performance and reliability by executing test code directly within the Visual Studio instance. It also standardizes diagnostics collection (screenshots, Windows event logs, the activity log, and IDE state at point of failure).

This change ports one notoriously flaky test from Roslyn's harness to the new harness, along with its required supporting methods. The new code differs primarily in the following ways:

  • Where possible, non-DTE Visual Studio APIs are preferred to DTE APIs
  • Where possible, operations are written as asynchronous (this especially includes the test itself, since the test entry point is now the main thread of a Visual Studio instance)
  • Since tests run directly within Visual Studio, there is no need to separate in-proc and out-of-proc test objects

The goal of the new test project in the current limited form is twofold:

  1. Evaluate the reliability of the test when run under the new harness
  2. In the event of failure, compare our ability to diagnose the failure in the new harness with information provided by the old harness

@sharwell sharwell force-pushed the vs-extension-testing branch from 734f5bf to b2e1d2f Compare November 9, 2021 00:38
@sharwell sharwell force-pushed the vs-extension-testing branch from c66ecb5 to 8bb0dcf Compare November 10, 2021 17:19
@sharwell sharwell marked this pull request as ready for review November 10, 2021 20:07
@sharwell sharwell requested review from a team as code owners November 10, 2021 20:07
await JoinableTaskFactory.SwitchToMainThreadAsync(cancellationToken);

var filePath = await GetAbsolutePathForProjectRelativeFilePathAsync(projectName, relativeFilePath, cancellationToken);
VsShellUtilities.OpenDocument(ServiceProvider.GlobalProvider, filePath, VSConstants.LOGVIEWID.Code_guid, out _, out _, out _, out var view);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iirc there is an async document open service now - https://devdiv.visualstudio.com/DevDiv/_git/VS.RPC.Contracts?path=%2Fsrc%2FMicrosoft.VisualStudio.RpcContracts%2FOpenDocument%2FIOpenDocumentService.cs&_a=contents&version=GBmain

It might more reliably set the focus and there are options to activate it, set selection, set visible ranges, etc.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea how one would use this. @olegtk ?

@sharwell sharwell merged commit f948cdc into dotnet:main Nov 12, 2021
@sharwell sharwell deleted the vs-extension-testing branch November 12, 2021 19:22
@ghost ghost added this to the Next milestone Nov 12, 2021
@allisonchou allisonchou modified the milestones: Next, 17.1.P2 Nov 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants