Description

For a defect reproducable from the vstest command line, describe the issue you've observed.

We NuGet.Client team are working on running tests in blame mode to collect dumps when tests hang on the CI pipeline.

Linux tests job on the CI pipeline is configured to timeout after 45 mins. We publish build artifacts such as dump,sequence.xml and test results.xml from TestResultsDirectory for trouble shooting purposes only if job status is either failed or canceled for troubleshooting.

Tests are executed in parallel. one of the PR builds yesterday, blame detector identified a hang, collected the dump, and test framework started executing other test assemblies. In the mean-time Azure DevOps pipeline cancelled this job due to 45 min timeout. Unfortunately, only sequence.xml was uploaded to build artifacts but not dump file. As per below log dump of the hang was collected but not sure why the dump was not saved to TestResultsDirectory. Hence it was not published to the artifacts (see image below).

[Long Running Test] 'NuGet.Commands.Test.FallbackFolderRestoreTests.FallbackFolderRestore_AllPackagesFoundInFallback_NuspecModeAsync', Elapsed: 00:14:02
[Long Running Test] 'NuGet.Commands.Test.RuntimeTargetsTests.RestoreTargets_RestoreWithRuntimesAsync', Elapsed: 00:14:03
[Long Running Test] 'NuGet.Commands.Test.NETCoreProject2ProjectTests.NETCoreProject2Project_VerifyCompileForTransitiveSettings(expected: "BDXY", ab: True, bc: False, cd: True, ax: True, xy: True, yd: True)', Elapsed: 00:14:02
  Blame: Creating hang dump of process dotnet (5225).
  The active test run was aborted. Reason: Test host process crashed
  Data collector 'Blame' message: The specified inactivity time of 15 minute/s has elapsed. Collecting a dump and killing the test host process.

Here is the .runsettings file set to 15 mins test timeout to collect hang dump.

<RunSettings>
    <LoggerRunSettings>
        <Loggers>
            <Logger friendlyName="blame" enabled="True" />
        </Loggers>
    </LoggerRunSettings>
    <DataCollectionRunSettings>
        <DataCollectors>
            <DataCollector friendlyName="blame" enabled="True">
                <Configuration>
                    <CollectDumpOnTestSessionHang TestTimeout="15min" HangDumpType="full" />
                    <ResultsDirectory>%AGENT_TEMPDIRECTORY%</ResultsDirectory>
                </Configuration>
            </DataCollector>
        </DataCollectors>
    </DataCollectionRunSettings>
</RunSettings>

Expected behavior

Dump that was collected due to test hang saved to the TestResults Directory so that next steps in the pipeline could have published it to artifacts for troubleshooting.

Actual behavior

CollectDumpOnTestSessionHang doesn't save dump file to TestResults directory when Azure DevOps pipeline job was canceled due to the timeout

Environment

• Tests are executed using .NET 5 Preview 7 SDK
• As per this PR description, no additional tools are to collect hang dumps on Windows and Linux. Hence added only DataCollector > CollectDumpOnTestSessionHang in .runsettings file.
• xUnit.net VSTest Adapter v2.4.1 for running tests

I appreciate any work around or additional guidance on this issue. Thanks.