[Bug]: Tests hang indefinitely when InitializeAsync throws exception

[andrewbabbittdev]

Description

When a set of ClassDataSources are set up in a particular manner, throwing an exception in InitializeAsync can cause the test process to hang indefinitely.

Expected Behavior

The tests should never hang due to an exception.

Actual Behavior

The tests hang.

Steps to Reproduce

I have a set of fixtures used as a part of integration tests using Aspire and WebApplicationFactory.

The setup uses a TestAppHost that sets up the testing DistributedApplication:

Note the WaitAsync call, as this throws an exception due to the timeout being exceeded. This works fine if its removed. You could also just generally throw an exception, any exception and it will cause it to hang.

public class TestAppHost : IAsyncDiscoveryInitializer, IAsyncDisposable
{
    public IDistributedApplicationTestingBuilder Builder { get; private set; } = default!;
    public DistributedApplication Application { get; private set; } = default!;

    public async Task InitializeAsync()
    {
        Builder = await DistributedApplicationTestingBuilder.CreateAsync<Projects.WebApp_AppHost>();

        Builder.Services.ConfigureHttpClientDefaults(http =>
        {
            http.AddStandardResilienceHandler();
        });

        Application = await Builder.BuildAsync();

        var resourceNotificationService = Application.Services.GetRequiredService<ResourceNotificationService>();

        await Application.StartAsync();

        await resourceNotificationService.WaitForResourceAsync("web", KnownResourceStates.Running)
            .WaitAsync(TimeSpan.FromMilliseconds(1));
    }

    public async ValueTask DisposeAsync()
    {
        await Application.DisposeAsync();
        await Builder.DisposeAsync();

        GC.SuppressFinalize(this);
    }
}

Then we have a TestWebApplicationFactory that uses the TestAppHost:

public class TestWebApplicationFactory : WebApplicationFactory<Program>, IAsyncDiscoveryInitializer
{
    [ClassDataSource<TestAppHost>(Shared = SharedType.PerTestSession)]
    public required TestAppHost AppHost { get; init; }

    private readonly Dictionary<string, string?> _environmentVariables = [];

    public async Task InitializeAsync()
    {
        var web = (IResourceWithEnvironment)AppHost.Builder.Resources.First(r => r.Name == "web");

        var options = new DistributedApplicationExecutionContextOptions(DistributedApplicationOperation.Run)
        {
            ServiceProvider = AppHost.Application.Services
        };

        var executionContext = new DistributedApplicationExecutionContext(options);

        var executionConfiguration = await ExecutionConfigurationBuilder.Create(web)
            .WithEnvironmentVariablesConfig()
            .BuildAsync(executionContext);

        foreach (var variable in executionConfiguration.EnvironmentVariables)
        {
            _environmentVariables[variable.Key.Replace("__", ":")] = variable.Value;
        }

        StartServer();
    }

    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        var configuration = new ConfigurationBuilder()
            .AddInMemoryCollection(_environmentVariables)
            .Build();

        builder.UseConfiguration(configuration);
    }
}

Then we have an AspireDataSourceAttribute that uses the TestWebApplicationFactory:

public class AspireDataSourceAttribute : DependencyInjectionDataSourceAttribute<IServiceScope>
{
    [ClassDataSource<TestWebApplicationFactory>(Shared = SharedType.PerTestSession)]
    public TestWebApplicationFactory Factory { get; init; } = default!;

    public override IServiceScope CreateScope(DataGeneratorMetadata dataGeneratorMetadata)
    {
        return Factory.Server.Services.CreateAsyncScope();
    }

    public override object? Create(IServiceScope scope, Type type)
    {
        if (type == typeof(HttpClient))
        {
            return Factory.AppHost.Application.CreateHttpClient("web");
        }

        return scope.ServiceProvider.GetService(type);
    }
}

Then we can generally just have a test use this DI attribute and it will cause the testing process to hang forever:

[AspireDataSource]
public class HealthCheckTests(HttpClient httpClient)
{
    [Test]
    public async Task HealthEndpointWorking()
    {
        var result = await httpClient.GetStringAsync("/health");

        await Assert.That(result)
            .IsEqualTo("Healthy");
    }
}

TUnit Version

1.13.11

.NET Version

10

Operating System

Windows

IDE / Test Runner

dotnet CLI (dotnet test / dotnet run)

Error Output / Stack Trace

████████╗██╗   ██╗███╗   ██╗██╗████████╗
╚══██╔══╝██║   ██║████╗  ██║██║╚══██╔══╝
   ██║   ██║   ██║██╔██╗ ██║██║   ██║
   ██║   ██║   ██║██║╚██╗██║██║   ██║
   ██║   ╚██████╔╝██║ ╚████║██║   ██║
   ╚═╝    ╚═════╝ ╚═╝  ╚═══╝╚═╝   ╚═╝

   TUnit v1.13.11.0 | 64-bit | Microsoft Windows 10.0.26200 | win-x64 | .NET 10.0.2 | Microsoft Testing Platform v2.1.0

   Engine Mode: SourceGenerated


[✓0/x0/↓0] Cortex.Web.Tests.dll (net10.0|x64)                                                                                                                        (9m 15s)

Additional Context

Issue occurs both on CLI and in VisualStudio.

I don't even know if this is the best way to actually do the apphost and waf spinup with TUnit, I know using IAsyncDiscoveryInitializer probably isnt ideal, but swapping it for IAsyncInitializer also causes all tests to fail with the following:

failed HealthEndpointWorking (0ms)
  TUnit.Engine.Exceptions.TestFailedException: InvalidOperationException: Failed to expand data source for test 'HealthEndpointWorking': Cannot access a disposed object.
  Object name: 'IServiceProvider'.
  --->  System.ObjectDisposedException: Cannot access a disposed object.
  Object name: 'IServiceProvider'.
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.ThrowHelper.ThrowObjectDisposedException()
    at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.GetService(Type serviceType)
    at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService(IServiceProvider provider, Type serviceType)
    at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.GetRequiredService[T](IServiceProvider provider)
    at Microsoft.Extensions.DependencyInjection.ServiceProviderServiceExtensions.CreateAsyncScope(IServiceProvider provider)
    at TestApp.Web.Tests.Testing.AspireDataSourceAttribute.CreateScope(DataGeneratorMetadata dataGeneratorMetadata) in Testing\AspireDataSourceAttribute.cs:22
    at TUnit.Core.DependencyInjectionDataSourceAttribute`1.<>c__DisplayClass0_0.<GenerateDataSources>b__0()
    at TUnit.Core.UntypedDataSourceGeneratorAttribute.<>c__DisplayClass1_0.<GenerateDataSourcesAsync>b__0()
    at TUnit.Engine.Building.TestBuilder.BuildTestsFromMetadataAsync(TestMetadata metadata, TestBuildingContext buildingContext)
    at TUnit.Engine.Building.TestBuilder.BuildTestsFromMetadataAsync(TestMetadata metadata, TestBuildingContext buildingContext)

Our general goal was to add a timeout, but it also doesn't seem like InitializeAsync supports timeouts natively which is a shame. If there's a better way of doing this id love to know what it is, but I dont feel like TUnit should entirely hang if an exception is thrown like this.

IDE-Specific Issue?

• I've confirmed this issue occurs when running via dotnet test or dotnet run, not just in my IDE

[thomhurst]

Can you try the latest version please

[andrewbabbittdev]

@thomhurst the update did not fix the issue. I found out I can get a stack trace for why this hangs when run in visual studio, the trace never gets logged on console even if I increase the log level:

========== Starting test discovery ==========
Connecting to client host '127.0.0.1' port '59725'
[ServerTestHost.OnTaskSchedulerUnobservedTaskException] Unhandled exception: System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (The operation has timed out.)
 ---> System.TimeoutException: The operation has timed out.
   at TestApp.Web.Tests.Testing.TestAppHost.InitializeAsync() in D:\source\me\TestApp\tests\Web\Testing\TestAppHost.cs:line 44
   at TUnit.Core.ObjectInitializer.InitializeCoreAsync(Object obj, IAsyncInitializer asyncInitializer, CancellationToken cancellationToken)
   at TUnit.Engine.Services.ObjectLifecycleService.InitializeObjectCoreAsync(Object obj, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events, CancellationToken cancellationToken)
   at TUnit.Engine.Services.ObjectLifecycleService.EnsureInitializedAsync[T](T obj, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events, CancellationToken cancellationToken)
   at TUnit.Engine.Services.PropertyInjector.ResolvePropertyDataAsync(PropertyInitializationContext context)
   at TUnit.Engine.Services.PropertyInjector.ResolvePropertyDataAsync(PropertyInitializationContext context)
   at TUnit.Engine.Services.PropertyInjector.InjectSourceGeneratedPropertyAsync(Object instance, PropertyInjectionMetadata metadata, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events, ConcurrentDictionary`2 visitedObjects)
   at TUnit.Core.Helpers.ParallelTaskHelper.ForEachAsync[T](T[] items, Func`2 action)
   at TUnit.Engine.Services.PropertyInjector.InjectPropertiesRecursiveAsync(Object instance, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events, ConcurrentDictionary`2 visitedObjects)
   --- End of inner exception stack trace ---
[ServerTestHost.OnTaskSchedulerUnobservedTaskException] Unhandled exception: System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. (Failed to inject properties for type 'TestWebApplicationFactory': The operation has timed out.)
 ---> System.InvalidOperationException: Failed to inject properties for type 'TestWebApplicationFactory': The operation has timed out.
 ---> System.TimeoutException: The operation has timed out.
   at TestApp.Web.Tests.Testing.TestAppHost.InitializeAsync() in D:\source\me\TestApp\tests\Web\Testing\TestAppHost.cs:line 44
   at TUnit.Core.ObjectInitializer.InitializeCoreAsync(Object obj, IAsyncInitializer asyncInitializer, CancellationToken cancellationToken)
   at TUnit.Engine.Services.ObjectLifecycleService.InitializeObjectCoreAsync(Object obj, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events, CancellationToken cancellationToken)
   at TUnit.Engine.Services.ObjectLifecycleService.EnsureInitializedAsync[T](T obj, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events, CancellationToken cancellationToken)
   at TUnit.Engine.Services.PropertyInjector.ResolvePropertyDataAsync(PropertyInitializationContext context)
   at TUnit.Engine.Services.PropertyInjector.ResolvePropertyDataAsync(PropertyInitializationContext context)
   at TUnit.Engine.Services.PropertyInjector.InjectSourceGeneratedPropertyAsync(Object instance, PropertyInjectionMetadata metadata, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events, ConcurrentDictionary`2 visitedObjects)
   at TUnit.Core.Helpers.ParallelTaskHelper.ForEachAsync[T](T[] items, Func`2 action)
   at TUnit.Engine.Services.PropertyInjector.InjectPropertiesRecursiveAsync(Object instance, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events, ConcurrentDictionary`2 visitedObjects)
   --- End of inner exception stack trace ---
   at TUnit.Engine.Services.PropertyInjector.InjectPropertiesRecursiveAsync(Object instance, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events, ConcurrentDictionary`2 visitedObjects)
   at TUnit.Engine.Services.PropertyInjector.InjectPropertiesAsync(Object instance, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events)
   at TUnit.Engine.Services.ObjectLifecycleService.InitializeObjectCoreAsync(Object obj, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events, CancellationToken cancellationToken)
   at TUnit.Engine.Services.ObjectLifecycleService.EnsureInitializedAsync[T](T obj, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events, CancellationToken cancellationToken)
   at TUnit.Engine.Services.PropertyInjector.ResolvePropertyDataAsync(PropertyInitializationContext context)
   at TUnit.Engine.Services.PropertyInjector.ResolvePropertyDataAsync(PropertyInitializationContext context)
   at TUnit.Engine.Services.PropertyInjector.InjectSourceGeneratedPropertyAsync(Object instance, PropertyInjectionMetadata metadata, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events, ConcurrentDictionary`2 visitedObjects)
   at TUnit.Core.Helpers.ParallelTaskHelper.ForEachAsync[T](T[] items, Func`2 action)
   at TUnit.Engine.Services.PropertyInjector.InjectPropertiesRecursiveAsync(Object instance, ConcurrentDictionary`2 objectBag, MethodMetadata methodMetadata, TestContextEvents events, ConcurrentDictionary`2 visitedObjects)
   --- End of inner exception stack trace ---

This stalling occurs on both CLI and Visual Studio still

[thomhurst]

@andrewbabbittdev Would you mind creating a repro project for this please?

[andrewbabbittdev]

@thomhurst I just went and put one together for you here: https://github.com/andrewbabbittdev/tunit-repro