OpenApiDescriptorBuilder reads ApiExplorer during startup, racing endpoint registration: "Collection was modified" aborts the host #4500

⚠️ Bug report written by Claude.

Describe the bug

When something reads Wolverine's HTTP diagnostics while the ASP.NET pipeline is still being built, Wolverine.Http.Diagnostics.OpenApiDescriptorBuilder.TryBuildForWolverine enumerates IApiDescriptionGroupCollectionProvider, which walks the DI EndpointDataSource. If endpoints are still being added on the startup thread, the enumeration throws and the host fails to start:

crit: Microsoft.AspNetCore.Hosting.Diagnostics[6]
      Application startup exception
      System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
         at System.Collections.Generic.List`1.Enumerator.MoveNext()
         at Microsoft.AspNetCore.Routing.CompositeEndpointDataSource.CreateEndpointsUnsynchronized()
         at Microsoft.AspNetCore.Routing.CompositeEndpointDataSource.EnsureEndpointsInitialized()
         at Microsoft.AspNetCore.Routing.CompositeEndpointDataSource.get_Endpoints()
         at Microsoft.AspNetCore.Routing.DataSourceDependentCache`1.Initialize()
         at System.Threading.LazyInitializer.EnsureInitializedCore[T](T& target, Boolean& initialized, Object& syncLock, Func`1 valueFactory)
         at Microsoft.AspNetCore.Authorization.Policy.AuthorizationPolicyCache..ctor(EndpointDataSource dataSource)

In the same host log, on another thread:

warn: Wolverine.Runtime.WolverineRuntime[0]
         at Wolverine.Http.Diagnostics.OpenApiDescriptorBuilder.TryBuildForWolverine(HttpChain chain, IApiDescriptionGroupCollectionProvider apiDescriptions)
         ... Wolverine.CritterWatch.Messages.Outbound.HealthCheckRegistrationsDescriptor.ReadFrom(IWolverineRuntime runtime)
         ... Wolverine.CritterWatch.CritterWatchObserver.processBatch(Object[] messages, CancellationToken token)

The caller here is CritterWatch's capabilities snapshot (AddCritterWatchMonitoring), which runs as soon as the runtime starts — before the web host's pipeline has finished adding endpoints. Any other early ApiExplorer reader would do the same.

Versions

WolverineFx 6.39.0, Wolverine.CritterWatch 1.1.0-beta.3, .NET 10, Marten 9.37.0. The monitored service is a modular monolith with MapWolverineEndpoints, several hundred endpoints, and ASP.NET authorization.

Why this is a Wolverine issue, not only a CritterWatch one

OpenApiDescriptorBuilder reads the DI-level EndpointDataSource through ApiExplorer, rather than Wolverine's own HttpGraph, which is complete at MapWolverineEndpoints time. That is the same root cause as the empty-OpenAPI-document problem we see intermittently: an ApiExplorer read taken before the pipeline is built caches an empty ApiDescriptionGroupCollectionProvider for the host's lifetime (the provider caches its first read and never invalidates). One symptom is an empty /openapi/*.json; the other, when the read lands while endpoints are still being added, is a failed startup.

To Reproduce

1. A Wolverine.Http host with many endpoints and ASP.NET authorization (AddAuthorization).
2. Register a hosted service or observer that reads HTTP diagnostics at startup — AddCritterWatchMonitoring(...) reproduces it, as its observer builds a capabilities snapshot on start.
3. Start the host repeatedly under load (a CI runner reproduces it far more often than a developer machine).

It is timing-dependent: for us it is rare (two failures in the last forty CI runs), and it grew more likely as the host gained endpoints and a boot-time hosted service that publishes messages.

Expected behavior

Reading HTTP diagnostics should never be able to abort host startup, and should describe the endpoints Wolverine knows about. Either:

• build descriptors from Wolverine's own HttpGraph rather than the DI EndpointDataSource/ApiExplorer, or
• defer the read until after the application has started (IHostApplicationLifetime.ApplicationStarted), and/or take a snapshot rather than enumerating the live source.

Workaround

Disable CritterWatch monitoring wherever the read is not needed (CritterWatch__Enabled=false) — CI jobs, CLI commands (codegen, resources setup, wolverine-diagnostics). Deployed hosts that do not enable monitoring are unaffected.