Skip to content

[Bug] Sync-over-async in IssuerValidator causes thread starvation and container restart under high throughput #3884

Description

@marcelomuszalskimews

Microsoft.Identity.Web Library

Microsoft.Identity.Web

Microsoft.Identity.Web version

4.10.0

Web app

Not Applicable

Web API

Protected web APIs (validating tokens)

Token cache serialization

Not Applicable

Description

Under high throughput (≥600 RPS), new Azure Container App replicas fail to start and continuously restart. The root cause is a sync-over-async anti-pattern in the issuer validator code path inside Microsoft.Identity.Web, which causes thread pool starvation during startup, preventing the OIDC metadata endpoint from being reached within the configured timeout.

This issue is not visible at normal load — it only manifests when the container app scales out and new replicas start while the service is under pressure.

Reproduction steps

  1. Deploy an ASP.NET Core Web API to Azure Container Apps using AddMicrosoftIdentityWebApi(configuration) with default configuration (no explicit ValidIssuers set).
  2. Configure the Container App with: min replicas = 1–3, max replicas = 6, HTTP scale rule = 50 concurrent requests.
  3. Apply sustained load ≥ 600 RPS against the service.
  4. Observe the replicas blade — newly started replicas will fail on startup with increasing restart counts.

Error message

IDX20804: Unable to retrieve document from: 'https://login.microsoftonline.com/{tenantId}/.well-known/openid-configuration'. InnerException: System.Threading.Tasks.TaskCanceledException.

IDX20806: Unable to obtain an updated configuration from: 'https://login.microsoftonline.com/{tenantId}/.well-known/openid-configuration'. Returning the current configuration.
Exception: System.IO.IOException
---> System.Threading.Tasks.TaskCanceledException: The request was canceled due to the configured HttpClient.Timeout of 100 seconds elapsing.
---> System.TimeoutException: A task was canceled.

Id Web logs

Not available — the issue manifests as thread pool starvation; the OIDC endpoint was reachable via curl from within the failing replica at all times, confirming the problem is application-level and not a network failure.

Relevant code snippets

// Problematic default setup — omitting ValidIssuers triggers the sync-over-async path
services
    .AddAuthentication()
    .AddMicrosoftIdentityWebApi(configuration);

// Workaround — pre-populating ValidIssuers short-circuits the issuer validator
// before it reaches the async HTTP call (await GetBaseConfigurationAsync)
services.Configure<JwtBearerOptions>(JwtBearerDefaults.AuthenticationScheme, options =>
{
    options.BackchannelTimeout = TimeSpan.FromSeconds(5);
    options.TokenValidationParameters.ValidIssuers =
    [
        $"https://login.microsoftonline.com/{tenantId}/v2.0",
        $"https://sts.windows.net/{tenantId}/",
    ];
});

Regression

Unknown — this appears to be a structural issue rather than a regression from a specific version.

Expected behavior

The issuer validator should not block thread pool threads. The async code path that fetches OIDC configuration (GetBaseConfigurationAsync) should be awaitable end-to-end. At minimum, when ValidIssuers is not pre-configured, the library should not introduce a sync-over-async pattern that causes thread starvation under high load.

Metadata

Metadata

Labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions