-
Notifications
You must be signed in to change notification settings - Fork 850
Description
Description
In the latest version of Microsoft.Extensions.Http.Resilience, HttpClientResiliencePredicates.IsHttpConnectionTimeout assume that a OperationCanceledException which Source is System.Private.CoreLib and InnerException is TimeoutException is a connection timeout exception.
Lines 53 to 55 in 9e8d935
| internal static bool IsHttpConnectionTimeout(in Outcome<HttpResponseMessage> outcome, in CancellationToken cancellationToken) | |
| => !cancellationToken.IsCancellationRequested | |
| && outcome.Exception is OperationCanceledException { Source: "System.Private.CoreLib", InnerException: TimeoutException }; |
It usually works well, but after aot publish. The Exception.Source will always return <unknown> (If metadata for current method cannot be established).
public virtual string? Source
{
[UnconditionalSuppressMessage("ReflectionAnalysis", "IL2026:RequiresUnreferencedCode",
Justification = "The API will return <unknown> if the metadata for current method cannot be established.")]
get => _source ??= HasBeenThrown ? (TargetSite?.Module.Assembly.GetName().Name ?? "<unknown>") : null;
set => _source = value;
}Which case any resilience pipeline using HttpRetryStrategyOptions with default ShouldHandle will never retry if connection timeout error occured after aot publish.
Reproduction Steps
Using follow program:
#!/usr/local/share/dotnet/dotnet run
#:package Microsoft.Extensions.Http.Resilience@10.1.0
#:package Microsoft.Extensions.Logging.Console@10.0.0
using Microsoft.Extensions.Http.Resilience;
using Microsoft.Extensions.Logging;
using Polly;
using ILoggerFactory factory = LoggerFactory.Create(builder => builder.AddConsole());
ILogger logger = factory.CreateLogger("Program");
var socketHttpHandler = new SocketsHttpHandler
{
ConnectTimeout = TimeSpan.FromMilliseconds(1) // to trigger connect timeout
};
var retryPipeline = new ResiliencePipelineBuilder<HttpResponseMessage>()
.AddRetry(new HttpRetryStrategyOptions
{
MaxRetryAttempts = 1,
// just fail fast
Delay = TimeSpan.FromMilliseconds(1),
BackoffType = DelayBackoffType.Linear
})
.ConfigureTelemetry(factory)
.Build();
var client = new HttpClient(new ResilienceHandler(retryPipeline)
{
InnerHandler = socketHttpHandler
});
try
{
await client.GetAsync("https://example.com");
}
catch (Exception ex)
{
logger.LogInformation("Exception.Source: {ExceptionSource}", ex.Source); // Should be `System.Private.CoreLib`, but `<unknown>` after aot publish
}Just dotnet run
dotnet run .\retry-fail-aot.cs
warn: Polly[3]
Execution attempt. Source: '(null)/(null)/Retry', Operation Key: '', Result: 'The operation was canceled.', Handled: 'True', Attempt: '0', Execution Time: 30.3317ms
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.TimeoutException: A connection could not be established within the configured ConnectTimeout.
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.CreateConnectTimeoutException(OperationCanceledException oce)
......
at Microsoft.Extensions.Http.Resilience.ResilienceHandler.<>c.<<SendAsync>b__3_0>d.MoveNext()
warn: Polly[0]
Resilience event occurred. EventName: 'OnRetry', Source: '(null)/(null)/Retry', Operation Key: '', Result: 'The operation was canceled.'
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.TimeoutException: A connection could not be established within the configured ConnectTimeout.
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.CreateConnectTimeoutException(OperationCanceledException oce)
......
at Microsoft.Extensions.Http.Resilience.ResilienceHandler.<>c.<<SendAsync>b__3_0>d.MoveNext()
fail: Polly[3]
Execution attempt. Source: '(null)/(null)/Retry', Operation Key: '', Result: 'The operation was canceled.', Handled: 'True', Attempt: '1', Execution Time: 17.4763ms
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.TimeoutException: A connection could not be established within the configured ConnectTimeout.
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.CreateConnectTimeoutException(OperationCanceledException oce)
......
at Microsoft.Extensions.Http.Resilience.ResilienceHandler.<>c.<<SendAsync>b__3_0>d.MoveNext()
info: Program[0]
Exception.Source: System.Private.CoreLib
After aot publish
.\retry-fail-aot.exe
info: Polly[3]
Execution attempt. Source: '(null)/(null)/Retry', Operation Key: '', Result: 'The operation was canceled.', Handled: 'False', Attempt: '0', Execution Time: 21.8876ms
System.Threading.Tasks.TaskCanceledException: The operation was canceled.
---> System.TimeoutException: A connection could not be established within the configured ConnectTimeout.
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.CreateConnectTimeoutException(OperationCanceledException) + 0xb3
......
at Microsoft.Extensions.Http.Resilience.ResilienceHandler.<>c.<<SendAsync>b__3_0>d.MoveNext() + 0x91
info: Program[0]
Exception.Source: <unknown>
Resilience event occurred. EventName: 'OnRetry' was missing from program after aot publish. And Exception.Source is <unknown>.
Expected behavior
HttpClientResiliencePredicates.IsHttpConnectionTimeout should return true if exception is connect timeout exception after aot publish.
Actual behavior
HttpClientResiliencePredicates.IsHttpConnectionTimeout always return false after aot publish.
Regression?
10.0.0 have same issue.
Known Workarounds
Write a own HttpClientResiliencePredicates and replace IsHttpConnectionTimeout() to following version:
internal static bool IsHttpConnectionTimeout(in Outcome<HttpResponseMessage> outcome, in CancellationToken cancellationToken)
=> !cancellationToken.IsCancellationRequested
&& outcome.Exception is OperationCanceledException { InnerException: TimeoutException };new HttpRetryStrategyOptions()
{
ShouldHandle = args => new ValueTask<bool>(CustomHttpClientResiliencePredicates.IsTransient(args.Outcome, args.Context.CancellationToken))
}Configuration
.NET SDK:
Version: 10.0.101
Commit: fad253f51b
Workload version: 10.0.100-manifests.c57ac48b
MSBuild version: 18.0.6+fad253f51
OS Name: Windows
OS Version: 10.0.26100
OS Platform: Windows
RID: win-x64
Base Path: C:\Program Files\dotnet\sdk\10.0.101\
No .net workloads was installed.
Host:
Version: 10.0.1
Architecture: x64
Commit: fad253f51b
.NET SDKs installed:
9.0.305 [C:\Program Files\dotnet\sdk]
10.0.101 [C:\Program Files\dotnet\sdk]
.NET runtimes installed:
Microsoft.AspNetCore.App 8.0.20 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 9.0.9 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 10.0.1 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.36 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 7.0.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.20 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 8.0.22 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 9.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 10.0.1 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.WindowsDesktop.App 8.0.20 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 9.0.9 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Microsoft.WindowsDesktop.App 10.0.1 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
Other architectures found:
x86 [C:\Program Files (x86)\dotnet]
registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation]
Environment variables:
Not set
global.json file:
Not found
Other information
No response