Resiliency in .NET content (#37270)

* Initial bits

* Trying to explore the resilience libs

* Updates to resiliency

* More updates

* Updates to docs

* More fixes and updates

* Fix heading

* More HTTP bits

* Apply suggestions from code review

Co-authored-by: Martin Taillefer <geeknoid@users.noreply.github.com>

* More tweaks

* Added image

* Additions

* More updates

* Apply suggestions from code review

Co-authored-by: Martin Costello <martin@martincostello.com>
Co-authored-by: martintmk <103487740+martintmk@users.noreply.github.com>

* Clean up and updates

* Add context

* Retitle a bit

* A quick fix

* Resilience in TOC

* Updates to headings

* Fix TOC entry

* Added a bit more detail

* Update diagram

* Updates to HTTP resiliency content

* Add more configuration bits

* From peer feedback

* Clean up

* Apply suggestions from code review

Co-authored-by: Martin Costello <martin@martincostello.com>

* Update image

* Apply suggestions from code review

Co-authored-by: martintmk <103487740+martintmk@users.noreply.github.com>

* Updates from peer review

* Major updates to the resilience bits

* A bit closer

* The id's should work

* There we go

* code bits for resilience

* Update source with markers

* Apply suggestions from code review

Co-authored-by: martintmk <103487740+martintmk@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Genevieve Warren <24882762+gewarren@users.noreply.github.com>
Co-authored-by: martintmk <103487740+martintmk@users.noreply.github.com>
Co-authored-by: Martin Costello <martin@martincostello.com>

* Address all feedback

* More clean up, final touches

---------

Co-authored-by: Martin Taillefer <geeknoid@users.noreply.github.com>
Co-authored-by: Martin Costello <martin@martincostello.com>
Co-authored-by: martintmk <103487740+martintmk@users.noreply.github.com>
Co-authored-by: Genevieve Warren <24882762+gewarren@users.noreply.github.com>

Author: 5 people

docs/core/resilience/http-resilience.md

@@ -0,0 +1,116 @@
+---
+title: Introduction to resilient app development
+description: Learn about resiliency as it relates to .NET and how to build a resilience pipeline.
+author: IEvangelist
+ms.author: dapine
+ms.date: 10/20/2023
+---
+
+# Introduction to resilient app development
+
+Resiliency is the ability of an app to recover from failures and continue to function. In the context of .NET programming, resilience is achieved by designing apps that can handle failures gracefully and recover quickly. To help build resilient apps in .NET, the following two packages are available on NuGet:
+
+| NuGet package | Description |
+|--|--|
+| [📦 Microsoft.Extensions.Resilience](https://www.nuget.org/packages/Microsoft.Extensions.Resilience) | This NuGet package provides mechanisms to harden apps against transient failures. |
+| [📦 Microsoft.Extensions.Http.Resilience](https://www.nuget.org/packages/Microsoft.Extensions.Http.Resilience) | This NuGet package provides resilience mechanisms specifically for the <xref:System.Net.Http.HttpClient> class. |
+
+These two NuGet packages are built on top of [Polly](https://github.com/App-vNext/Polly), which is a popular open-source project. Polly is a .NET resilience and transient fault-handling library that allows developers to express strategies such as retry, circuit breaker, timeout, bulkhead isolation, rate-limiting, fallback, and hedging in a fluent and thread-safe manner.
+
+> [!IMPORTANT]
+> The [Microsoft.Extensions.Http.Polly](https://www.nuget.org/packages/Microsoft.Extensions.Http.Polly) NuGet package is deprecated. Use either of the aforementioned packages instead.
+
+## Get started
+
+To get started with resilience in .NET, install the [Microsoft.Extensions.Resilience](https://www.nuget.org/packages/Microsoft.Extensions.Resilience) NuGet package.
+
+### [.NET CLI](#tab/dotnet-cli)
+
+```dotnetcli
+dotnet add package Microsoft.Extensions.Resilience --version 8.0.0
+```
+
+### [PackageReference](#tab/package-reference)
+
+```xml
+<PackageReference Include="Microsoft.Extensions.Resilience" Version="8.0.0" />
+```
+
+---
+
+For more information, see [dotnet add package](../tools/dotnet-add-package.md) or [Manage package dependencies in .NET applications](../tools/dependencies.md).
+
+## Build a resilience pipeline
+
+To use resilience, you must first build a pipeline of resilience-based strategies. Each configured strategy executes in order of configuration. In other words, order is important. The entry point is an extension method on the <xref:Microsoft.Extensions.DependencyInjection.IServiceCollection> type, named `AddResiliencePipeline`. This method takes an identifier of the pipeline and delegate that configures the pipeline. The delegate is passed an instance of `ResiliencePipelineBuilder`, which is used to add resilience strategies to the pipeline.
+
+Consider the following string-based `key` example:
+
+:::code language="csharp" source="snippets/resilience/Program.cs" id="setup":::
+
+The preceding code:
+
+- Creates a new `ServiceCollection` instance.
+- Defines a `key` to identify the pipeline.
+- Adds a resilience pipeline to the `ServiceCollection` instance.
+- Configures the pipeline with a retry strategy, circuit breaker strategy, and timeout strategy.
+
+Each pipeline is configured for a given `key`, and each `key` is used to identify its corresponding `ResiliencePipeline` when getting the pipeline from the provider. The `key` is a generic type parameter of the `AddResiliencePipeline` method.
+
+### Resilience pipeline builder extensions
+
+To add a strategy to the pipeline, call any of the available `Add*` extension methods on the `ResiliencePipelineBuilder` instance.
+
+- `AddRetry`: Try again if something fails, which is useful when the problem is temporary and might go away.
+- `AddCircuitBreaker`: Stop trying if something is broken or busy, which benefits you by avoiding wasted time and making things worse.
+- `AddTimeout`: Give up if something takes too long, which can improve performance by freeing up resources.
+- `AddRateLimiter`: Limit how many requests you make or accept, which enables you to control load.
+- `AddFallback`: Do something else when experiencing failures, which improves user experience.
+- `AddHedging`: Do more than one thing at the same time and take the fastest one, which can improve responsiveness.
+
+For more information, see [Resilience strategies](https://www.pollydocs.org/strategies).
+
+## Add enrichment
+
+Enrichment is the automatic augmentation of telemetry with well-known state, in the form of name/value pairs. For example, an app might emit a log that includes the _operation_ and _result code_ as columns to represent the outcome of some operation. In this situation and depending on peripheral context, enrichment adds _Cluster name_, _Process name_, _Region_, _Tenant ID_, and more to the log as it's sent to the telemetry backend. When enrichment is added, the app code doesn't need to do anything extra to benefit from enriched metrics.
+
+Imagine 1,000 globally distributed service instances generating logs and metrics. When you encounter an issue on your service dashboard, it's crucial to quickly identify the problematic region or data center. Enrichment ensures that metric records contain the necessary information to pinpoint failures in distributed systems. Without enrichment, the burden falls on the app code to internally manage this state, integrate it into the logging process, and manually transmit it. Enrichment simplifies this process, seamlessly handling it without affecting the app's logic.
+
+### Add resilience enrichment
+
+In addition to registering a resilience pipeline, you can also register resilience enrichment. To add enrichment, call the `AddResilienceEnricher` extensions method on the `IServiceCollection` instance.
+
+:::code language="csharp" source="snippets/resilience/Program.cs" id="enricher":::
+
+By calling the `AddResilienceEnricher` extension method, you're adding dimensions on top of the default ones that are built into the underlying Polly library. The following enrichment dimensions are added:
+
+- Exception enrichment based on the <xref:Microsoft.Extensions.Diagnostics.ExceptionSummarization.IExceptionSummarizer>, which provides a mechanism to summarize exceptions for use in telemetry.
+- Result enrichment based on the <xref:Microsoft.Extensions.Resilience.FailureResultContext>, which captures the dimensions metered for transient fault failures.
+- Request metadata enrichment based on <xref:Microsoft.Extensions.Http.Telemetry.RequestMetadata>, which holds the request metadata for telemetry.
+
+For more information, see [Polly: Telemetry metrics](https://www.pollydocs.org/advanced/telemetry#metrics).
+
+## Use resilience pipeline
+
+To use a configured resilience pipeline, you must get the pipeline from a `ResiliencePipelineProvider<TKey>`. When you added the pipeline earlier, the `key` was of type `string`, so you must get the pipeline from the `ResiliencePipelineProvider<string>`.
+
+:::code language="csharp" source="snippets/resilience/Program.cs" id="pipeline":::
+
+The preceding code:
+
+- Builds a service provider from the `ServiceCollection` instance.
+- Gets the `ResiliencePipelineProvider<string>` from the service provider.
+- Retrieves the `ResiliencePipeline` from the `ResiliencePipelineProvider<string>`.
+
+## Execute resilience pipeline
+
+To use the resilience pipeline, call any of the available `Execute*` methods on the `ResiliencePipeline` instance. For example, consider an example call to `ExecuteAsync` method:
+
+:::code language="csharp" source="snippets/resilience/Program.cs" id="execute":::
+
+The preceding code executes the delegate within the `ExecuteAsync` method. When there are failures, the configured strategies are executed. For example, if the `RetryStrategy` is configured to retry three times, the delegate is executed three times before the failure is propagated.
+
+## Next steps
+
+> [!div class="nextstepaction"]
+> [Build resilient HTTP apps: Key development patterns](http-resilience.md)

docs/core/resilience/index.md

@@ -0,0 +1,4 @@
+namespace Http.Resilience.Example;
+
+public record class Comment(
+    int PostId, int Id, string Name, string Email, string Body);

docs/core/resilience/media/http-get-comments-flow.png

@@ -0,0 +1,18 @@
+using System.Net.Http.Json;
+
+namespace Http.Resilience.Example;
+
+/// <summary>
+/// An example client service, that relies on the <see cref="HttpClient"/> instance.
+/// </summary>
+/// <param name="client">The given <see cref="HttpClient"/> instance.</param>
+internal sealed class ExampleClient(HttpClient client)
+{
+    /// <summary>
+    /// Returns an <see cref="IAsyncEnumerable{T}"/> of <see cref="Comment"/>s.
+    /// </summary>
+    public IAsyncEnumerable<Comment?> GetCommentsAsync()
+    {
+        return client.GetFromJsonAsAsyncEnumerable<Comment>("/comments");
+    }
+}

docs/core/resilience/snippets/http-resilience/Comment.cs

@@ -0,0 +1,68 @@
+using System.Net;
+using Microsoft.Extensions.DependencyInjection;
+using Microsoft.Extensions.Http.Resilience;
+using Polly;
+
+internal partial class Program
+{
+    private static void WithCustomHandler(IHttpClientBuilder httpClientBuilder)
+    {
+        // <custom>
+        httpClientBuilder.AddResilienceHandler(
+            "CustomPipeline",
+            static builder =>
+        {
+            // See: https://www.pollydocs.org/strategies/retry.html
+            builder.AddRetry(new HttpRetryStrategyOptions
+            {
+                // Customize and configure the retry logic.
+                BackoffType = DelayBackoffType.Exponential,
+                MaxRetryAttempts = 5,
+                UseJitter = true
+            });
+
+            // See: https://www.pollydocs.org/strategies/circuit-breaker.html
+            builder.AddCircuitBreaker(new HttpCircuitBreakerStrategyOptions
+            {
+                // Customize and configure the circuit breaker logic.
+                SamplingDuration = TimeSpan.FromSeconds(10),
+                FailureRatio = 0.2,
+                MinimumThroughput = 3,
+                ShouldHandle = static args =>
+                {
+                    return ValueTask.FromResult(args is
+                    {
+                        Outcome.Result.StatusCode:
+                            HttpStatusCode.RequestTimeout or
+                                HttpStatusCode.TooManyRequests
+                    });
+                }
+            });
+
+            // See: https://www.pollydocs.org/strategies/timeout.html
+            builder.AddTimeout(TimeSpan.FromSeconds(5));
+        });
+        // </custom>
+    }
+
+    private static void WithAdvancedCustomHandler(IHttpClientBuilder httpClientBuilder)
+    {
+        // <advanced>
+        httpClientBuilder.AddResilienceHandler(
+            "AdvancedPipeline",
+            static (ResiliencePipelineBuilder<HttpResponseMessage> builder,
+                ResilienceHandlerContext context) =>
+            {
+                // Enable reloads whenever the named options change
+                context.EnableReloads<HttpRetryStrategyOptions>("RetryOptions");
+
+                // Retrieve the named options
+                var retryOptions =
+                    context.GetOptions<HttpRetryStrategyOptions>("RetryOptions");
+
+                // Add retries using the resolved options
+                builder.AddRetry(retryOptions);
+            });
+        // </advanced>
+    }
+}

docs/core/resilience/snippets/http-resilience/ExampleClient.cs

@@ -0,0 +1,58 @@
+using Microsoft.Extensions.DependencyInjection;
+using Microsoft.Extensions.Http.Resilience;
+
+internal partial class Program
+{
+    private static void WithStandardHedgingHandler(IHttpClientBuilder httpClientBuilder)
+    {
+        // <standard>
+        httpClientBuilder.AddStandardHedgingHandler();
+        // </standard>
+    }
+
+    private static void WithConfiguredStandardHedgingHandler(IHttpClientBuilder httpClientBuilder)
+    {
+        // <ordered>
+        httpClientBuilder.AddStandardHedgingHandler(static (IRoutingStrategyBuilder builder) =>
+        {
+            // Hedging allows sending multiple concurrent requests
+            builder.ConfigureOrderedGroups(static options =>
+            {
+                options.Groups.Add(new UriEndpointGroup()
+                {
+                    Endpoints =
+                    {
+                        // Imagine a scenario where 3% of the requests are 
+                        // sent to the experimental endpoint.
+                        new() { Uri = new("https://example.net/api/experimental"), Weight = 3 },
+                        new() { Uri = new("https://example.net/api/stable"), Weight = 97 }
+                    }
+                });
+            });
+        });
+        // </ordered>
+
+        // <weighted>
+        httpClientBuilder.AddStandardHedgingHandler(static (IRoutingStrategyBuilder builder) =>
+        {
+            // Hedging allows sending multiple concurrent requests
+            builder.ConfigureWeightedGroups(static options =>
+            {
+                options.SelectionMode = WeightedGroupSelectionMode.EveryAttempt;
+
+                options.Groups.Add(new WeightedUriEndpointGroup()
+                {
+                    Endpoints =
+                    {
+                        // Imagine A/B testing
+                        new() { Uri = new("https://example.net/api/a"), Weight = 33 },
+                        new() { Uri = new("https://example.net/api/b"), Weight = 33 },
+                        new() { Uri = new("https://example.net/api/c"), Weight = 33 }
+                    }
+                });
+            });
+        });
+        // </weighted>
+    }
+}
+

docs/core/resilience/snippets/http-resilience/Program.CustomHandler.cs

@@ -0,0 +1,22 @@
+using Microsoft.Extensions.DependencyInjection;
+using Polly;
+
+internal partial class Program
+{
+    private static void WithStandardHandler(IHttpClientBuilder httpClientBuilder)
+    {
+        // <standard>
+        httpClientBuilder.AddStandardResilienceHandler();
+        // </standard>
+    }
+
+    private static void WithConfiguredStandardHandler(IHttpClientBuilder httpClientBuilder)
+    {
+        // <configure>
+        httpClientBuilder.AddStandardResilienceHandler(static options =>
+        {
+            options.Retry.BackoffType = DelayBackoffType.Linear;
+        });
+        // </configure>
+    }
+}

docs/core/resilience/snippets/http-resilience/Program.HedgingHandler.cs

@@ -0,0 +1,16 @@
+using Microsoft.Extensions.Configuration;
+using Microsoft.Extensions.DependencyInjection;
+using Microsoft.Extensions.Hosting;
+using Microsoft.Extensions.Http.Resilience;
+
+internal partial class Program
+{
+    private static void ConfigureRetryOptions(HostApplicationBuilder builder)
+    {
+        // <options>
+        var section = builder.Configuration.GetSection("RetryOptions");
+
+        builder.Services.Configure<HttpRetryStrategyOptions>(section);
+        // </options>
+    }
+}

docs/core/resilience/snippets/http-resilience/Program.ResilienceHandler.cs

@@ -0,0 +1,19 @@
+using Microsoft.Extensions.DependencyInjection;
+
+namespace Http.Resilience.Example;
+
+internal partial class Program
+{
+    private static void CreateServiceCollection()
+    {
+        // <services>
+        var services = new ServiceCollection();
+
+        var httpClientBuilder = services.AddHttpClient<ExampleClient>(
+            configureClient: static client =>
+            {
+                client.BaseAddress = new("https://jsonplaceholder.typicode.com");
+            });
+        // </services>
+    }
+}