Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
---
title: Implementing custom HTTP call retries with exponential backoff
description: .NET Microservices Architecture for Containerized .NET Applications | Implementing custom HTTP call retries with exponential backoff
title: Explore custom HTTP call retries with exponential backoff
description: Learn how you could implement, from scratch, HTTP call retries with exponential backoff to handle possible HTTP failure scenarios.
author: CESARDELATORRE
ms.author: wiwagn
ms.date: 05/26/2017
ms.date: 06/08/2018
---
# Implementing custom HTTP call retries with exponential backoff
# Explore custom HTTP call retries with exponential backoff

In order to create resilient microservices, you need to handle possible HTTP failure scenarios. For that purpose, you could create your own implementation of retries with exponential backoff.
To create resilient microservices, you need to handle possible HTTP failure scenarios. One way of handling those failures, although not recommended, is to create your own implementation of retries with exponential backoff.

In addition to handling temporal resource unavailability, the exponential backoff also needs to take into account that the cloud provider might throttle availability of resources to prevent usage overload. For example, creating too many connection requests very quickly might be viewed as a Denial of Service ([DoS](https://en.wikipedia.org/wiki/Denial-of-service_attack)) attack by the cloud provider. As a result, you need to provide a mechanism to scale back connection requests when a capacity threshold has been encountered.
**Important note:** This section shows you how you could create your own custom code to implement HTTP call retries. However, it is not recommended to do it by your own but to use more powerful and reliable while simpler to use mechanisms, such as `HttpClientFactory` with Polly, available since .NET Core 2.1. Those recommended approaches are explained in the next sections.

As an initial exploration, you could implement your own code with a utility class for exponential backoff as in [RetryWithExponentialBackoff.cs](https://gist.github.com/CESARDELATORRE/6d7f647b29e55fdc219ee1fd2babb260), plus code like the following (which is also available on a [GitHub repo](https://gist.github.com/CESARDELATORRE/d80c6423a1aebaffaf387469f5194f5b)).
As an initial exploration, you could implement your own code with a utility class for exponential backoff as in [RetryWithExponentialBackoff.cs](https://gist.github.com/CESARDELATORRE/6d7f647b29e55fdc219ee1fd2babb260), plus code like the following (which is also available at this [GitHub repo](https://gist.github.com/CESARDELATORRE/d80c6423a1aebaffaf387469f5194f5b)).

```csharp
public sealed class RetryWithExponentialBackoff
Expand Down Expand Up @@ -107,9 +107,11 @@ public async Task<Catalog> GetCatalogItems(int page,int take, int? brand, int? t
}
```

However, this code is suitable only as a proof of concept. The next topic explains how to use more sophisticated and proven libraries.
Remember that this code is suitable only as a proof of concept.
The next sections explain how to use more sophisticated approaches while simpler, by using HttpClientFactory.
HttpClientFactory is available since .NET Core 2.1, with proven resiliency libraries like Polly.


>[!div class="step-by-step"]
[Previous](implement-resilient-entity-framework-core-sql-connections.md)
[Next](implement-http-call-retries-exponential-backoff-polly.md)
[Next](use-httpclientfactory-to-implement-resilient-http-requests.md)
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ title: Handling partial failure
description: .NET Microservices Architecture for Containerized .NET Applications | Handling partial failure
author: CESARDELATORRE
ms.author: wiwagn
ms.date: 05/26/2017
ms.date: 06/08/2018
---
# Handling partial failure

In distributed systems like microservices-based applications, there is an ever-present risk of partial failure. For instance, a single microservice/container can fail or might not be available to respond for a short time, or a single VM or server can crash. Since clients and services are separate processes, a service might not be able to respond in a timely way to a client’s request. The service might be overloaded and responding extremely slowly to requests, or might simply not be accessible for a short time because of network issues.
In distributed systems like microservices-based applications, there is an ever-present risk of partial failure. For instance, a single microservice/container can fail or might not be available to respond for a short time, or a single VM or server can crash. Since clients and services are separate processes, a service might not be able to respond in a timely way to a client’s request. The service might be overloaded and responding extremely slowly to requests or might simply not be accessible for a short time because of network issues.

For example, consider the Order details page from the eShopOnContainers sample application. If the ordering microservice is unresponsive when the user tries to submit an order, a bad implementation of the client process (the MVC web application)—for example, if the client code were to use synchronous RPCs with no timeout—would block threads indefinitely waiting for a response. In addition to creating a bad user experience, every unresponsive wait consumes or blocks a thread, and threads are extremely valuable in highly scalable applications. If there are many blocked threads, eventually the application’s runtime can run out of threads. In that case, the application can become globally unresponsive instead of just partially unresponsive, as show in Figure 10-1.

Expand All @@ -21,15 +21,15 @@ In a large microservices-based application, any partial failure can be amplified

**Figure 10-2**. The impact of having an incorrect design featuring long chains of HTTP requests

Intermittent failure is virtually guaranteed in a distributed and cloud based system, even if every dependency itself has excellent availability. This should be a fact you need to consider.
Intermittent failure is guaranteed in a distributed and cloud-based system, even if every dependency itself has excellent availability. It is fact you need to consider.

If you do not design and implement techniques to ensure fault tolerance, even small downtimes can be amplified. As an example, 50 dependencies each with 99.99% of availability would result in several hours of downtime each month because of this ripple effect. When a microservice dependency fails while handling a high volume of requests, that failure can quickly saturate all available request threads in each service and crash the whole application.

![](./media/image3.png)

**Figure 10-3**. Partial failure amplified by microservices with long chains of synchronous HTTP calls

To minimize this problem, in the section "*Asynchronous microservice integration enforce microservice’s autonomy*” (in the architecture chapter), we encouraged you to use asynchronous communication across the internal microservices. We briefly explain more in the next section.
To minimize this problem, in the section "*Asynchronous microservice integration enforce microservice’s autonomy*” (in the architecture chapter), this guidance encourages you to use asynchronous communication across the internal microservices.

In addition, it is essential that you design your microservices and client applications to handle partial failures—that is, to build resilient microservices and client applications.

Expand Down
Loading