Add Failsafe

[ian4hu]

There is another Failure handling and resilience patterns for the JVM - Failsafe, I would like to help to make a implementation based on it.

[kubamarchwicki]

An open - design - question. Failsafe supports both retry policies and circuit breakers. While both RetryPolicy and CircuitBreaker are available, they are wrapper by an executor (Failsafe) which handles circuit breaker's state changes (among others). While this makes a super easy and natural implementation for synchronous and blocking circuit breakers (https://github.com/spring-cloud-incubator/spring-cloud-circuitbreaker/pull/33/files#diff-cc0f345312855800a6928f93a1d78de4R60), it makes reactive implementation a bit more cumbersome. Failsafe doesn't come with reactive stream support (and it doesn't look like from the roadmap to have the support in the near future failsafe-lib/failsafe#159). Failsafe 2.0 (Jan 2019) comes with Java8+ support, so it's clear the Failsafe goal is to work with a broad Java ecosystem, not the bleeding edge latest releases (this is a personal opinion, assumption).

The reactive challenge is related to an overlap between reactive operators (Mono and Flux) and the Failsafe executor itself. I can see multiple approaches and would be keen on understanding which one is better.

1. Do the tiniest possible glue layer between reactive operators and Failsafe (https://github.com/kubamarchwicki/spring-cloud-circuitbreaker/blob/e8dc218f155ddb3b2f0358c00ca54c61df356399/spring-cloud-circuitbreaker-failsafe/src/main/java/org/springframework/cloud/circuitbreaker/failsafe/ReactiveFailsafeCircuitBreaker.java#L59-L61) and delegate dirty work to failsafe asynchronous processing.

• pros: less work, Failsafe does the job, no surprises for existing Failsafe users (fulfilled failsafe users' expectations)
• cons: potential lost the backpressure mechanism (poor mans' reactive), not efficient testing (no notion of virtual time) - (not fulfilled reactive operators users' expectations)

2. Rewrite the Failsafe operations as MonoOperation and FluxOperation. Do not use Failsafe but only utilize Policy classes as a configuration mechanism and to keep the state, but perform all operations as reactive (repeat(), onErrorResume()).

• pros: fully reactive approach
• cons: effectively a rewrite of the FailsafeExecutor mechanism on the top of reactive operators. Would require regular catchup with an upstream library and can introduce some compatibility bugs).

3. Keep only synchronous implementation until Failsafe natively supports reactive streams (or through a thrids party library) as spring-cloud-circuit-breakers should be home to any reactive operators (in a way spring-cloud-sleuth use brave in favour of the home-grown implementation of zipkin client).

The spring-cloud-circuit-breaker - as it defines itself - doesn't point into any directions.

Spring Cloud Circuit breaker provides an abstraction across different circuit breaker implementations. It provides a consistent API to use in your applications allowing you the developer to choose the circuit breaker implementation that best fits your needs for your app.

Guidance would be appreciated.

[ryanjbaxter]

@kubamarchwicki I think option 3 is the approach to take. There is no reactive implementation for Spring Retry either so we just implemented the non-reactive apis.

[kubamarchwicki]

Fair enough. So I think we are good to go here, except for some documentation (I'll add it as well). I've started baking reactive implementation but in another branch

[wind57]

this is a lovely idea! we have been using failsafe for a lot of time and it has been great. I will try to get on par with the existing code base and may be implement this one.