- Adaptive rate limiting overview
- Features
- Background
- Thinking
- Architecture
- Functionality
- Installation and Usage
- Example
- E2E testing
- Easy to use, just submit
SmartLimiter
resources to achieve the purpose of service rate limiting. - Adaptive rate limiting, which dynamically triggers rate limiting based on the service's resource content and metrics.
On the one hand, with the removal of Mixer
, users have to face the exceptionally complicated EnvoyFilter
configuration, on the other hand, the fixed rate limiting policy is not flexible enough, and the ordinary rate limiting cannot cover the scenario that the service CPU
usage reaches the threshold to open the rate limiting. In order to solve these two pain points, we introduced the adaptive rate limiting component slime/limiter
. Users can simply submit a SmartLimiter
that meets our definition to complete flexible service rate limiting requirements.
The primary goal of adaptive rate limiting is to free users from the tedious EnvoyFilter
configuration, so we take advantage of kubernetes
CRD
mechanism and we define an easy API
, the SmartLimiter
resource within kubernetes
. Users only need to submit a CR
according to the SmartLimiter
specification.
apiVersion: microservice.slime.io/v1alpha1
kind: SmartLimiter
metadata:
name: productpage
namespace: default
spec:
sets:
v1:
descriptor:
- action:
fill_interval:
seconds: 1
quota: "10"
condition: "{{.v1.cpu.sum}}>10"
Another goal of adaptive rate limiting is to make the rate limiting policy flexible enough, for example, to turn on rate limiting when the CPU
usage of a service reaches a threshold, for this we introduce the prometheus
component, which uses PromQL
to query the relevant metric
and produce an envoyfilter
when the metric reaches the threshold.
The main architecture of adaptive rate limiting is divided into two parts, one part includes the logical transformation from SmartLimiter
to EnvoyFilter
, and the other part includes the acquisition of monitoring data within the cluster, including the number of CPU
, Memory
, POD
and other metric data of the service.
The Adaptive rate limiting module can limit the rate of a service or a group under a service, see Adaptive Limiter based on monitoring, subset rate limiting, service_limiters
smartlimiter
depends on the prometheus
component and some necessary CRD
declarations, see Installation and Usage
Enable adaptive limiting for bookinfo
's reviews
service, see example
Module functionality can be verified by the E2E module when the functionality is developed