rate limit: configuration/filter enhancements

[mattklein123]

We are considering a number of enhancements/changes to how rate limiting can be configured on a per virtual host / route basis for HTTP requests. This will include a shift in configuration from the filter to the route/vhost, as well as new rules (potentially logical and/or, etc.) for setting descriptors.

@ccaraman when we have a better idea of what we are planning please post a summary here.

cc @chowchow316 @mandarjog let's discuss here if you have any requests since I think this issue will be relevant to what you are working on.

[ccaraman]

Envoy HTTP Rate Limit Configuration Proposal

What we are trying to solve?

With our current configuration, Envoy is limited to simple rate limit rules that can only be configured per route.

Going forward, we want to provide a flexible rate limit configuration to allow for more complex rate limiting scenarios such as rate limiting based on IP tags and IP address prior to authentication by the Auth filters in Front Envoy or allowing a virtual host to have a global rate limit for all traffic and have a more restrictive rate limit for a particular endpoint.

HTTP Rate Limits

In an envoy config, there are two parts to enable HTTP rate limits. The first part is installing the Rate Limit filter in the HTTP Connection Manager. The second part is to configure the rate limit settings within the virtual host.

HTTP Rate Limit Filter

{
  "type": "decoder",
  "name": "rate_limit",
  "config": {
     // domain (required, string)
     // The rate limit domain to use when calling the rate limit service.
     "domain": "my_domain_name",
     // stage (optional, integer)
     // If set, rate limit configurations will only be applied with the same stage
     // number.
     // If not set, all rate limit configurations will be applied. 
     "stage": 1 
  }
}

Specifying Rate Limits Configurations Scope

A rate limit configuration can be specified at the virtual host level and at the per route level. For a request, all virtual host level configurations that match the filter stage setting will be applied. If there are route specific configurations that match the filter stage setting, those will be applied in addition. More than one configuration can apply to a request. Each configuration results in a descriptor being sent to the rate limit service. Rate limits are enacted based on the result of logically ‘or’-ing the response from each descriptor.

Virtual Host

Rate limit configurations to be applied for the virtual host.

{
  "name": "...",
  "domains":[
    "..."
  ],
  "routes" : [],
  ...
  // rate_limits (optional, array)
  // A list of rate limit configurations to be applied for all requests for this virtual
  // hosts. Order of configurations does not matter. 
  "rate_limits": []
}

Per Route

Rate limit configurations to be applied for the route within a virtual host.

{
  "name": "...", 
  "domains": [
    "..."
  ], 
  "routes": [
    {
      ...
      // rate_limits (optional, array)
      // A list of rate limit configurations to be applied for all requests for this route
      // Order of configurations does not matter. 
      "rate_limits": []
      ...
    }
  ]
}

HTTP rate limit configuration

{
  // stage (optional, integer)
  // Refers to the stage set in the filter.
  // If set, the rate limit configuration only applies to filters with the same stage number 
  // and for filters set to default. If not set, the rate limit configuration will apply for 
  // all rate limit filters set to default.
  "stage": ”...”,
  // kill_switch_key (optional, string)
  // The key to be set in runtime to disable this rate limit configuration. 
  "kill_switch_key": "...",
  // action (required, array)
  // A list of actions that are to be applied for this rate limit configurations. Order 
  // matters as the actions are processed sequentially and the descriptors will be composed 
  // in that sequence.
  "actions": []
}

Actions

There are several existing action types:

• Service To Service
• Request Headers
• Remote Address

For more information on these actions, please refer to the documentation.

In addition, we will be adding the following types:

Rate Limit Key Action

This action is to replace the existing ‘route key’ value in the rate limit configuration.

{
  "type" : "rate_limit_key",
  // descriptor_value (required, string)
  // The value to send in the descriptor.
  "descriptor_value": "..."
}

The following descriptor will be sent for this action type

• (“rate_limit_key”, “<descriptor_value>”)

Header Value Matching Action

{
  "type": "header_value_match",
  // headers (required, array)
  // Specifies a list of headers that the request should match on.
  "headers" : []
  // descriptor_value (required, string)
  // The value to use in the descriptor if there is a match. 
  "descriptor_value": "..."
}

The following descriptor will be sent for this action type:

• (“header_match”, “<descriptor_value>”)

Headers

{ 
  // name (required, string)
  // Specifies the name of the header in the request
  "name" : "...", 
  // value (optional, string)
  // Specifies the value of the header. The value to match on can be a regex. If the value 
  // is absent a request that has the name header will match, regardless of the header’s 
  // value.
  "value": "...",
  // regex_matching (optional, boolean)
  // If set to true, regex matching will be used when comparing the value from the header     
  // to 'value'. If set to false, a direct comparison will be made. Defaults to 
  // false.
  "regex_matching": "..."
}

Example 1

{
  "type": "header_value_match",
  "headers": [
    {
      "name": ":path",
      "value": "/v1/random",
    }
  ]
  "descriptor_value": "endpointmatch"
}

The above action type will match for this request:

• ”:path” : “/v1/random”

And will populate the descriptor with this value:

• (“header_match”, “endpointmatch”)

It will not match this request:

• ”:path” : “/v1/random/test”

And will not populate a descriptor.

Example 2

{
  "type": "header_value_match",
  "headers": [
    {
      "name": ":method",
      "value": "POST",
    },
    {
      "name": ":path",
      "value": "\/v1\/.*\/rides",
      "regex_matching": true
    }
  ]
  "descriptor_value": "rides_post_request"
}

The above action type will match these requests:

• ”:path” : “/v1/myuserid/rides”, “:method” : “POST”

• “:path” : “/v1//rides, “:method”: “POST”

And will populate the descriptor with this value:

• (“header_match”, “rides_post_request”)

It will not match these requests:

• “:path”, “/v1/myuserid/rides”, “:method” : “GET”

• “:path” : “/rides”, “:method” : “POST”

And it will not populate a descriptor.

IP Tag Action

IP Tags will be implemented to populate a header value. To support rate limiting based on IP tags, the header value will be pulled from the request and forwarded to the rate limit service. This will be done using the Request Header Action.

Example of Applying Rate Limit Configurations

This example is to demonstrate how a rate limit configuration within the virtual hosts and routes will be applied.

Sample Filter Configuration

{
  "filters": [
    {
      "type": "decoder",
      "name": "rate_limit",
      "config": {
        "domain": "envoy_front",
        "stage": 1
      }
    },
    {
      "type": "decoder",
      "name": "lyft_bearer_token_auth",
      "config": {
        "cluster_name": "auth"
      }
    },
    {
      "type": "decoder", 
      "name": "rate_limit", 
      "config": {
        "domain": "envoy_front"
        "stage" : 2
      }
    }, 
    {
      "type": "decoder", 
      "name": "router", 
      "config": {}
    }
  ]
}

Sample Virtual Host Configuration

{
  "name": "users",
  "domains": [
    "users"
  ],
  “rate_limits” : [
    {
      "stage": 1,
      "actions": [
        {
          "type": "remote_address"
        }
      ]
    }
  ],
  "routes": [
    {
      "prefix": "/bob",
      "rate_limits":[ {
        "stage": 2,
        "actions": [
          {
            "type": "request_headers",
            "header_name": "x-lyft-user-id",
            "descriptor_key": "user_id"
          }
        ]
      }],
      "cluster": "users"
    },
    {
      "prefix": "*",
      "cluster": "users"
    }
  ]
}

Note: Filters are applied in sequential order.
Assume a Users request is being processed.

The first rate limit has a stage value of ‘1’. When this filter is processing, it will only apply the rate limit specified at the virtual host layer since it has the same stage value.

The second rate limit has a stage value of ‘2’. If the prefix matches /bob, the route rate limit configuration will be applied. If not, no rate limit configurations are applicable and no rate limiting will occur.

[mattklein123]

@chowchow316 @mandarjog please look at ^ depending on which implementation option you decide on for rate limiting.

FYI @junr03 and @twoism are working on open sourcing our ratelimit service. Probably 3-4 weeks out.

cc @louiscryan

[chowchow316]

LGTM

[CarsarSky]

Any updates?

[mattklein123]

The only two thing that haven't been implemented in the above doc is a generic header matching rate limit rule as well as IP tagging. The public docs are up to date.

@ccaraman is on vacation this week. She will finish header matching rule next week, then we will close this GH issue out, and track IP tagging as a different issue as that is a larger and more complicated project.

[mattklein123]

complete other than ip tagging so closing this.

[crazyproger]

Is there any possibility to use Network layer rate limit filter with auto filled client IP descriptor? Like remote_address action in http route rate limit configuration.
This is mainly to limit number of connections from same IP.