Envoy Rate Limiting

Overview

In this document, we show how to use Gloo with Envoy’s rate-limit API. We make the distinction here that this is “Envoy’s” rate-limit API because Gloo offers a much simpler rate-limit API as an alternative.

Gloo Enterprise includes a rate limit server based on Lyft’s Envoy rate-limit server. It is already installed when doing glooctl install gateway --license-key=... or using the Helm install. To get your trial license key, go to https://www.solo.io/gloo-trial

Two steps are needed in configuring Gloo to leverage the full Envoy Rate Limiter.

  1. In the Gloo Settings manifest, you need to configure all of your rate limiting descriptors. These are based on the Lyft Rate Limiting service included with Gloo Enterprise.
  2. For each Virtual Service, you need to configure Envoy rate limiting actions at the Virtual Service level or for each route or both.

Rate limiting descriptors define a set (tuple) of keys that must match for the associated rate limit to be applied. The set of keys are expressed as a hierarchy to make configuration easy; its the set of keys matching or not that is important. Each descriptor key can have an associated value that is matched as a literal. If there is no value associated with a key, then each unique value is used for rate limiting, that is if one of the keys is ‘user_id’ with no associated value then Envoy does per user rate limiting with each unique user_id value being used as the id for rate limiting. See the Lyft rate limiting descriptors for full details.

For example, the following descriptor will match against two keys ('account_id', '<unique value>'), ('plan', 'BASIC | PLUS'). Since the account_id key does not specify a descriptor value, it uses each unique value passed into the rate limiting service to match, i.e., per account rate limiting. The plan descriptor key has two values specified and depending on which one matches (BASIC or PLUS) determines the rate limit, either 1 request per minute for BASIC or 20 requests per minute for PLUS. You can specify multiple descriptors, and the most specific match is what determines the rate limit.

descriptors:
- key: account_id
  descriptors:
  - key: plan
    value: BASIC
    rateLimit:
      requestsPerUnit: 1
      unit: MINUTE
  - key: plan
    value: PLUS
    rateLimit:
      requestsPerUnit: 20
      unit: MINUTE

The descriptors are defined in the Gloo Settings manifest.

apiVersion: gloo.solo.io/v1
kind: Settings
metadata:
  name: default
  namespace: gloo-system
spec:
  extensions:
    configs:
      envoy-rate-limit:
        customConfig:
          descriptors:
          - # list of descriptors here

The Envoy rate limiting actions associated with the Virtual Service or the individual routes allow you to specify how parts of the request are associated to rate limiting descriptor keys. For example, you can use the requestHeaders configuration to pass the value of a request header in as the value of the associated rate limiting descriptor key. Or you can use the genericKey to pass in a literal value such as a route identifier to allow you to rate limit differently based on the matched route.

You can specify more than one rate limit action, and the request is throttled if any one of the actions triggers the rate limiting service to signal throttling, i.e., the rate limiting actions are effectively OR’d together.

An example of the rate limiting actions is as follows. This action says to pass in the request header values as the descriptor values. That is, the x-account-id request header is sent to the rate limiting service as the descriptor key account_id.

rate_limits:
- actions:
  - requestHeaders:
      descriptorKey: account_id
      headerName: x-account-id
  - requestHeaders:
      descriptorKey: plan
      headerName: x-plan

Rate limit actions can be specified both at the Virtual Service level and on a per route basis. For example, the following sets up a default (virtual service level) rate limit action that maps header x-account-id to descriptor key account_id and x-plan to key plan. Any requests matching route prefix /service/service2 use this rate limit action. Requests matching prefix /service/service route use a different rate limiting action that maps header x-plan-other to key plan. Note the route also has an includeVhRateLimits: <boolean> configuration that defaults to false. If true, any specified Virtual Service level rate limit actions would be added to (OR’d) the routes list of actions. Each rate limit action is matched to descriptors independently.

The Envoy rate limiter uses a plugin extension name of `envoy-rate-limit`. The Gloo simpler rate limiter uses a plugin extension of `rate-limit`.
apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  name: default
  namespace: gloo-system
spec:
  virtualHost:
    domains:
    - '*'
    name: gloo-system.default
    routes:
    - matcher:
        prefix: /service/service1
      routeAction:
        single:
          upstream:
            name: default-echo-server-8080
            namespace: gloo-system
      routePlugins:
        extensions:
          configs:
            envoy-rate-limit:
              includeVhRateLimits: false
              rateLimits:
              - actions:
                - requestHeaders:
                    descriptorKey: account_id
                    headerName: x-account-id
                - requestHeaders:
                    descriptorKey: plan
                    headerName: x-plan-other
    - matcher:
        prefix: /service/service2
      routeAction:
        single:
          upstream:
            name: default-echo-server-8080
            namespace: gloo-system
    virtualHostPlugins:
      extensions:
        configs:
          envoy-rate-limit:
            rate_limits:
            - actions:
              - requestHeaders:
                  descriptorKey: account_id
                  headerName: x-account-id
              - requestHeaders:
                  descriptorKey: plan
                  headerName: x-plan

Rate Limiting Example

Install the pet clinic demo app and configure a route to that service in Gloo

kubectl apply \
  --filename https://raw.githubusercontent.com/solo-io/gloo/master/example/petclinic/petclinic.yaml

glooctl add route --name default --namespace gloo-system \
  --path-prefix / \
  --dest-name default-petclinic-8080 \
  --dest-namespace gloo-system

And check that everything is working as expected.

curl --head $(glooctl proxy url)
HTTP/1.1 200 OK
content-type: text/html;charset=UTF-8
content-language: en
content-length: 3939
date: Sun, 17 Mar 2019 15:42:04 GMT
x-envoy-upstream-service-time: 13
server: envoy

Configuring Envoy Rate Limits

Edit the rate limit server settings. This opens your default editor controlled by the EDITOR environment variable on most operating systems.

glooctl edit settings --namespace gloo-system --name default ratelimit custom-server-config

That command opens the rate limit server configuration in your editor. Paste the following descriptor block into the editor. This descriptor uses generic_key key, which works with the generic_key rate limiting action in Envoy, and effectively passes a literal value to the rate limiting service. For your convenience, you can download the descriptor block here. The structure of the rate limit server configuration is a list of keys and is expressed as a hierarchy for ease of configuration. For more information, see here.

descriptors:
  - key: generic_key
    rate_limit:
      requests_per_unit: 1
      unit: MINUTE
    value: some_value

The glooctl tool merges those descriptors into the Gloo Settings manifest like as follows.

apiVersion: gloo.solo.io/v1
kind: Settings
metadata:
  annotations:
    helm.sh/hook: pre-install
  labels:
    app: gloo
    gloo: settings
  name: default
  namespace: gloo-system
spec:
  bindAddr: 0.0.0.0:9977
  discoveryNamespace: gloo-system
  extensions:
    configs:
      envoy-rate-limit:
        customConfig:
          descriptors:
          - key: generic_key
            rateLimit:
              requestsPerUnit: 1
              unit: MINUTE
            value: some_value
      extauth:
        extauthzServerRef:
          name: extauth
          namespace: gloo-system
      rate-limit:
        ratelimit_server_ref:
          name: rate-limit
          namespace: gloo-system
  kubernetesArtifactSource: {}
  kubernetesConfigSource: {}
  kubernetesSecretSource: {}
  refreshRate: 60s

Edit Virtual Service Rate Limit Settings

Edit the virtual service settings:

glooctl edit virtualservice --namespace gloo-system --name default ratelimit custom-envoy-config

That command opens the virtual service rate limit configuration in your editor. Paste the following rate limit action block into the editor. For your convenience, you can download the rate limiting action here. The structure of the virtual service configuration is as described in the Envoy documentation. This configuration will be passed to Envoy as is.

rate_limits:
- actions:
  - generic_key:
      descriptor_value: "some_value"
You can run the same command for a *route* as well (`glooctl edit route ...`). When providing configuration for a `route`, you can also specify a boolean `include_vh_rate_limits` to include the rate limit actions from the virtual service.

Test

You can use glooctl proxy url to get Gloo proxy’s external endpoint. Run curl --head $URL a few times and eventually the curl response shows it is being rate limited:

curl --head $URL
HTTP/1.1 429 Too Many Requests
x-envoy-ratelimited: true
date: Sun, 17 Mar 2019 15:42:17 GMT
server: envoy
transfer-encoding: chunked

Other Configuration Options

Envoy queries an external server (backed by redis) to achieve global rate limiting. You can set a timeout for the query, and what to do in case the query fails. By default, the timeout is set to 100ms, and the failure policy is to allow the request.

You can check if envoy has errors with rate limiting by examining its stats that end in `ratelimit.error`. `glooctl proxy stats` displays the stats from one of the envoys in your cluster.

To change the timeout to 200ms, use the following command:

glooctl edit settings --name default --namespace gloo-system ratelimit --request-timeout=200ms

To deny requests when there’s an error querying the rate limit service, use this command:

glooctl edit settings --name default --namespace gloo-system ratelimit --deny-on-failure=true

Conclusion

With the custom rate-limit configuration option, you have the full power of Envoy rate limits to use for your custom use cases. The downside to this is the API is a bit more complicated. To leverage a simpler API that can do true per-user (logged-in, authenticated user) rate limits, take a look at Gloo’s simplified ratelimit API.