rate limiter as access control

[johakoch]

We could implement a rate limiter as an access control, e.g.:

• respond with 429 if the request is blocked
• fixed/sliding window (or more?)
• config could be similar to rate limiting in backend
• could be one "global" rate limiter; or one per key expression value, e.g. depending on request.<property> or a JWT claim (request.context.at.sub, if used after an "at" jwt access control)

server {
  api {
    access_control = ["global_rl", "at", "jwt_rl"]
    ...
  }
}

definitions {
  rate_limiter "global_rl" {
    period_window = "sliding" # or "fixed" or ?
    period = "1s"
    per_period = 1000
  }

  jwt "at" {
    ...
  }

  rate_limiter "jwt_rl" {
    period = "1s"
    per_period = 1
    key = request.context.at.sub
  }
}

It could be handy to add more request. sub-properties, e.g. from go Request.RemoteAddr if available.

[filex]

I like the idea. How would you define the order? We could inspect expressions (especially those using request.context), like we do to order request sequences. Or would you use the order of references in the access_control list?

[johakoch]

Or would you use the order of references in the access_control list?

I guess that's the order in which access controls are (already) applied.

[malud]

I would apply the ordered list - no magic reordering.

Should we reply with some additional headers like https://github.com/sashabaranov/go-openai/blob/master/ratelimit.go#L10 ? This could also be interesting for our BE limiters.

[johakoch]

Should we reply with some additional headers like https://github.com/sashabaranov/go-openai/blob/master/ratelimit.go#L10 ? This could also be interesting for our BE limiters.

See also

• https://github.com/ietf-wg-httpapi/ratelimit-headers and https://datatracker.ietf.org/doc/draft-ietf-httpapi-ratelimit-headers/ or https://greenbytes.de/tech/webdav/draft-ietf-httpapi-ratelimit-headers-latest.html
• https://developer.okta.com/docs/reference/rl-best-practices/

[filex]

Let's start with a simple implementation that just uses status 429 to communicate current overuse.

[filex]

The key in the proposal is the "key" feature :).

Without it, rate limiting is almost useless. We could even make it a mandatory field.

Is it worth to provide a non-dynamic alternative to the key (which has to be evaluated in cty land) to be more efficient or make typical use-cases simpler? Example:

by_header = "API-Key"
by_client_ip = true

Anyways, we can start with the dynamic key and introduce those optimized shortcuts later once typical use is clear.