Rate Limiter

What is a Rate Limiter?

A rate limiter caps how many requests a client (by IP, user ID, or API key) can make in a given time window. Requests over the limit receive 429 Too Many Requests.

Key operations:

• Check: Is this client under their limit?
• Increment: Record this request against their quota
• Block: Reject if over limit, pass through if under

Functional

• Limit requests per client per time window (e.g. 100 req/min)
• Support multiple limit keys: IP, user ID, API key
• Return Retry-After header on 429
• Allow different limits per endpoint/tier

Non-Functional

• Latency: < 5ms overhead per request
• Highly available — must not block all traffic if limiter fails
• Distributed: works across multiple app servers

Capacity

• 100K req/sec × 1 Redis INCR = 100K ops/sec
• Redis handles 1M+ ops/sec — well within limits

Core Problem: Distributed Counter

Local counters don't work — 10 servers with limit=100 means a user could make 1,000 requests. You need a shared counter across all servers.

Algorithm Comparison

Sliding window counter with Redis is the best balance.

Redis atomicity

Redis Lua scripts make increment + check race-condition-free. Single Redis node handles 1M+ ops/sec.

Fail open vs fail closed

If Redis is down: fail open (let traffic through) for availability, fail closed (block all) for security. Most APIs choose fail open with alerting.

Request Flow

1. Request hits API Gateway
2. Gateway calls Rate Limiter with (client_id, endpoint, timestamp)
3. Rate Limiter runs Redis Lua script — increment counter, check against limit
4. Over limit → 429 + Retry-After header
5. Under limit → forward to App Server

Redis Key Schema

rate:{client_id}:{endpoint}:{window_unix}  →  integer
TTL = window_size_seconds

Rules DB

Stores per-client/per-tier limits. Cached in-memory in the Rate Limiter (refreshed every 60s).

Sliding Window Counter

Approximates a sliding window using two fixed buckets:

smoothed = prev_count × (1 - elapsed_fraction) + curr_count

If smoothed > limit → reject.

Atomic Redis Lua Script

local count = redis.call('INCR', KEYS[1])
if count == 1 then
  redis.call('EXPIRE', KEYS[1], ARGV[1])
end
return count

High Scale

For very high throughput, shard Redis by client_id hash. No cross-shard ops needed.

What Interviewers Look For

1. Distributed counter — why local counters break at scale
2. Algorithm comparison — token bucket vs sliding window trade-offs
3. Redis atomicity — Lua scripts prevent race conditions
4. Headers — X-RateLimit-Remaining, Retry-After
5. Failure mode — fail open vs closed
6. Tiers — different limits for free vs paid users

Common Mistakes

• Local in-memory counters only
• Not addressing race conditions
• Forgetting what happens when the limiter itself is down