Design a rate limiter as a standalone service that multiple APIs can call

#Why pull rate limiting out of each API

Every API needs some form of rate limiting — to protect itself from abuse, to enforce a pricing tier's quota, or to protect a downstream dependency from being overwhelmed. The naive approach is to implement it inside each service, but that means every team reinvents the same logic, policies drift out of sync between services, and a client hitting three different APIs owned by three different teams can't be limited consistently against a single quota unless those teams somehow share state. Pulling rate limiting out into a standalone service — with a shared backing store — gives you one place to define policy, one place to change it, and a single source of truth for "how much of their quota has this client used right now," no matter which API they're calling.

#The algorithms, and what each one actually gets you

Fixed window counter. Count requests in discrete windows (e.g., a counter that resets every 60 seconds). Trivial to implement and cheap to store — one integer and one expiry per key — but it lets a client burst up to twice the intended limit at a window boundary: 100 requests in the last second of one window, then another 100 in the first second of the next, and both windows individually look compliant.

Sliding window log. Store the timestamp of every request in a sorted structure, and count how many fall within the trailing window on each check. This is exactly accurate — no boundary burst problem — but memory cost scales with request volume per key, which gets expensive at high request rates.

Sliding window counter. A practical middle ground: keep a counter for the current fixed window and the previous one, and estimate the sliding-window count as a weighted blend of the two based on how far into the current window you are. It's an approximation, but a good one, and it costs the same tiny amount of storage as the fixed-window approach.

Token bucket. A bucket holds up to `capacity` tokens and refills at `refill_rate` tokens per second; each request consumes one token (or more, for weighted costs) and is rejected if the bucket is empty. This is the most widely used algorithm in production rate limiters because it naturally allows short bursts up to the bucket's capacity while still enforcing a strict average rate over time — which matches how real client traffic actually behaves far better than a hard per-second cap does.

Leaky bucket. Conceptually the inverse of token bucket: requests fill a queue (the bucket), and they're processed ("leaked") out at a constant rate regardless of how bursty the input was. This is less about rejecting excess traffic and more about shaping it into a steady output rate — useful when the concern is protecting a downstream system from bursty input rather than strictly enforcing a per-client quota.

A closely related, more storage-efficient cousin of token bucket worth knowing by name is GCRA (the Generic Cell Rate Algorithm): instead of storing both a token count and a timestamp per key, it stores a single "theoretical arrival time" value and derives the decision arithmetically, which several high-throughput rate limiters use specifically to cut the per-key storage in half.

#Architecture of a standalone rate limiter service

There are two common ways to wire this in. The first is a client library embedded in each service that talks directly to the shared Redis cluster — lowest latency, since there's no extra network hop through an intermediary service, but every service now needs to speak Redis and agree on the exact key format and script logic, which is a form of coupling. The second is a dedicated rate-limiter microservice (commonly exposed over gRPC for low overhead) that every other service calls before proceeding — this is the pattern used by API gateways and service meshes: the gateway calls the rate-limit service on every request, gets an allow/deny decision back, and only forwards the request downstream if allowed. It costs one extra network round trip per request, but it means policy logic, the storage backend, and any future algorithm change live in exactly one place.

Whichever shape you choose, the store behind it needs to be fast (in-memory), shared (every instance of the rate limiter must see the same counts, or you're not actually enforcing a global limit), and support atomic read-modify-write — which is exactly what Redis, accessed through a Lua script, is built for.

#Implementing token bucket atomically in Redis

The naive version of this — `GET` the current token count, compute the new value in application code, `SET` it back — has a race condition: two concurrent requests can both read the same token count before either writes back, and both get allowed when only one should have been. The fix is to do the entire check-and-update as a single atomic operation inside Redis itself, using a Lua script (Redis guarantees a script runs to completion without interleaving with any other command):

-- token_bucket.lua
-- KEYS[1]  = bucket key, e.g. "ratelimit:{client_id}"
-- ARGV[1]  = capacity (max tokens the bucket can hold)
-- ARGV[2]  = refill_rate (tokens added per second)
-- ARGV[3]  = now (current unix timestamp, as a float)
-- ARGV[4]  = requested (tokens this request costs, usually 1)

local key = KEYS[1]
local capacity = tonumber(ARGV[1])
local refill_rate = tonumber(ARGV[2])
local now = tonumber(ARGV[3])
local requested = tonumber(ARGV[4])

local bucket = redis.call("HMGET", key, "tokens", "timestamp")
local tokens = tonumber(bucket[1])
local last_refill = tonumber(bucket[2])

if tokens == nil then
    tokens = capacity
    last_refill = now
end

local elapsed = math.max(0, now - last_refill)
tokens = math.min(capacity, tokens + elapsed * refill_rate)

local allowed = 0
if tokens >= requested then
    tokens = tokens - requested
    allowed = 1
end

redis.call("HMSET", key, "tokens", tokens, "timestamp", now)
redis.call("EXPIRE", key, math.ceil(capacity / refill_rate) + 1)

return {allowed, tokens}

The service layer around it loads the script once, caches its SHA, and invokes it with `EVALSHA` on every check (falling back to loading the script again if Redis has evicted it, which surfaces as a `NOSCRIPT` error):

const crypto = require("crypto");
const fs = require("fs");

const script = fs.readFileSync("./token_bucket.lua", "utf8");
const scriptSha = crypto.createHash("sha1").update(script).digest("hex");

async function checkRateLimit(redis, params) {
  const bucketKey = "ratelimit:" + params.clientId;
  const now = Date.now() / 1000;

  try {
    const result = await redis.evalsha(
      scriptSha,
      1,
      bucketKey,
      params.capacity,
      params.refillRate,
      now,
      params.cost || 1
    );
    return { allowed: result[0] === 1, remaining: result[1] };
  } catch (err) {
    if (err.message && err.message.indexOf("NOSCRIPT") !== -1) {
      await redis.script("LOAD", script);
      return checkRateLimit(redis, params);
    }
    throw err;
  }
}

#The response contract

Whatever algorithm backs it, the service's HTTP or gRPC response should let callers self-regulate: return the decision (`allowed`), the remaining quota, and when the client can retry. Over HTTP, this typically means standard headers on every response — `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset` — and a `429 Too Many Requests` with a `Retry-After` header when denied, so well-behaved clients can back off instead of hammering the service again immediately.

#Comparing the algorithms

🧠 Check your understanding
0/2 · 0/2 answered
1. Why does implementing rate limiting as separate Redis INCR and EXPIRE calls from application code create a race condition, even though each command is individually atomic?
Redis does not support the EXPIRE command on integer keysINCR and EXPIRE are not real Redis commandsINCR always returns the wrong value under concurrencyThe two commands together are not atomic as a pair — a crash or delay between them can leave a key with no TTL, or a concurrent request can reset the TTL unexpectedly
2. What is the main advantage of token bucket over a fixed-window counter for rate limiting a public API?
Token bucket requires no shared storage at allToken bucket is easier to implement without RedisToken bucket allows short bursts up to the bucket's capacity while still enforcing a strict long-term average rate, instead of allowing up to 2x the limit right at a window boundaryFixed-window counters cannot be used with HTTP APIs