How do you implement rate limiting that's correct across multiple server instances, not just one process?

#When "Just Count Requests in Memory" Stops Working

A naive rate limiter keeps a counter per client key in a local data structure — a Map, a fixed-size array of buckets, whatever — and resets or slides it over time. That works perfectly in a single process. It falls apart the moment you run more than one instance of your service, which in practice means almost immediately: two containers behind a load balancer, a Kubernetes? Deployment with replicas: 3, or even a single instance that gets restarted mid-window.

The failure mode is simple to see. If your limit is "100 requests per minute per API key" and you run 4 instances behind a round-robin load balancer, each instance independently believes it owns the full quota. A client can now make close to 400 requests per minute — one hundred against each instance — before any single process refuses a request. The limiter is not wrong on any one node; it is wrong about the system. Correct rate limiting requires state that is shared across every instance that can see a given client's traffic, and that state has to be updated atomically, or you reintroduce the same race at a smaller scale.

#Picking an Algorithm

Before worrying about where the state lives, decide what "100 requests per minute" should actually mean at the boundary of the window, because the algorithms disagree.

For most API rate limits, a sliding window log (exact) or token bucket (burst-tolerant) is the right trade-off; fixed windows are simplest but leak the boundary-burst bug, which is a common interview follow-up.

#The Real Hazard: Check-Then-Act Is Not Atomic

Even once you move the counter into Redis, a naive implementation still races:

1. Instance A runs GET count → 99 (limit is 100)
2. Instance B runs GET count → 99, at nearly the same moment
3. Both see 99 < 100 and decide to allow the request
4. Both run INCR count → the counter ends at 101, and two requests were admitted that shouldn't both have been

This is a classic check-then-act race, the distributed-systems cousin of a non-atomic i++ across threads. The fix is the same shape as the in-process fix: make the "read current state, decide, mutate state" sequence a single atomic operation, so no other instance can observe an intermediate state.

#Making the Whole Decision Atomic With a Lua Script

Redis executes each Lua script as a single, uninterruptible operation — no other command from any client (on that server/shard) runs in the middle of it. That gives you exactly the atomicity the check-then-act sequence needs, without a distributed lock. A sliding-window log limiter looks like this:

Wrap long lines
-- KEYS[1]  = rate limit key, e.g. "ratelimit:{user:42}"
-- ARGV[1]  = window size in milliseconds
-- ARGV[2]  = max requests allowed in the window
-- ARGV[3]  = unique id for this request (sorted set member)
local key = KEYS[1]
local window_ms = tonumber(ARGV[1])
local limit = tonumber(ARGV[2])
local member = ARGV[3]

local time_parts = redis.call('TIME')
local now_ms = (tonumber(time_parts[1]) * 1000) + math.floor(tonumber(time_parts[2]) / 1000)

redis.call('ZREMRANGEBYSCORE', key, '-inf', now_ms - window_ms)

local current = redis.call('ZCARD', key)

if current < limit then
  redis.call('ZADD', key, now_ms, member)
  redis.call('PEXPIRE', key, window_ms)
  return {1, limit - current - 1}
else
  return {0, 0}
end

Note the script asks Redis for the current time with TIME rather than trusting the caller's clock — four app instances on four hosts do not agree on the exact millisecond, and letting each one pass its own Date.now() into the script would make the window boundary fuzzy across instances. Letting the server be the single source of truth for time removes that source of drift.

The application side just needs a client that can register and call the script:

const Redis = require('ioredis');
const redis = new Redis(process.env.REDIS_URL);

const SCRIPT = require('fs').readFileSync('sliding_window.lua', 'utf8');

redis.defineCommand('slidingWindowLimit', {
  numberOfKeys: 1,
  lua: SCRIPT,
});

async function checkRateLimit(clientId, limit, windowMs) {
  const key = 'ratelimit:{' + clientId + '}';
  const member = Date.now() + '-' + Math.random().toString(36).slice(2);
  const result = await redis.slidingWindowLimit(key, windowMs, limit, member);
  return { allowed: result[0] === 1, remaining: result[1] };
}

// Express middleware
app.use(async (req, res, next) => {
  const { allowed, remaining } = await checkRateLimit(req.apiKey, 100, 60000);
  res.set('X-RateLimit-Remaining', String(remaining));
  if (!allowed) {
    return res.status(429).set('Retry-After', '1').json({ error: 'rate_limited' });
  }
  next();
});

#How This Looks Across Instances

Every instance talks to the same Redis key, and every check-and-increment is one atomic script execution, so it does not matter which instance a given request lands on — the decision is made against one consistent piece of shared state.

#Operational Details That Come Up in Interviews

• Cluster key placement. If Redis runs as a cluster, a Lua script can only touch keys that hash to the same slot. Wrap the client id in a hash tag — ratelimit:{user:42} — so Redis routes the whole key to one slot regardless of what else is in the key name.
• Fail-open vs. fail-closed. Decide up front what happens if Redis itself is unreachable. Fail-closed (reject all requests) protects your backend but turns a Redis outage into a total outage. Fail-open (allow all requests) keeps you available but removes protection exactly when a spike might be causing the Redis problem in the first place. Most production systems fail open for rate limiting specifically, because losing rate limiting for a few seconds is usually cheaper than an outage.
• Latency budget. A round trip to Redis adds real latency to every request. Co-locate the rate-limiting Redis in the same region/AZ as your app instances, and consider a local, short-TTL cache of "definitely still allowed" results to skip the round trip for the common case.
• TTL hygiene. Always set an expiry on the rate-limit key (as the script does with PEXPIRE) so abandoned keys for clients who stop sending traffic don't accumulate forever.

🧠 Check your understanding
0/3 · 0/3 answered
1. Why does a naive in-memory rate limiter under-enforce its limit once you run multiple instances behind a load balancer?
Because Redis is required for any rate limiting to workBecause load balancers cache responses and skip the limiterBecause each instance tracks its own counter, so the effective limit multiplies by the number of instancesBecause in-memory counters overflow past a few hundred requests
2. What specific race does a naive GET-then-INCR rate limiter implementation in Redis suffer from, even though Redis itself is single-threaded?
INCR is not supported on string keysRedis silently drops commands under high loadThe GET and INCR are two separate commands, so two clients can both read the same 'under limit' value before either writes backRedis commands are not durable across restarts
3. Why does the Lua script fetch the current time with Redis's TIME command instead of accepting a timestamp from the calling application?
It is required by the sorted set data typeUsing the server's clock avoids clock skew between different application instances, keeping the window boundary consistentTIME is faster than any client-side clock callLua cannot access the system clock at all