What are the main load-balancing algorithms — round robin, least connections, consistent hashing — and when does each fit?

#Why the algorithm choice matters

A load balancer's job sounds simple — pick a backend for each incoming request — but the algorithm behind that choice determines whether your backends stay evenly loaded, whether a slow request on one server drags others down, and whether a cache-fronting layer keeps its hit rate when you add or remove a node. Interviewers ask about this to see if you know that "load balancing" isn't one algorithm — it's a family of trade-offs, and picking wrong for the traffic shape in front of you causes real incidents.

#Round robin and weighted round robin

Round robin sends request 1 to server A, request 2 to server B, request 3 to server C, and back to A — cycling through the pool in order. It requires no state about the backends beyond the pool list itself, which makes it cheap and predictable. Weighted round robin extends this by giving each server a weight proportional to its capacity, so a server with weight 3 gets three requests for every one a weight-1 server gets — useful when your fleet is heterogeneous (a mix of instance sizes during a migration, for example).

Round robin's blind spot is that it assumes every request costs roughly the same amount of work. It doesn't track how busy a server actually is, so if one request happens to be expensive (a large report generation, a slow downstream call), round robin will happily send the next request to that same overloaded server anyway.

#Least connections

Least connections routes each new request to whichever backend currently has the fewest open connections. This directly addresses round robin's blind spot: it reacts to actual load rather than assuming uniform request cost, which makes it the better default for workloads with variable request duration — long-lived WebSocket connections, gRPC streams, or APIs where some endpoints are far more expensive than others.

Its cost is that the load balancer must track connection counts per backend in real time, which is a small amount of state and coordination that round robin doesn't need — usually trivial for a single load balancer instance, but worth knowing if you're running multiple LB instances that don't share connection state.

#IP hash / source hashing

IP hash (or source-IP hashing) computes a hash of the client's IP address and uses it to consistently pick the same backend for that client on every request. This gives you session affinity ("sticky sessions") without a shared session store — convenient, but with real downsides: it distributes load unevenly if your client IPs aren't uniformly distributed (a large NAT gateway, a corporate proxy, or a mobile carrier's shared IP pool can concentrate huge traffic onto one hash bucket), and it breaks the moment a client's IP changes mid-session (mobile network handoff) or a server is removed from the pool, since removing one node reshuffles which bucket every other client hashes into.

#Consistent hashing

Consistent hashing solves exactly that reshuffling problem. Instead of mapping `hash(key) mod N` (which remaps almost every key when N changes), it places both servers and keys on a hash ring, and a key is served by the next server clockwise from its position on the ring. Adding or removing one server only reshuffles the keys between that server and its neighbor on the ring — roughly `K/N` keys move instead of nearly all of them. Real implementations add virtual nodes (each physical server gets many points on the ring) to avoid uneven bucket sizes.

This is the algorithm you reach for in front of a stateful backend layer — a cache tier, a sharded key-value store — where which node a key lands on matters for cache locality, not just for spreading connection count. Getting this wrong on a cache layer means a rolling deploy that adds or removes one cache node can cause a massive, correlated cache-miss stampede against your database.

#Real configuration

# Round robin (default) with weights — good for a homogeneous, stateless API tier
upstream api_backend {
    server 10.0.1.10:8080 weight=3;
    server 10.0.1.11:8080 weight=1;
    server 10.0.1.12:8080 backup;
}

# Least connections — good when request cost varies a lot (websockets, long polling)
upstream ws_backend {
    least_conn;
    server 10.0.2.10:8080;
    server 10.0.2.11:8080;
    server 10.0.2.12:8080;
}

# Consistent hashing on the request URI — good for a cache-fronting layer
upstream cache_backend {
    hash $request_uri consistent;
    server 10.0.3.10:11211;
    server 10.0.3.11:11211;
    server 10.0.3.12:11211;
}

server {
    listen 80;

    location /api/ {
        proxy_pass http://api_backend;
    }

    location /ws/ {
        proxy_pass http://ws_backend;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}

# Equivalent intent expressed in HAProxy

backend api_servers
    balance roundrobin
    server web1 10.0.1.10:8080 weight 3 check
    server web2 10.0.1.11:8080 weight 1 check

backend ws_servers
    balance leastconn
    server ws1 10.0.2.10:8080 check
    server ws2 10.0.2.11:8080 check

backend cache_servers
    balance uri
    hash-type consistent
    server cache1 10.0.3.10:11211 check
    server cache2 10.0.3.11:11211 check
    server cache3 10.0.3.12:11211 check

#Matching the algorithm to the traffic

🧠 Check your understanding
0/2 · 0/2 answered
1. Why does removing one node from a consistent-hashing ring only remap roughly K/N keys, instead of nearly all of them like plain hash(key) mod N would?
Keys are placed on a ring and only the keys between the removed node and its neighbor need to move, not keys served by unrelated nodesIt relies on TTL-based cache expiry instead of remappingConsistent hashing uses a smaller hash spaceConsistent hashing does not actually redistribute any keys on removal
2. A load balancer sits in front of an API where a handful of endpoints do heavy report generation and take 10-20x longer than the rest, while most backends are otherwise identical. Which algorithm best handles this out of the box?
Least connectionsWeighted round robin with static weightsRound robinIP hash