Design a chat/messaging backend — walk through message delivery, ordering, and online presence

#Scope the problem before you draw a single box

"Design a chat backend" is really three problems wearing a trenchcoat: durable message delivery, ordering inside a conversation, and presence (who's online, who's typing, who read what). A senior answer starts by pinning down which of these the interviewer cares about most, then states explicit requirements instead of guessing:

• 1:1 and group conversations, message history that syncs across a user's devices.
• At-least-once delivery — a message must never silently vanish, even if a client is offline for days.
• Ordering that's consistent within a conversation, not globally across the whole system.
• Presence (online / offline / typing / last-seen) that can be a little stale — nobody notices if "last seen" is 2 seconds behind reality.
• Scale: assume hundreds of millions of users, with a small fraction concurrently connected at any moment.

Notice the asymmetry already: messages need strong durability guarantees, presence needs almost none. Treating them the same — e.g., writing every heartbeat into the same durable store as messages — is the single most common design mistake in this question.

#The shape of the system

Four pieces do almost all the work: a connection/gateway layer that terminates persistent WebSocket (or long-lived HTTP/2) connections, a message service that owns the send/receive business logic, a message store that durably persists conversation history, and a presence service that tracks who's connected right now. A connection registry glues the gateway layer to everything else — it's the answer to "how does server B know that user X is connected to gateway node 3, not node 7."

#The path a single message actually takes

1. Client A sends a message over its open WebSocket to whichever gateway it's connected to, tagged with a client-generated UUID (for dedup) and the conversation id.
2. The gateway forwards it to the message service, which durably appends it to the message store before doing anything else — durability first, delivery second.
3. The message service looks up the recipient(s) in the connection registry. Online recipients get the message pushed immediately through their gateway. Offline recipients get an entry written to a per-user offline queue.
4. The sender gets an ack once step 2 succeeds — that's the "sent" checkmark. A second ack, generated by the recipient's client, is what turns it into "delivered."
5. On reconnect, a client pulls anything sitting in its offline queue (and/or asks the message store for "everything since sequence N"), so a missed push is never a lost message — just a delayed one.

#Ordering: per-conversation, never global

Global ordering across the whole system is both unnecessary and expensive to build. What actually matters is that everyone looking at conversation 42 sees the same order. The standard trick: route every message for a given conversation through a single logical writer (in practice, a partition of your message store keyed by conversation_id), and let that partition hand out a monotonically increasing sequence number as it appends. Because one partition is always written by one owner at a time, "increment and append" is trivially ordered without a distributed lock.

#Presence: cheap, ephemeral, and fine with being a little wrong

Presence is high-write-volume and low-value-per-write — a heartbeat every 15-30 seconds from every connected client. Put it in something like Redis with a short TTL key per user (presence:user_id expiring in, say, 45 seconds); a missed heartbeat naturally expires the key without anyone having to run cleanup logic. When a key is set or expires, publish a pub/sub? event so any gateway holding a socket for someone who has that user in their contact list can push a presence update.

Two refinements senior candidates usually raise on their own: debounce "went offline" by a few seconds so a brief network blip on the subway doesn't flap a user's status, and never write presence into the same durable, replicated store you use for messages — it doesn't need that durability and the write volume would swamp it.

#Fan-out: one-to-one, small groups, and the "celebrity" channel

For 1:1 chats and small groups, fan-out-on-write is simplest: when a message is sent, write a pointer to it into every member's inbox/timeline at send time, so reads are just "give me my inbox," O(1) per client. That breaks down for a broadcast channel with a million subscribers — writing a million inbox rows for one message is a write storm. For those, switch to fan-out-on-read: store the message once in the channel's log, and let each subscriber's client pull from that shared log (with a per-user "last read position" cursor). Most real systems run both strategies side by side, picking per-conversation based on member count.

#Storage shape

🧠 Check your understanding
0/3 · 0/3 answered
1. Why should message ordering be scoped per-conversation instead of global across the whole system?
Global ordering would violate at-least-once deliveryGlobal ordering is impossible to implement in any distributed systemPer-conversation ordering is cheaper and is the only guarantee users actually perceive or needDatabases cannot generate sequence numbers across partitions
2. Why is a wall-clock timestamp a poor choice for ordering messages within a conversation?
Timestamps take up more storage than integersClients are not allowed to see raw timestampsClock skew between servers/devices means timestamps aren't guaranteed to reflect true write orderTimestamps cannot be indexed efficiently
3. What is the connection registry for?
Rate-limiting abusive clientsStoring message history for complianceEncrypting messages in transitMapping a user id to the specific gateway node holding their live socket, so other services know where to push