Kafka vs. RabbitMQ vs. SQS — how do you actually choose for a given use case?

#Why this question separates candidates

Anyone can list three message broker names. What separates a senior answer is recognizing that Kafka, RabbitMQ, and SQS are not interchangeable implementations of "a queue" — they encode three different mental models of what a message is and how long it should live. Get the mental model right and the "which one do I pick" question mostly answers itself.

#What each one actually is

Kafka is a distributed, partitioned, replicated commit log. Producers append events to a topic's partitions; the broker does not track which consumer has "processed" what. Instead, each consumer group tracks its own offset into each partition. Events are retained for a configured window (or indefinitely, or compacted down to the latest value per key) regardless of whether anyone has read them yet. This is what makes Kafka a replay system: a new consumer group can attach today and read a topic's entire history from offset zero.

RabbitMQ is a message broker built around the AMQP model: producers publish to an exchange, and bindings route each message to zero or more queues based on a routing key (direct, topic, fanout, or headers exchange). A message is removed from a queue once a consumer acknowledges it. RabbitMQ does not retain history for replay — once consumed and acked, the message is gone. What it buys you instead is expressive routing, per-message priorities, delayed/TTL messages, and dead-lettering, all as first-class broker features.

SQS is AWS's fully managed distributed queue. A standard queue gives at-least-once delivery with best-effort ordering and near-unlimited throughput; a FIFO queue gives strict per-message-group ordering and exactly-once processing (via a 5-minute deduplication window) at a lower throughput ceiling (up to ~3,000 messages/sec with batching). SQS has no exchange concept of its own — fan-out to multiple queues is done by pairing it with SNS (SNS topic → multiple SQS subscriber queues).

#The core distinction: log vs. broker

This single distinction drives almost every downstream difference: retention, replay, ordering guarantees, and how you scale consumers.

#Comparison table

#Choosing by use case

High-throughput streaming / event sourcing / analytics pipelines → Kafka. If multiple independent teams need to consume the same event stream at different paces (fraud detection reading in real time, a data warehouse loading it hourly, a new team backfilling from day one), Kafka's log-and-replay model is the only one of the three built for it. It's also the right choice when ordering within a key matters at scale — e.g., all events for a given user ID must be processed in order — because Kafka partitions by key and guarantees order within a partition.

Task queues / complex routing / RPC-like patterns → RabbitMQ. If you need a classic work queue (many workers competing for jobs), priority queues, delayed retries with per-message TTL, or routing a message to different queues based on content (topic exchange with wildcard bindings), RabbitMQ's broker-side routing does this natively without extra infrastructure. It also shines for request/reply (RPC over messaging) using a reply-to queue and correlation IDs.

Simple decoupling with minimal ops → SQS. If you just need to decouple two AWS-native services — a Lambda that needs to retry safely, an S3 upload that should trigger asynchronous processing, a background job queue for a small team without dedicated infra staff — SQS gives you durable delivery with zero servers to run. Use Standard queues unless you have a specific, narrow requirement for strict ordering or exactly-once semantics, since FIFO's throughput ceiling and added latency are a real cost.

#A routing example

The diagram below shows why RabbitMQ's topic exchange is a different tool than Kafka's partitioning: one message can be routed to multiple queues based on pattern matching on the routing key, each queue serving a completely different consumer.

#What interviewers listen for

A strong answer doesn't just recite the table — it explains the consequence of the underlying model: that Kafka's replay changes how you think about reprocessing after a bug (replay from an earlier offset instead of replaying a dead-letter queue), that RabbitMQ's ack-based delivery means an unacked message is redelivered and your consumer must be idempotent?, and that SQS's visibility timeout is effectively a lease that must be extended for long-running jobs or the message reappears and gets processed twice.

🧠 Check your understanding
0/3 · 0/3 answered
1. A team needs three independent services — fraud detection (real-time), a data warehouse (hourly batch load), and a new analytics team backfilling from day one — to each consume the full history of the same event stream at their own pace. Which system's model fits natively?
RabbitMQ, using three separate queues bound to a fanout exchangeAny of the three, since replay is a standard feature of all message brokersKafka, since each consumer group tracks its own offset and can replay from any pointSQS, using three FIFO queues subscribed to one SNS topic
2. Why is an SQS Standard queue a poor fit for a workflow that requires strict per-key ordering at high volume?
Standard queues cannot scale past a few hundred messages per secondStandard queues require polling, which is inherently unorderedStandard queues do not support at-least-once deliveryStandard queues offer best-effort ordering only, not a guarantee
3. What operational cost does a team take on by choosing Kafka for a low-volume integration between two services that don't need replay or high throughput?
Partition rebalancing, offset/consumer-lag monitoring, and cluster management for a problem a simple queue would solveNone — Kafka has no operational overhead beyond RabbitMQ or SQSKafka requires rewriting consumers in a JVM languageKafka cannot guarantee at-least-once delivery for low-volume topics