At-most-once vs. at-least-once vs. exactly-once delivery — what does each guarantee actually mean, and is exactly-once really possible?

#"Exactly-once" is not what it sounds like

Delivery semantics describe what a messaging system promises to do at the exact moment something goes wrong — a broker restarts, a network blip drops a connection, a consumer crashes mid-processing. There are exactly three possible contracts, and understanding the failure each one produces is more useful than memorizing the definitions.

#At-most-once: messages can be lost, never duplicated

The system delivers a message zero or one times. If anything goes wrong after the broker sends it but before the consumer finishes, the message is simply gone — nobody retries it. This is what you get by default from fire-and-forget UDP-style protocols, or from a consumer that acknowledges (or commits its offset) before processing the message.

At-most-once is acceptable for data where losing an occasional message is cheaper than the complexity of guaranteeing delivery — a stream of periodic sensor telemetry where the next reading arrives in a second anyway, or a "user is typing" indicator. It is almost never acceptable for anything with business consequences: payments, inventory decrements, order confirmations.

#At-least-once: messages can be duplicated, never (silently) lost

Flip the order: the consumer acknowledges (or commits) after it finishes processing. Now if the consumer crashes mid-processing, the broker never received the ack, so it redelivers the same message — possibly to the same consumer after it restarts, possibly to a different one. The message is never silently lost, but it might be processed twice.

At-least-once is the default posture of most production systems (Kafka with manual offset commits, SQS standard queues, RabbitMQ with manual acks) because losing data is almost always worse than processing it twice — provided you can neutralize the duplicate. Which brings us to the real answer to "is exactly-once really possible."

#Exactly-once: the delivery guarantee you actually want, achieved by not needing it

True end-to-end exactly-once — the message crosses the network exactly one time and produces its side effect exactly one time, guaranteed by the messaging layer alone — runs into the same wall as the classic "Two Generals Problem": you cannot get two independent parties (the broker and the consumer's external side effect, like a database write or a third-party API call) into perfect agreement over an unreliable network, because the last acknowledgment in any protocol can always be lost.

What the industry actually calls "exactly-once" is one of two things, both of which quietly rely on at-least-once delivery underneath:

1. Idempotent processing on top of at-least-once delivery. The message might arrive twice, but the effect only happens once, because the consumer recognizes the duplicate and no-ops on the second attempt. This is the practical, portable answer, and it works with any broker.
2. A narrower, protocol-level guarantee inside a single system. Kafka's "exactly-once semantics" (idempotent? producer + transactions) genuinely prevents duplicate writes within Kafka — a producer retry won't create two copies of a record in a partition, and a consume-transform-produce pipeline commits input offsets and output records atomically. That's real and valuable, but it stops at Kafka's boundary: the moment your consumer's side effect is an external HTTP call, a row in Postgres?, or an email send, you're back to needing idempotency at that boundary too.

The usual ways to make processing idempotent:

• Natural idempotency: SET balance = 100 is idempotent no matter how many times it runs; balance = balance + 10 is not. Prefer the former shape when possible.
• Idempotency keys / dedup tables: store the message's unique ID in the same transaction as its side effect (e.g., "insert into processed_messages(msg_id), then update the order — both in one DB transaction"). A retry sees the ID already present and skips.
• Upserts instead of inserts: INSERT ... ON CONFLICT (order_id) DO NOTHING turns a duplicate insert into a no-op.
• The transactional outbox pattern: write the business change and the "event to publish" row in the same local transaction, then a separate relay publishes from the outbox table — this avoids the classic dual-write problem (DB committed, publish failed, or vice versa) that creates duplicates or losses in the first place.