How does Change Data Capture (CDC) with something like Debezium work, and what problem does it solve?
CDC turns every row-level change already happening in your database into a stream of events, without touching application code or risking the dual-write problem.
LEARN BY DOINGPut your intuition to work3 steps
How does Change Data Capture (CDC) with something like Debezium work, and what problem does it solve?
Use the lesson’s examples and explanation to check your idea.
Personal reflection. Nothing is sent to an AI service.#The problem CDC exists to solve: the dual-write
Imagine an order service that needs to both save an order to its database and publish an OrderPlaced event so other services can react. The naive implementation does both in application code:
save(order) // write #1: the databasepublish(OrderPlaced) // write #2: the message broker
These are two separate systems with no shared transaction. If the process crashes between the two calls, or the broker is unreachable, you get a database row with no corresponding event (a downstream system never hears about a real order) or — if you publish first — an event for an order that never actually committed. This is the dual-write problem, and it is not a rare edge case; it happens on every deploy, every network blip, every broker outage. CDC solves it by removing the second write entirely: instead of the application explicitly publishing an event, a separate process reads the database's own internal change log and turns each committed row change into an event, guaranteed to reflect only what was actually committed.
#How Debezium actually works, mechanically
Debezium does not poll tables with SELECT queries — that would miss deletes, be slow, and put load on the database it's trying to observe. Instead, it reads the database engine's own write-ahead log: PostgreSQL's logical replication stream (via a replication slot and an output plugin like pgoutput), or MySQL's binlog (in row-based format). This is the exact same mechanism the database already uses to replicate to its own read replicas — Debezium just attaches as another replication consumer, which means capturing changes adds negligible load to the primary and, critically, cannot miss a committed change, because the WAL/binlog is the source of truth the database itself relies on for durability.
Debezium runs as a source connector inside Kafka Connect. On startup it (optionally) takes an initial consistent snapshot of the existing table data, then switches to streaming: every subsequent insert, update, and delete is decoded from the log and published to a Kafka topic, typically one topic per table. Each event carries the row's state before and after the change, the operation type (c = create, u = update, d = delete, r = read/snapshot), and source metadata like the transaction ID and log position — enough for a consumer to reconstruct exactly what happened without ever querying the source database.
#A real Debezium PostgreSQL connector config
This is the JSON body you'd PUT/POST to the Kafka Connect REST API to register the connector:
{"name": "orders-db-connector","config": {"connector.class": "io.debezium.connector.postgresql.PostgresConnector","database.hostname": "orders-db.internal","database.port": "5432","database.user": "debezium_replicator","database.password": "${file:/opt/kafka/secrets/db-creds.properties:debezium_password}","database.dbname": "orders","topic.prefix": "orders-db","table.include.list": "public.orders,public.order_items,public.outbox_event","plugin.name": "pgoutput","slot.name": "debezium_orders_slot","publication.name": "debezium_orders_pub","snapshot.mode": "initial","heartbeat.interval.ms": "10000","transforms": "outbox","transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter","transforms.outbox.table.field.event.key": "aggregate_id","transforms.outbox.table.field.event.payload": "payload","transforms.outbox.route.by.field": "aggregate_type"}}
A few fields worth explaining in an interview: slot.name creates a durable PostgreSQL replication slot — the database will retain WAL segments the connector hasn't consumed yet, which is what makes CDC resilient to a connector restart but also a real operational hazard (see below). table.include.list scopes capture to specific tables rather than the whole database. snapshot.mode: initial means it captures existing rows once at startup before switching to streaming.
#The outbox pattern: CDC's best-practice pairing
Capturing raw table changes directly is workable but leaks internal schema into your event contract — a downstream consumer ends up coupled to your orders table's exact columns. The stronger pattern, visible in the config above, is the transactional outbox: the application writes its business change and a row describing the event (aggregate_id, aggregate_type, payload) into an outbox_event table, in the same database transaction as the business write. Debezium captures changes to that outbox table specifically, and the EventRouter single-message transform reshapes each captured row into a clean domain event routed to a topic named by aggregate_type — so consumers see OrderPlaced on an orders topic, never a raw Postgres row.
This combination — an outbox table plus CDC — gets you the reliability of "the event is guaranteed to exist if and only if the business write committed" without ever performing a manual dual write in application code, because the only write the application code does is a single local database transaction.
#Gotchas worth naming in an interview
Other things worth mentioning if asked to go deeper: an initial snapshot on a very large table can take hours and adds load exactly when you're bringing up a new connector; schema changes on the source table (a new column, a type change) need to be handled by whatever's consuming the topic, since CDC faithfully propagates the new shape immediately; and if you need strict per-entity ordering downstream, you generally key the Kafka topic by the row's primary key so all changes to one row land in the same partition.
🧠 Check your understanding
0/3 · 0/3 answered1. An application writes a row to its `orders` table, then makes a separate call to publish an OrderPlaced event to a message broker. What specific failure mode does this dual-write pattern create?
2. Why does Debezium read the database's write-ahead log/binlog instead of periodically polling tables with SELECT queries?
3. What operational risk does a Debezium PostgreSQL replication slot introduce if the connector is down for a long period?