What is backpressure in a distributed system, and how do you actually implement it?

#What backpressure actually means

Backpressure is the mechanism by which a slow consumer tells a fast producer to slow down, instead of the producer piling up unbounded work somewhere in between. It sounds like a niche streaming concept, but it shows up everywhere: an HTTP server accepting connections faster than it can handle them, a Kafka consumer falling behind its producer, a downstream database rejecting writes while an upstream service keeps queuing them, or a UI event handler that can't keep up with mouse-move events. The absence of backpressure doesn't mean the system copes silently — it means the excess work goes somewhere: an in-memory queue grows until the process runs out of heap, a thread pool's queue grows until latency becomes unbounded, or the OS's TCP receive buffer fills and the kernel starts dropping packets. None of those are "handling" the problem; they're deferring an outage.

#The naive failure: unbounded queues

The most common anti-pattern is an unbounded work queue between a producer and a consumer — a LinkedBlockingQueue<Runnable>() with no capacity, an ArrayList used as a buffer, a channel with no size limit. Under normal load this is invisible. Under a traffic spike, a slow downstream dependency, or a GC pause, the producer keeps enqueueing and the queue keeps growing, and you get an out-of-memory kill or a latency cliff right when you can least afford it — usually during an incident, which makes the incident worse.

#Strategies for implementing it

There are really four moves available once you decide to push back deliberately:

1. Bounded buffer + blocking — cap the queue size and make put()/send() block (or suspend) when full. The producer naturally slows to the consumer's pace. Simple, but a blocking producer can itself become a bottleneck upstream (backpressure propagates backward through the whole chain, which is usually what you want).
2. Bounded buffer + drop/reject — cap the queue and, when full, reject new work immediately (fail fast, return 429 Too Many Requests) or drop the oldest/newest item (load shedding). Good when freshness matters more than completeness (metrics, telemetry, live video frames).
3. Pull-based / reactive streams — instead of the producer pushing whenever it wants, the consumer explicitly requests n more items when it's ready for them. This is the Reactive? Streams / java.util.concurrent.Flow model and it's the only approach that gives you backpressure without buffering at all in the ideal case, because the producer never produces more than was asked for.
4. Rate limiting at the edge — a token bucket or leaky bucket in front of the whole pipeline, independent of any single queue, caps the rate of admission rather than reacting to a queue filling up. Complementary to the other three, not a replacement.

#A bounded-queue example with real backpressure

Kotlin coroutines' Channel is a clean way to show this because a bounded channel's send suspends — not blocks a thread, suspends a coroutine — when the channel is full, which is exactly the "producer slows to consumer's pace" behavior:

import kotlinx.coroutines.*
import kotlinx.coroutines.channels.Channel

// Capacity of 100: once 100 items are buffered, send() suspends
// until the consumer makes room by receiving one.
val workQueue = Channel<WorkItem>(capacity = 100)

fun CoroutineScope.startProducer(source: Iterator<WorkItem>) = launch {
    for (item in source) {
        // This suspends the coroutine (not the thread) whenever
        // the channel is full -- that suspension IS the backpressure.
        workQueue.send(item)
    }
    workQueue.close()
}

fun CoroutineScope.startConsumer(process: suspend (WorkItem) -> Unit) = launch {
    for (item in workQueue) {
        process(item) // slow processing here directly throttles the producer
    }
}

data class WorkItem(val id: Long, val payload: ByteArray)

If process() takes 200ms per item and the producer can generate 10,000 items/second, the channel fills within 10ms and every subsequent send() suspends until the consumer drains one — the producer's effective rate becomes exactly the consumer's rate, with no unbounded growth anywhere.

#A pull-based (Reactive Streams) example

java.util.concurrent.Flow (the JDK's built-in Reactive Streams interfaces) makes the "consumer asks for n" contract explicit:

import java.util.concurrent.Flow.*;
import java.util.concurrent.SubmissionPublisher;

class ThrottledSubscriber implements Subscriber<WorkItem> {
    private Subscription subscription;
    private static final int BATCH = 10;

    @Override
    public void onSubscribe(Subscription subscription) {
        this.subscription = subscription;
        subscription.request(BATCH); // ask for the first 10, no more
    }

    @Override
    public void onNext(WorkItem item) {
        process(item);
        // Only ask for one more once this one is done -- the publisher
        // physically cannot send faster than we consume.
        subscription.request(1);
    }

    @Override
    public void onError(Throwable t) { /* handle */ }

    @Override
    public void onComplete() { /* handle */ }

    private void process(WorkItem item) {
        // slow work here
    }
}

Here the SubmissionPublisher is contractually forbidden from calling onNext more times than were requested via request(n) — the flow control is baked into the protocol itself rather than bolted on with a queue.

🧠 Check your understanding
0/2 · 0/2 answered
1. What is the main risk of using an unbounded in-memory queue between a producer and a slow consumer?
The queue will deadlock immediatelyThe process eventually runs out of memory or latency becomes unbounded, causing a failure at the worst possible timeUnbounded queues automatically apply rate limitingIt has no risk as long as the consumer eventually catches up
2. In the Reactive Streams (java.util.concurrent.Flow) model, how does the subscriber control the rate of items it receives?
By throwing an exception when overwhelmedBy calling subscription.request(n) to explicitly ask for n more items before the publisher may send themBy setting a fixed delay before onSubscribe is calledBy closing and reopening the subscription