How do you implement a graceful shutdown that drains in-flight requests instead of dropping them?

#Why this matters every single deploy

Every rolling deploy, every autoscaler scale-down, and every pod eviction ends the same way for an individual instance: the orchestrator sends a termination signal (SIGTERM on any POSIX system, which is what Kubernetes?, ECS, and systemd all use) and gives the process a grace period before following up with an unconditional, unhandleable SIGKILL. Kubernetes defaults that grace period to 30 seconds via terminationGracePeriodSeconds. What the process does in that window determines whether the requests it was actively handling finish successfully or get abruptly cut off mid-response.

A process with no shutdown handling either dies the instant the default signal action fires (many runtimes terminate immediately on SIGTERM if nothing intercepts it) or, if it does intercept it but does nothing useful, keeps accepting brand-new connections right up until SIGKILL — at which point everything in flight, new and old, is dropped with no chance to finish or even flush a log line. Both are bad; graceful shutdown means neither happens.

#The sequence a correct shutdown follows

1. Catch the signal. Register a handler for SIGTERM (and typically SIGINT for local Ctrl+C during development).
2. Stop accepting new work immediately, while continuing to serve requests already in flight. Concretely: flip the readiness probe to failing right away, and stop the listening socket from accepting new connections — most HTTP server libraries expose a "close" or "shutdown" call that does exactly this (stop listening, let existing connections finish).
3. Wait for in-flight requests to actually finish, not just for the listener to close. This is the step people skip, and it is the one that actually prevents dropped responses.
4. Enforce a hard deadline. If requests have not drained by some safety margin before the orchestrator's grace period expires, log it and exit anyway — hanging forever just means you get SIGKILLed with even less warning and no chance to flush final logs or metrics.
5. Close downstream resources in order after requests have drained: stop background workers/consumers, close database connection pools, flush any buffered telemetry, then exit with status 0.

There is a subtle but important race worth naming explicitly: in Kubernetes, the moment SIGTERM is sent to the pod is not perfectly synchronized with the moment the pod is removed from every Service's endpoint list across the cluster (that update has to propagate through kube-proxy / the mesh's data plane). If the app stops accepting connections the instant it gets SIGTERM, a small window exists where the load balancer may still route a request to it and get a connection refused. The common mitigation is a short preStop hook that just sleeps for a few seconds before the container actually receives SIGTERM (or before the app acts on it), giving the endpoint removal time to propagate while the pod is still accepting traffic normally.

#A real implementation

Go's standard library models this almost exactly as designed above, via http.Server.Shutdown, which stops the listener and blocks until active connections finish (or the passed context is cancelled):

Wrap long lines
package main

import (
	"context"
	"log"
	"net/http"
	"os"
	"os/signal"
	"syscall"
	"time"
)

func main() {
	mux := http.NewServeMux()
	mux.HandleFunc("/healthz", healthzHandler)
	mux.HandleFunc("/orders", ordersHandler)

	srv := &http.Server{Addr: ":8080", Handler: mux}

	go func() {
		if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
			log.Fatalf("listen error: %v", err)
		}
	}()

	// Block until we receive a termination signal.
	sigCh := make(chan os.Signal, 1)
	signal.Notify(sigCh, syscall.SIGTERM, syscall.SIGINT)
	<-sigCh
	log.Println("shutdown signal received: no longer accepting new connections")

	// Flip readiness to failing here (omitted: a shared atomic bool
	// read by the /healthz handler) so the load balancer stops
	// routing new traffic even before the listener fully closes.

	// Give in-flight requests up to 25s to finish (leaving margin
	// under Kubernetes' default 30s terminationGracePeriodSeconds).
	ctx, cancel := context.WithTimeout(context.Background(), 25*time.Second)
	defer cancel()

	if err := srv.Shutdown(ctx); err != nil {
		log.Printf("forced shutdown after timeout, some requests may have been cut off: %v", err)
	} else {
		log.Println("all in-flight requests drained cleanly")
	}

	closeDatabasePool()
	closeBackgroundWorkers()
	log.Println("shutdown complete")
}

srv.Shutdown(ctx) closes the listener immediately (no new connections accepted), then waits for every connection currently handling a request to become idle, returning as soon as that happens or when the context deadline is hit — whichever comes first. That is the entire "drain" step in one call, which is why leaning on the standard library's shutdown primitive (rather than hand-rolling connection tracking) is almost always the right implementation choice when the runtime provides one.

🧠 Check your understanding
0/2 · 0/2 answered
1. A Kubernetes pod is being terminated during a rolling deploy. What is the correct order of steps for a graceful shutdown inside the app?
Ignore SIGTERM entirely and rely on SIGKILL after the grace periodStop accepting new connections and fail readiness, let in-flight requests finish (bounded by a timeout), then close downstream resources and exitClose the database pool first to stop new writes, then stop accepting connections, then exitExit immediately on SIGTERM to free resources as fast as possible, then let the orchestrator route around it
2. Why is a hard timeout on the drain step necessary even though the goal is to let in-flight requests finish?
It reduces CPU usage during shutdownWithout it, one stuck request can hold up shutdown until SIGKILL forcibly and uncleanly terminates the process anywayIt is required by the HTTP/1.1 specificationIt prevents the readiness probe from failing