How does service discovery work — client-side vs. server-side discovery?

#The Problem: Nothing Stays Where You Left It

In a static deployment, "the inventory service" lives at a fixed IP forever, so you hardcode it. In a dynamic one — containers rescheduled by an orchestrator, instances autoscaling up and down, canary deploys shifting traffic — the set of healthy addresses behind "the inventory service" changes every few minutes, sometimes every few seconds. Service discovery is the mechanism that answers "which addresses can I actually send this request to, right now?" without a human editing a config file every time something restarts.

Every discovery system needs three moving parts: a service registry (the directory of who's alive and where), a registration mechanism (how an instance gets into the directory), and a health check (how a dead or unresponsive instance gets out of the directory before every client learns the hard way). The interesting design decision is who queries the registry and where the load-balancing decision gets made — and that's exactly the line between client-side and server-side discovery.

#Client-side discovery

The calling service embeds a discovery-aware client — historically a library like Netflix Ribbon paired with a Eureka registry, though the same shape shows up anywhere a client library caches the registry and load-balances itself. The instance registers itself with the registry on startup (self-registration) and sends periodic heartbeats to prove it's alive. The calling client asks the registry directly for the current list of healthy instances (often caching that list locally and refreshing it periodically rather than hitting the registry per request), then applies its own load-balancing algorithm — round robin, weighted, zone-aware, latency-based — to pick one and call it directly.

The upside is one fewer network hop and a lot of flexibility: the client can make routing decisions with information a generic load balancer doesn't have (which zone am I in, which instance served me fastest last time). The downside is that every calling service, in every language your organization uses, needs a compatible discovery client library talking to the same registry API — painful in a polyglot shop — and every client now carries discovery, caching, and load-balancing logic that has to be gotten right and kept consistent everywhere.

#Server-side discovery

The calling service doesn't know the registry exists. It calls a single, stable address — a load balancer, an API gateway, or a DNS name — and that intermediary is the one that queries the registry (or maintains its own view of it) and routes the request to a healthy instance. This is how a classic AWS setup works (clients call an Elastic Load Balancer; the ELB's target group is kept in sync with healthy ECS tasks) and, more importantly today, how Kubernetes? works by default: a client calls a Service's stable DNS name or ClusterIP, the Kubernetes control plane keeps that Service's Endpoints in sync with whichever Pods currently pass their readiness probe, and kube-proxy programs the node's networking (iptables/IPVS) to route connections to one of those healthy Pod IPs — so in practice there's often no extra user-space hop at all, just kernel-level connection routing.

The upside is that the calling client is dumb by design: no library, no language-specific dependency on the registry's API, works identically for a bash script or a fifteen-year-old Java service. Discovery and load-balancing logic lives in one place and can change without touching a single caller. The downside is that the load balancer or platform networking layer is now critical, shared infrastructure — if it's a literal proxy hop, it adds latency and becomes something you must scale and operate; if it's DNS-based, you inherit DNS's caching and TTL quirks (a resolver that ignores TTLs will keep sending traffic to an instance that's already gone).

#Self-registration vs. third-party registration

Orthogonal to client-side vs. server-side is who writes to the registry. In self-registration, the instance's own code calls the registry on startup and sends its own heartbeats — simple, but it couples your application code to the registry's API and to getting shutdown/heartbeat logic right in every service. In third-party registration, something outside the application does it: a sidecar, or the platform itself. Kubernetes is the cleanest example — your application doesn't know Kubernetes exists; the kubelet reports Pod readiness, and the control plane updates Endpoints accordingly. Third-party registration is generally preferred today because it keeps discovery out of business logic entirely.

#Picking one

Most teams building on Kubernetes today get server-side discovery for free and never think about it — which is itself evidence for how it won: it removes a whole category of per-language client complexity that the Netflix-era stack had to solve explicitly.

:::danger Stale entries are worse than no registry A registry that only removes instances via a slow TTL (e.g., "expire after 90 seconds without a heartbeat") will keep routing traffic to a dead instance for up to that whole window, on every client that cached its listing. Pair heartbeat-based expiry with active health checks and keep the cache TTL on the client (or load balancer) short relative to how fast instances actually die in your environment — a container OOM-killed by the kernel doesn't get to deregister itself gracefully. :::

:::note Service mesh muddies the line, on purpose A mesh sidecar (Envoy in Istio/Linkerd) is technically a server-side discovery point sitting next to every client, so it gets the "no per-language client library" benefit of server-side discovery while avoiding a shared, potentially bottlenecked central load balancer — each hop only goes through the local proxy. It's worth naming as a third option once a team has outgrown "pick one." :::