Design a background-sync engine that reliably ships local writes to the server

#Clarify the requirements

The first question is what kind of writes. A single-writer note-taking app has a very different conflict surface than a shared shopping list edited from two devices at once. I'd also ask: is offline-first mandatory (must every write succeed instantly from the user's point of view, even mid-flight to Antarctica with no signal), does write order matter within a record, how fresh must a peer device's view be after a sync, and do we need to support deletes as first-class operations (a delete that arrives after a concurrent edit is a classic conflict case). For this design I'll assume: offline-first is mandatory, most conflicts are same-user-multiple-devices rather than true multi-user collaboration, and eventual consistency within seconds-to-minutes is acceptable once connectivity returns.

#The core pattern: transactional outbox

The foundation is the outbox pattern. Every local mutation — create, update, delete — is written to the local database and appended to an outbox table inside the same local transaction. This guarantees that a write the user believes succeeded can never be silently lost, even if the app crashes or is killed by the OS a millisecond later: either both writes commit, or neither does. The outbox is then drained by a background worker (WorkManager on Android, BGTaskScheduler / BackgroundTasks on iOS) that is independent of whether the user is still looking at the screen.

#Retry policy: classify before you retry

Not every failure deserves the same response. A 5xx, a timeout, or a dropped connection is retryable — the mutation is still valid, the world just needs to try again, so it goes back to pending with exponential backoff plus jitter (jitter matters at scale: without it, every device that lost connectivity during the same outage retries in lockstep and hammers the server the moment it recovers). A 4xx validation error is terminal — retrying an invalid payload forever just wastes battery and never succeeds, so after a small number of attempts it moves to a dead-letter state that surfaces to the user or gets logged for investigation. A 409 version conflict is neither: it means the mutation was structurally fine but the server's state moved since the client last saw it, and that's a distinct third path, conflict resolution, covered next.

#Conflict resolution: pick the cheapest strategy that's still correct

Most apps don't need CRDTs, and reaching for one by default is a common interview trap. The right approach is a ladder, from simplest to most complex:

• Last-write-wins (LWW), keyed by a server timestamp or a monotonically increasing version. Cheap, simple, and correct enough when concurrent edits to the same field are rare — the tradeoff is that the loser's write silently disappears, so it's a poor fit for anything the user would consider "my data just vanished."
• Field-level merge, where the server diffs incoming and existing state per field rather than per record, so two devices editing different fields of the same object both survive. This covers a large share of real "conflicts" that are actually non-overlapping edits.
• CRDTs or operational transforms, reserved for genuinely collaborative, high-concurrency surfaces (shared documents, shared lists edited live by multiple people) where losing a write is unacceptable and merges must be automatic and mathematically convergent regardless of arrival order.
• Surface to the user, the last resort when automatic merge can't safely decide — show both versions and let a human pick, rather than silently guessing wrong on something important.

#Ordering, idempotency, and pulling changes back down

Two more guarantees make the queue trustworthy. First, every mutation carries a client-generated idempotency? key (a UUID), so if a request is retried after a timeout where the server actually processed it, the server can detect the duplicate and return the original result instead of applying it twice. Second, mutations to the same record are drained in the order they were enqueued (a per-record sequence number), while mutations to different records can be sent in parallel — you want ordering guarantees exactly where they matter and no broader than that, or you sacrifice throughput for no benefit.

Sync isn't just outbound. The engine also needs to pull server-side changes down through a change feed addressed by a cursor or "since" token, rather than re-downloading the entire dataset on every sync. Incoming remote deltas must be reconciled against anything still sitting in the local outbox — a remote update to a record with a pending local mutation is exactly the 409 case above, just discovered on the pull side instead of the push side.

#Operational concerns that decide whether this survives contact with real users

Battery and network constraints shape scheduling: bulk syncs run under WorkManager constraints like unmetered network or charging, while a user-triggered save gets a best-effort immediate attempt regardless of those constraints, because the user is watching. Observability matters as much as correctness — expose queue depth, error rates, and a simple sync-status indicator in the UI, because "my changes didn't save" is one of the hardest bug reports to act on without that telemetry already in place. And multi-account apps need every mutation tagged with the account it belongs to, with the outbox flushed or discarded — never silently sent — on logout, so one user's pending writes can never leak into another user's session on a shared device.