Design a file storage service like S3 or Dropbox — walk through the architecture

#Separate "what a file is" from "where its bytes live"

The single most important design decision in this question is splitting the system into two planes that scale completely differently: a metadata plane (the file/folder tree, filenames, permissions, version history — small, relational-shaped, needs strong consistency) and a data plane (the actual bytes — huge, immutable once written, needs to scale to exabytes and doesn't care about relationships). Conflating them — storing file bytes inside the same database that tracks folder structure — is the mistake that sinks most attempts at this problem in the first five minutes.

Requirements worth stating out loud before designing further:

• Files can be huge (multi-GB) and uploads must be resumable.
• Many files (or parts of files) are duplicates across users — dedup should be automatic and content-based, not filename-based.
• Durability is the headline metric — "eleven nines" isn't a marketing number, it reflects a real replication/erasure-coding strategy.
• Sync clients need a consistent view of "what changed since I last looked," and conflicting concurrent edits need a defined resolution (not silent data loss).

#High-level architecture

#Chunking: why you cut files up before storing them

Storing a whole file as one blob makes two things expensive: resumable uploads (retry the whole multi-GB file after one dropped connection) and deduplication (two files that differ by one inserted byte look 100% different as whole-file hashes). The fix is to split every file into chunks — typically a few MB each — and store, hash, and deduplicate chunks independently.

Two chunking strategies:

• Fixed-size chunking — simplest, but a single byte inserted near the start of a file shifts every subsequent chunk boundary, so nothing after the edit dedups against the previous version.
• Content-defined chunking (CDC) — a rolling hash (Rabin fingerprinting is the classic choice) scans the byte stream and places a chunk boundary wherever the hash matches a pattern (e.g., its low N bits are zero), so boundaries are determined by content, not by position. Insert a byte anywhere and only the chunk(s) touching that byte change — everything else still dedups.

CDC costs more CPU per upload but is what makes deduplication actually pay off across versions of a file (think: syncing a large spreadsheet you tweak daily) and across different files that share large common regions (think: two people's virtual machine images built from the same base).

#Deduplication and the write path

Each chunk gets content-addressed: hash it (SHA-256 is standard), and use that hash as its identity. Before writing a chunk to the blob store, look it up in a chunk index (hash to reference count and storage location). If it already exists, just increment the reference count and move on — no bytes are written. If it's new, write it to the blob store and insert it into the index with refcount 1. A file's metadata record then becomes, at its core, an ordered list of chunk hashes — reconstructing the file is just concatenating chunks in that order.

This has an important side effect: chunks are shared and immutable, so deleting a file never deletes a chunk directly — it decrements refcounts, and a chunk's storage is only reclaimed once its refcount hits zero (usually via an async garbage-collection sweep, not synchronously on delete).

#Upload and download flow

Upload: client asks the metadata service for an upload session; the file is split into chunks client-side (or at an edge ingest service); each chunk is hashed and the client first asks "do you already have this hash?" — chunks the server already has are skipped entirely, saving bandwidth, not just storage. Remaining chunks are uploaded (in parallel, with per-chunk retry, which is what makes multi-GB uploads resumable — a dropped connection only costs you the in-flight chunk, not the whole file). Once every chunk is acknowledged, the client commits the new file version by sending the ordered chunk-hash list to the metadata service.

Download: the metadata service returns the chunk list and locations; the client (or a CDN edge in front of the blob store) fetches chunks in parallel and reassembles them in order. Frequently-accessed chunks are excellent CDN cache candidates precisely because they're immutable and content-addressed — a cache key can never go stale.

#Replication and durability

Three physical strategies, in increasing order of storage efficiency and decreasing order of simplicity:

Most production systems combine these: erasure coding within a region for cost-efficient durability against disk/node failures, plus asynchronous cross-region replication for disaster recovery against a whole region going dark.

#Consistency: strong where it's cheap, eventual where it has to be

Metadata (the file tree, permissions, "does this file exist") should be strongly consistent — a rename or delete needs to be immediately visible, or sync clients diverge. The metadata store is comparatively small, so this consistency is affordable (a well-partitioned relational or strongly-consistent key-value store handles it fine). Blob data replicated across regions, by contrast, is usually eventually consistent — waiting for a synchronous write to three continents on every upload would make the product unusable.

The sharpest edge case is concurrent edits from multiple devices while offline: two devices editing the same file without network contact will each commit a new version pointing at different chunk lists. The system can't silently merge byte content it doesn't understand, so the standard resolution is a version vector or last-writer-wins with a conflicted copy — keep both versions, surface both to the user (that's exactly where "conflicted copy" filenames in Dropbox-like products come from), and let the human resolve it.