Productivity & collaboration · Advanced

Dropbox file sync

Content-addressed 4 MB blocks, a metadata journal with cursors, and long-poll change notifications.

Every part, and why it is there

Desktop sync clients

Watch a local folder, split changed files into blocks, hash them and sync. Most of the time they just sit in a long-poll waiting to hear that something changed.

Web & mobile

Browse and preview files without syncing them. They use the same metadata API plus a preview service.

Load balancer

Splits traffic between metadata, block transfer, notifications and previews, which scale very differently: block servers move bytes, the metaserver mostly does database work.

Metaserver

Knows about users, folders and file versions but never stores file contents. POST /commit_batch says “file X is now this list of block hashes”; the reply names any blocks the server does not have yet.

Server File Journal (MySQL)

An append-only journal per folder namespace: each row is one version of one file with its block list. A client’s cursor is just the last journal id it has seen, so “what changed?” is a range scan.

Metadata cache

Caches hot namespace and journal reads so frequent listings do not all reach MySQL.

Blockserver

Stores and returns blocks by hash, knowing nothing about files or users. PUT /blocks/:hash uploads one encrypted 4 MB block; identical blocks are only ever stored once.

Block storage (Magic Pocket)

Exabytes of immutable blocks, erasure-coded across machines and regions. Dropbox moved this off S3 onto its own storage system for cost.

Notification server

Holds each idle client’s long-poll open and answers it the moment one of their folders changes. Far cheaper than every client asking “anything new?” every few seconds.

Namespace changes

Every committed change is published here so notification servers can wake the right clients and indexers can update search.

Preview service

Renders thumbnails and document previews from blocks on demand, so the web app can show a PDF without downloading the whole file.

Search indexer

Extracts file names and text and updates search after each change.

Search index

Find files by name or content across everything a user can access.

Asked in interviews at

  • meta
  • clio
  • apple
  • autodesk
  • coinbase

Concepts used here

  • Merkle tree: Two copies of a million rows find the few that differ by comparing about 41 short fingerprints, not a million rows.

Learn more

All system design examples