Messaging & realtime · Advanced

WhatsApp-style messaging

Store-and-forward relay with delivery receipts, E2E key distribution and out-of-band media.

Every part, and why it is there

Phones

Each phone keeps one persistent connection open while the app is active. The phone, not the server, is the long-term home of the chat history.

TCP load balancer

Balances long-lived TCP connections at layer 4. An HTTP-level balancer would add cost per message for no benefit, since messages travel over an already-open socket.

Chat servers (Erlang)

Hold millions of connections each and route messages: look up which server the recipient is connected to and forward it there, or park it in the offline mailbox. Erlang’s lightweight processes make one process per connection practical.

Session registry

Maps each user to the chat server holding their connection right now. Without it a server would have to ask every other server where the recipient is.

Offline mailbox

Messages for phones that are offline wait here and are deleted as soon as the phone acknowledges them. The server is a relay, not an archive, which keeps storage small and limits what could ever leak.

Offline push jobs

When a recipient is offline, a job is queued to wake their phone. Push providers are slow and rate-limited, so this must not block message routing.

Push service

Looks up the device token and sends a minimal, content-free push that tells the app to connect and fetch its messages.

APNs / FCM

The only way to wake a sleeping phone. Apple and Google run these; you just hand them a token and a payload.

Key distribution

Hands out each user’s public identity key and one-time “prekeys” so a sender can start an encrypted session even while the recipient is offline (the Signal protocol).

Identity & prekeys

Public keys only; private keys never leave the phone. Losing this would stop new conversations from starting, so it is replicated.

Media upload

Accepts an already-encrypted photo or voice note and returns a URL. The chat message then carries only that URL plus the decryption key.

Encrypted media

Encrypted blobs the server cannot read. Uploaded once even when forwarded to a group of 200, because every member gets the same URL and key.

Media CDN

Recipients download media from nearby edges. Group media is fetched by many people within minutes, which makes it very cacheable.

Asked in interviews at

  • meta
  • amazon

Concepts used here

  • WebSockets: Turn one HTTP request into a pipe that stays open, so the server can push a chat message the moment it arrives. Then pay for it: memory per user, heartbeats, reconnects and pub/sub.

Learn more

All system design examples