Messaging & realtime · Advanced

Slack-style team chat

Channel servers on a consistent-hash ring, regional gateway servers and an edge cache for boot.

Every part, and why it is there

Slack clients

Desktop, web and mobile apps. On startup they need a lot of workspace data (channels, members, unread counts); after that they mostly receive small realtime events.

Load balancer

Routes API calls to the webapp, boot requests to the edge cache, and WebSocket upgrades to the nearest gateway servers.

Gateway servers

Hold each client’s WebSocket and remember which channels it subscribes to. They run in several regions so users connect nearby, and a failing region can be drained to the next closest.

Presence servers

Track who is online. Clients only subscribe to presence for people visible on screen, which keeps the update volume sane in a 50,000-person workspace.

Edge cache (Flannel)

Keeps workspace data warm near users and serves it lazily, so a client boots with a small payload and fetches the rest on demand. Before this, large teams could stall on a huge startup response.

Webapp API

Where a posted message first lands: checks permissions, writes it to the database, then hands it to the realtime path. Persisting first means nobody sees a message that could later vanish.

Vitess (sharded MySQL)

Messages, channels and users, sharded behind Vitess so the application talks to it like one big MySQL.

Admin servers

Stateless routers between the webapp and the channel servers: they look up which channel server owns a channel on the consistent-hash ring.

Channel servers

Stateful, in-memory owners of recent channel history, each responsible for a slice of channels chosen by consistent hashing. They send each new message to every gateway server subscribed to that channel. A failed one is replaced in seconds and only its slice is affected.

Job queue

Work that should not slow down sending: link unfurls, search indexing, notifications, integrations. Spikes queue up rather than slowing the API.

Job workers

Drain the queue and do the slow work, retrying failures. Scaled independently of the web tier.

Search index

Full-text search over messages and files, kept up to date asynchronously by the workers. A few seconds of lag is fine for search.

File storage

Uploaded files and images. Messages only reference them.

Asked in interviews at

  • openai
  • coinbase

Concepts used here

  • WebSockets: Turn one HTTP request into a pipe that stays open, so the server can push a chat message the moment it arrives. Then pay for it: memory per user, heartbeats, reconnects and pub/sub.
  • Consistent hashing: Why adding one cache server can take a database down, and the ring that makes it move only about 1 key in 5 instead of 4 in 5.

Learn more

All system design examples