Networking · 15 min

WebSockets

Turn one HTTP request into a pipe that stays open, so the server can push a chat message the moment it arrives. Then pay for it: memory per user, heartbeats, reconnects and pub/sub.

Aiko and Ben are chatting in your web app. Aiko starts typing, and Ben should see “Aiko is typing…” straight away. When she presses Send, the message should land in Ben’s window before he looks away.

Here is the snag. HTTP, the language browsers speak to servers, only works one way round: the browser asks, the server answers. The server cannot speak first. So how does it tell Ben that Aiko said something?

This article tries the obvious fix, then a better one, then the real one, and ends with what the real one costs you.

Attempt 1: ask again every second

The obvious way: Ben’s browser asks the server “anything new?” once a second. This is called short polling. Most answers are an empty list, written “[]”. Now and then the answer holds Aiko’s message.

In the demo below, one “anything new?” request carries 403 bytes of headers and its answer carries 147 more. That is about 550 bytes to hear “[]”, which is 2 bytes. Real sites often send more, because their cookies are longer: 500 to 800 bytes a round trip is normal.

It is also slow. A message that arrives just after Ben’s browser asked waits almost the whole second for the next ask; one that arrives just before waits almost nothing. Messages land at any moment in the gap, so on average they wait half of it: 0.5 s if you poll every second, 1.5 s every 3 seconds. Polling faster cuts the wait and multiplies the requests.

Attempt 2: ask once and let the server wait

The better trick is long polling. Ben’s browser asks “anything new?”, and the server does not answer yet. It holds the request open until it has something to say, or until about 30 seconds pass. When Aiko’s message comes in, the server answers at once, and Ben’s browser immediately asks again.

The waiting is gone: Ben sees the message one network trip after the server gets it. A trip is the time bytes take to cross the network one way; the demo uses 50 ms. Quiet time is cheap too: one request per browser every 30 seconds.

But a chat with a typing hint is not quiet. Every event still costs a full HTTP request with its headers. When Aiko starts typing, her browser sends a POST (a request that carries data) with about 450 bytes of headers to deliver 31 bytes of “typing”. Then the server answers Ben’s held request (147 bytes of headers), and Ben’s browser sends a fresh request (403 more). Add the answer to her POST (148 bytes): 450 + 148 + 147 + 403 is about 1,100 bytes of headers to move one typing hint.

And there is a gap. After each answer, for two network trips (the answer travelling to Ben, then his new request travelling back), Ben has no request waiting at the server. A message that lands in that gap waits for the next request. Fast chats hit the gap often.

Notice one more thing: a held request is already a connection sitting open on the server, waiting. Long polling pays most of the cost of keeping a line open and still wraps every message in HTTP headers.

Attempt 3: turn the request into a pipe

The connection under HTTP can already carry bytes both ways. It is only HTTP’s rules that make the server wait to be asked. So WebSocket keeps the connection and drops the rules.

It starts as a normal HTTP request that asks to switch. This is the upgrade:

GET /chat HTTP/1.1
Host: chat.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==
Sec-WebSocket-Version: 13

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: s3pPLMBiTxaQ9kYGzzhZRbK+xOo=

The key is 16 random bytes, written as letters and digits (base64). The server glues a fixed string from the standard onto it, 258EAFA5-E914-47DA-95CA-C5AB0DC85B11, runs SHA-1 on the result, and sends the base64 of that back as Sec-WebSocket-Accept. SHA-1 is a hash function: it turns any text into a fixed 20-byte fingerprint.

This is not a password. It proves the server really understood the WebSocket request. Without it, an old web server, or a proxy (a machine in the middle, such as an office web filter, that passes HTTP along), could send back some ordinary answer and the browser would wrongly think the switch had worked.

After the 101, nobody speaks HTTP on that connection again. Either side sends a frame whenever it likes.

Aiko’s message goes as JSON: {"type":"message","from":"aiko","text":"Ramen at 7?"}, which is 53 bytes. From the server to Ben it travels with a 2-byte frame header. From Aiko’s browser to the server the header is 6 bytes: browsers must scramble (mask) what they send with a fresh random 4-byte key, and the key rides in the header. It is a safety rule for old proxies that might misread the bytes; you never handle it yourself. Compare 2 to 6 bytes with about 550 bytes of headers per poll.

In the browser it is a few lines: new WebSocket("wss://chat.example.com/chat") opens it, socket.send(text) sends, and an onmessage function runs for every frame that arrives. wss:// is WebSocket over the same encryption HTTPS uses.

A pulled cable makes no sound

When Aiko’s laptop drops off Wi-Fi, or her phone goes into a tunnel, nothing says goodbye. No close frame is sent. Her browser and the server both go on thinking the connection is open. TCP by itself may not notice for hours: TCP has an optional check for dead lines, but on Linux it is off unless the program turns it on, and even then it first runs after 2 hours of silence.

Meanwhile Ben’s messages vanish into a dead line, and Aiko stares at a chat that looks fine.

The WebSocket standard has ping and pong frames, and browsers answer a server’s ping on their own. That lets the server notice a dead Aiko. It doesn’t help Aiko’s app notice a dead server, because browser code cannot send a ping frame or see one arrive. So the app sends its own “ping” text message and expects “pong” back. That costs 16 bytes a round trip: 4 bytes of “ping” plus a 6-byte header, then 4 bytes of “pong” plus a 2-byte header.

In the demo, Aiko’s app pings every 5 s and gives up after 3 s without a pong, so a dead line is noticed within 8 s. Real apps ping every 20 to 30 s, so this takes longer.

Then it must reconnect, and how it retries matters. If your chat server restarts, 50,000 browsers lose their line in the same second. If they all retry every second, the server comes back to 50,000 handshakes a second and falls over again.

So clients use backoff: wait 1 s, then 2, 4, 8, 16, doubling each time up to a cap such as 30 s. Real clients also add a little randomness to each wait (jitter), so 50,000 browsers spread out instead of retrying in step.

Here is the client’s side in pseudocode. Look for three things: the heartbeat, the doubling wait, and the catch-up request after every reconnect.

attempt = 0

connect():
    socket = new WebSocket("wss://chat.example.com/chat")
    socket.onopen = () =>
        attempt = 0
        send({ type: "since", after: lastSeenId })   # id of the newest message shown
        every 25 s: send("ping"); no "pong" in 3 s → socket.close()
    socket.onclose = () =>
        wait = min(30 s, 1 s × 2^attempt) + a little randomness
        attempt = attempt + 1
        after wait: connect()

The catch-up matters as much as the reconnect. Anything sent while the line was dead is gone from that connection. Each message needs an id, so after reconnecting Aiko’s app can ask “everything after message 41, please”. Messages she typed while offline wait in an outbox until the server sends back their ids to say it has them.

Every open socket lives on one server

An HTTP request touches a server for a few milliseconds and leaves. A WebSocket stays. That changes three things about running it.

First, memory. Each open socket holds buffers (memory the operating system sets aside for bytes waiting to be sent or read) plus your app’s state for that user. Say 20 KB each; the real figure depends on your server and language. 100,000 people online is 100,000 × 20 KB = 2 GB, held all day, even while nobody types. It can scale a long way: in 2012 WhatsApp wrote about holding over 2 million connections on one server.

Second, everything between the browser and the server must let the line stay open and quiet. A load balancer is the server that spreads incoming connections across your chat servers (the load balancing page covers it). Many close quiet connections: an AWS Application Load Balancer closes one after 60 s of silence by default, and the nginx web server does the same unless you raise its timeout setting. Heartbeats under 60 s keep them open. nginx must also be told to pass the Upgrade header on to your chat server; by default it drops it, and the switch never happens.

Third, that way between servers. The usual answer is pub/sub, short for publish and subscribe. Each server subscribes to the chat rooms its users are in. Server 1 publishes Aiko’s message to the channel “room:aiko-ben”. Every server subscribed to that channel gets a copy, so server 2 receives it and writes it into Ben’s socket. Redis Pub/Sub is a common choice; WhatsApp and Slack each run their own versions of this layer.

Pick the smallest tool that does the job

WebSockets are the right tool when both sides talk often. If only the server needs to talk, Server-Sent Events (SSE) is simpler: the server sends the headers of an HTTP answer but never finishes it, adding a line each time there is news, and the browser reconnects by itself. It has its own page.

Aiko and Ben’s chat could even mix them: send with plain POSTs and receive over SSE. That works well when messages are rare. The typing hint is what tips it: both browsers fire one every few seconds, so a pipe that runs both ways pays off.

The chat at 100,000 people

Put it all together for 100,000 people with the chat open.

• Short polling every second: 100,000 requests a second, about 100,000 × 550 B = 55 MB of headers a second, nearly all answering “[]”, and messages still up to a second late.
• Long polling: no waiting, but about 1,100 bytes of headers for every message and typing hint, and 100,000 held requests are open connections anyway.
• WebSockets: one handshake per user and then 2 to 6 bytes of header per message. In exchange, about 2 GB of memory for sockets, a heartbeat every 25 s (100,000 ÷ 25 = 4,000 tiny pings a second), backoff on reconnect, and pub/sub so server 1 can reach server 2.

Now you can see why chat apps nearly all land on WebSockets, and why the hard part is not the upgrade. It is everything that keeps 100,000 quiet pipes alive and connected to each other.