Fundamentals · Intermediate

Unique ID generator

Snowflake-style, time-sortable 64-bit ids generated on many machines without coordination.

Every part, and why it is there

Mobile app

Creates orders, posts or messages, each needing a unique id the moment it is saved.

Web app

Same as mobile. The point is that many writers across many servers create rows at the same instant.

API gateway

Routes create requests to the order service. Nothing special for ids, but it shows the fan-in of traffic from many clients.

Order service

Needs an id before it writes a row. With a sharded database there is no single auto-increment counter to ask, so ids must come from somewhere else.

ID generators

Each instance builds an id from the current millisecond, its own worker number and a per-millisecond counter. No instance ever talks to another, so it is fast and has no single point of failure. Often it is a library inside the service instead of a network call.

Worker-id registry (ZooKeeper)

Gives each generator a unique worker number at startup and takes it back when it dies. Two generators with the same number could produce duplicate ids.

Sharded orders DB

Rows are spread over many shards. Because ids start with a timestamp, newer rows have bigger ids, so indexes stay append-mostly and “latest first” queries are a simple sort.

Order events

Downstream consumers get events keyed by the same id, so everyone refers to one record the same way.

Search indexer

Consumes events and indexes them. Ids that sort by time let it resume from “last id seen” without a separate timestamp column.

Order search

Can sort and paginate by id to get chronological order cheaply.

Asked in interviews at

  • microsoft
  • netflix
  • clickhouse
  • snowflake

Concepts used here

  • Sharding: Split one table across many database machines. It buys room and speed, and makes every query and transaction that crosses machines harder.
  • Bit tricks and bitmasks: Store “which cities have I visited?” as one number, and the shortest round trip through 20 stops drops from years of work to under a second.

Learn more

All system design examples