Design a unique ID generator

Sortable, 64-bit IDs across thousands of machines with no coordination - the Snowflake layout of timestamp, machine ID and sequence bits.

9 min read

Every row in every table needs an identifier. Getting one is trivial with a single database and surprisingly hard the moment there are two, because the properties you want, unique, sortable by time and cheap to mint, pull against each other.

Auto-increment works until there are two databases

One database hands out 1, 2, 3, and the problem is solved. Split it into two shards and each half keeps counting on its own, which means both of them issue ID 1.

Row 1 on shard A1duplicate
Row 2 on shard B1duplicate
Row 3 on shard A2duplicate
Row 4 on shard B2duplicate
Row 5 on shard A3duplicate
Row 6 on shard B3duplicate
Duplicates
3
Sorts by time
yes
Coordination
needed
Two shards counting on their own both issue 1, 2 and 3. Switch the scheme and the same six inserts get different IDs.

The obvious escapes each give something up. A random UUID never collides, but it sorts into nonsense, and that wrecks index locality: every insert lands in a different page of the B-tree. A central ticket server hands out perfectly ordered numbers, and puts a single point of failure on every write in the system. Uniqueness without coordination is easy, and so is sortability. Getting both at once is the design problem.

Put time in the high bits and identity in the middle

Snowflake builds both properties into the bits of a 64-bit number: a sign bit, a 41-bit millisecond timestamp, 5 bits of datacenter, 5 of machine, and a 12-bit counter for IDs within the same millisecond. Because the timestamp is the most significant part, a later ID is always a larger number.

signtimestamp, 41 bitsdatacenter, 5machine, 5sequence, 12
As a number
1925877702457212931
7
12
3
+0 ms
Press 'Next ID, same millisecond' and only the last twelve bits change. Move the clock and the high bits change, so the number grows.

Each field prevents one kind of collision. Time separates milliseconds. The machine and datacenter bits, assigned once at startup and never shared, separate two machines minting in the same millisecond. The sequence separates IDs within one machine's millisecond, up to 4,096 of them. Nobody coordinates anything at mint time. The 41 timestamp bits last about 69 years from the chosen epoch, which is a deadline rather than a guarantee.

It depends on a clock that only moves forwards

All of that rests on two assumptions: time never goes backwards, and no machine needs more than 4,096 IDs in a millisecond. Production breaks both, and the failures are very different.

Clock
Moving forward
Every new timestamp is later than the last, so every ID is larger than the one before.
Sequence
4,096 of 6,000 served
12 bits allow 4,096 IDs per machine per millisecond. The other 1,904 wait for the next millisecond: a latency blip, not corruption.
This millisecond's requests: teal issued, amber waiting, red unsafe to issue at all
6,000
Too many IDs in a millisecond is a tuning problem. A clock going backwards is an outage, by design.

When an NTP correction steps the clock back, carrying on is the dangerous choice: the generator would reuse timestamps it has already used and mint IDs that already exist. A duplicate primary key is a corruption bug that surfaces weeks later, so real implementations remember the last timestamp and simply refuse to issue until the clock passes it. A brief write pause on one node is the far better failure.

The modern alternatives are the same idea re-standardised. ULID and UUIDv7 put a timestamp in the high bits and randomness below, trading machine IDs for enough random bits that collisions are negligible. The choice is not about elegance; it is about which failure you can live with.

The short version

  • Per-shard counters collide; random UUIDs do not sort; ticket servers are a single point of failure.
  • Snowflake packs timestamp, machine and sequence into 64 bits: unique and time-sortable, no coordination.
  • 4,096 IDs per machine per millisecond; beyond that, requests wait a millisecond.
  • If the clock steps back, block rather than risk duplicate IDs.