A field manual for the day a famous person joined your app
The Feed.
You press POST once. Somewhere, the system decides who pays — for a star with ten million followers, the difference can be one write or ten million. You will post as a nobody and a star, refresh a timeline that follows too many people, then tune a hybrid model against explicit workload assumptions.
one post, top. the write path, below. nobody drew it in the wireframe.
BEGINPosting is never just posting.
You press POST, the API returns 200, and the sentence “my post is one write” feels like physics. It is a choice — the quietest one in the whole system. In the design most apps start with, that post is one write per follower: a row appended to every follower’s timeline, so that reading the feed later is a single precomputed fetch. The path exists, runs at full cost, and appears in no wireframe. It stays invisible until the day someone famous joins — and then it becomes a river.
Because follow graphs are a power law. The median user has a few hundred followers; the tail has ten million. The average is a lie — you heard that in The Cache — and here the tail is not a rounding error. The tail is the whole design problem.
The average is a lie. The tail is the system.A LESSON RECYCLED FROM THE CACHE — NOW WITH A BODY COUNT
Every timeline must answer one question: who pays, and when — the writer, once, or every reader, forever? Push pays at write time. Pull pays at read time. The third answer is a compromise with a cutoff. This is the “design Twitter” interview, verbatim — and by the end of this page you will have designed it, with numbers.
The path wakes up.
A push world: posting appends one row to each follower’s timeline. Post as everyone below — including AURORA, a pop star whose audience size is yours to set with the slider.
Read the scoreboard like a bill of lading. Your post: 412 writes — one per follower, set by who you are, not what you said. Then set AURORA to 10M and press her button: ten million writes for one post, minutes of fan-out before her last follower sees it. And notice the quiet line underneath — ~62% of those writes land on timelines nobody has opened this week. The zombie followers are the push world’s dirty secret: fame is mostly write-amplified into the void.
And the visibility skew is The Lag, wearing a feed: the first follower sees the post in under a second, the last one minutes later — on a system with zero errors, green everywhere.
MODEL NOTES — fan-out capacity 50K writes/s · zombie share 62% (modeled after published social-graph activity numbers) · one appended row per follower, no dedup, no batching shown · bars are log-scaled.
Rent, not mortgage.
Flip it. In a pull world, your post is one write into your own outbox — and nobody’s write path notices. Instead, every reader pays: to build a timeline you fetch from everyone you follow, merge the results, rank them, and serve. The write bill evaporates; the read bill is permanent.
Push is a mortgage: one enormous payment, then the timeline is yours. Pull is rent: the payment never ends — it just hides inside every refresh.THE ONLY TIMELINE EXCHANGE THAT MATTERS
And a follow is a permanent read tax: click FOLLOW on one more account and your refresh bill grows by one fetch — not once, but on every refresh, forever. The question the next figure asks: what does that rent cost when you follow five thousand accounts?
The fan-in.
FIG. 01’s arrows flew outward — one post to many timelines. Watch them come back: a pull refresh is a fan-in, hundreds of fetches converging on one merge. Drag FOLLOWING, press REFRESH, and watch your own patience become the bill.
Now set the two bills side by side and the whole design space appears. The celebrity’s post: push charges her 10M writes; pull charges every one of her followers a slice of rent at every refresh. The ordinary user’s refresh: push makes it one precomputed read; pull makes it a merge of everything they follow. Each config is cheap exactly where the other explodes. Which brings us to the third answer.
MODEL NOTES — 16 fetch lanes in parallel · 5 ms per source fetch · rank ~0.1 ms/item · animation is illustrative; the millisecond number is the model.
Now you design it.
Three configurations exist: push everyone, pull everyone, or a hybrid — push accounts below a follower cutoff and fetch posts from above-cutoff accounts at read time. Tune that cutoff for this model’s fixed workload: two celebrity posts, one ordinary post, and a refresh weighted ×1000. Change the read/write mix or who the reader follows and the best cutoff can move.
For this workload, a cutoff between DEV (900 followers) and NOVA (2M) makes a low-cost plateau: ordinary posts push, celebrity posts stay in their outboxes, and this reader merges two live sources. Move above NOVA and her post costs 2M writes. Move below DEV and this reader gains a third pull source. The merge bill depends on whom this reader follows, not how many followers a celebrity has.
Raffi Krikorian described a related hybrid timeline approach at Twitter in 2013: fan out ordinary accounts at write time and handle high-follower accounts at read time. It is a historical design example, not a claim about Twitter’s current implementation or a universal cutoff.
MODEL NOTES — fixed scenario weights: one post to 1,000 refreshes · this reader follows 500 accounts, including DEV and two celebrities · fan-out capacity 50K writes/s · refresh uses 16 lanes, 5 ms/fetch, and rank 0.1 ms/item. The score is illustrative for these inputs.
The field guide.
- Fan-out-on-write
- Push: append to every follower’s timeline at post time. Reads are precomputed and instant; the writer pays O(followers) — divine for the median, fatal for the tail.
- Fan-out-on-read
- Pull: one write at post time; each reader fetches and merges everything they follow. Writes are free; every follow is a permanent read tax.
- Hybrid timeline
- Push below a follower cutoff; merge above-threshold accounts at read time. The merge cost grows with the number of such accounts this reader follows, which can itself be large; it does not grow directly with a celebrity’s follower count.
- Write amplification
- One logical write becoming N physical ones. The push world’s tax rate: your follower count. You met its cousin in The Cache — one miss becoming N origin trips.
- Zombie followers
- Accounts that never read. In push they are writes into the void — the reason production systems skip inactive users and repair their timelines on return.
- Hot key
- The celebrity’s post row is The Ring’s hot key with a shadow: hashing spreads keys, not fame. Replicate the read path or cache the object — never let one row carry a nation.
- Follow/unfollow storms
- Unfollowing must remove (or tombstone-hide) rows already pushed to timelines. In push worlds, edge changes are write storms too — budget for them.
- Backpressure
- The fan-out pipeline is queues and worker pools; when a star posts, queues bloom. Shed, batch, or fall back to pull — The Ack’s retries and The Cache’s load shedding, on a stage this size.
WHY THE MERGE COST IS BOUNDED BY WHO YOU FOLLOW — NOT WHO FOLLOWS YOU
The hybrid’s read-time work is: fetch your precomputed timeline (1 op) plus a live fetch from every account you follow that sits above the cutoff.
An above-cutoff account is rare among all accounts, but a particular reader can still follow many of them. Read-time merge work scales with that reader’s above-cutoff follows, regardless of each author’s follower count.
That asymmetry is the whole trick: the celebrity’s fame is other people’s problem in a push world, and in the hybrid it becomes nobody’s — her post is one write, her readers pay one cached fetch each, and the load balancer never notices she exists. Compare with FIG. 01: the same post, ten million writes apart.
Three scenarios.
From these figures, you can estimate who pays for fan-out under a given follower and read/write workload. Try changing one assumption and check whether your explanation still holds.
The sealed sheetThree questions are sealed inside this sheet. Nobody is asked to open it — your timeline will wait.Break the seal
Pure push. A celebrity with 100M followers posts once. What happens?