Design a News Feed (Twitter / X)

Half a billion posts a day, forever. Every post is written once and never changed. Nothing is ever queried by its text. Nothing is joined. Nothing is aggregated. One more thing is true of the reads, and we're assuming it rather than deriving it. Every post is fetched by its id. That's really a call the read path makes later, because it depends on what a timeline turns out to store, and it does come back the same way. So that's the shape of the workload. The volume is 6k writes a second against hundreds of terabytes of accumulated posts. So: what kind of store fits that?

The reflex answer, and it's a strong one, is that a product at this scale needs Cassandra. Give that its due, because it's not a silly instinct. Cassandra is built for exactly this shape of work: enormous volume, lots of writes, no joins, rows read by key. It scales by adding nodes, and it has no primary machine that every write funnels through. If this were a message store taking hundreds of thousands of writes a second, Cassandra would be the answer and there'd be nothing to discuss.

Now read our number. 6k writes a second. That's small. One ordinary database node handles that without noticing, and we haven't even split anything yet. Cassandra's whole bargain is that you give up joins, give up ad-hoc queries, and accept eventual consistency. A write lands on one node and reaches the others a moment later, so a read that arrives in between can miss it. In exchange you get write throughput. We're being asked to pay that price for throughput we don't need. Twitter ran its posts on Cassandra for a while and then moved off it.

The second candidate is a managed key-value store. Twitter's own is called Manhattan, and the version you can actually buy is something like DynamoDB. This one is genuinely tempting. 'Post id in, post out' is literally its API, and throughput, replication, and operations become the provider's problem instead of yours. The cost is that you commit on day one to which field decides where a row lives. From then on, anything that isn't a plain key lookup gets harder. That's a real trade, and it's mostly a question about your team rather than your workload.

The third is a relational database, split across many machines. Plain MySQL, sharded. Looking up one row by its primary key is the single thing a relational database does fastest, and it has done it for decades. 6k inserts a second is nothing to it. Its durability tooling is the most boring, most proven thing in this design. Backups, restores, point-in-time recovery. That's what you want from the box holding the only copy of everything anyone ever said.

A relational database, split across many machines (MySQL, sharded). Sharded MySQL. Here's the sentence the whole decision rests on. The write rate doesn't force a distributed NoSQL store, but the size does force sharding. Those are two different problems with two different answers. Sharding is something you do to any store. NoSQL is a product you buy. Reaching for Cassandra here is pattern-matching on the word 'scale' instead of reading the number that came with it.

So the 300 on the board comes from the size of the stored pile alone, not the write rate. We haven't counted the read traffic yet. That comes later, and it will push this box bigger. Better to know that now than be surprised by it.

Now the shard key, the field that decides which machine a row lives on. Every post is read by its post id, so shard on the post id. One thing about that id: it's time-sortable, a timestamp plus a sequence number, minted by the post service at write time. That's what lets the read path ask for 'posts older than this one' and get newest-first for free, with no separate sort key. To place a row, hash that id and the hash picks the machine. Any server computes the same machine from the id, with no router in between. Writes land flat, because consecutive posts hash to unrelated places. The tempting alternative is to shard by time, which Twitter tried and abandoned: every new post carries the same timestamp, so all the writes land on the newest machine while older ones sit idle. This all rests on reading posts by id, and the read path later settles it our way.

One more thing lives near this table. A post row never changes, but the count of likes and reposts on it changes constantly. So those counts don't belong in the post row. You'd be rewriting an immutable row thousands of times a second, and you'd lose everything that makes it easy to copy. Counts live beside the posts. They're updated asynchronously, and they're allowed to be a second or two behind. A like count that's briefly stale is fine. A missing post is not. We're not drawing that box, because it changes nothing else on this board.

You open the app. You follow about a hundred accounts. You want their recent posts, newest first, already on the screen. 300k other people want the same thing this second, and every one of them follows a different set of accounts. So no two answers are the same. Nothing here can be computed once and handed to everybody. The day-one machine answered this with one query, and that query was correct. It's still correct. So before we replace it, let's be precise about what's wrong with it. It isn't that it's slow. It's what it costs, 300k times a second. So: where does the list come from?

The first is fan-out on read, the day-one query grown up. Nothing is prepared in advance. When you open the app, we look up the hundred or so accounts you follow, ask each one for its recent posts, merge them by time, and hand you the top twenty. A post is visible the instant it's written, because there's no delivery step to wait for. Following and unfollowing take effect immediately, because the follow list is read fresh every time. And there's no extra machinery at all. It's one query against stores you already have.

Now price a single read. It's roughly a hundred lookups in the follow data, then a hundred queries to the post store, then a merge of a hundred little lists. All of that has to finish inside a five-millisecond budget, while the person stares at a spinner. Multiply by 300k reads a second, and the post store is being asked for 30M queries a second, every one on somebody's latency budget. And notice who pays the most. The cost of your read scales with how many accounts you follow, so the more you use the product, the worse it gets for you. That's a strange thing to build on purpose.

The second answer is fan-out on write. Do the work when the post is written, not when it's read. The moment somebody posts, copy that post's id into a ready-made list belonging to every one of their followers. Now a read is one lookup: fetch your list, it's already assembled, done. The cost is on the other side. One post no longer means one write. The average post goes to about seventy-five followers, so one post means about seventy-five writes, and 6k posts a second becomes about 450k timeline writes a second. You're also doing work for followers who won't open the app today, and a post isn't visible until its copies have landed.

Fan-out on write — build every list when the post is written. Fan-out on write. Pay at write time, read for free. Put the two costs next to the ratio from the brief: fifty reads for every write. Fan-out on write does about seventy-five units of work once to build the lists, then one cheap lookup on each of those fifty reads. That's seventy-five to build plus fifty to serve, about a hundred and twenty-five units all in. Fan-out on read instead does about a hundred units on every one of those fifty reads, and does it from scratch each time. Same post, same readers, about forty times more work. You get fifty chances to collect on work you did once, and that's what decides it.

How it works, following one post through. The post service stores the post, then puts one message on the fan-out queue, about 6k a second. That's the last thing it does. A fan-out worker takes the message off the queue and reads the author's follower list from the follow graph. Then it writes the post's id into each follower's list in the timeline cache. One message becomes seventy-five writes, so this is where the volume lives, at about 450k timeline writes a second. The read path is separate, and it runs whether anybody is posting or not: the timeline service fetches your ready-made list and hands it back. One lookup, and it never touches the queue or the workers.

One detail on the read path: how you get the next page. Your timeline is newest-first, and a page is twenty posts. The obvious way to get the next twenty is to skip the ones you already saw. That breaks twice. Skipping over everything you've already seen to return twenty means the work grows the further you scroll, so page ten costs ten times page one. And the list changes while you read it. New posts arrive at the top, so 'skip the first twenty' now points somewhere else, and you see one post twice and miss another. Use a cursor instead. A cursor is just the id of the last post you saw. 'Give me twenty posts older than this one' is a position in the list, not a count into it, so it costs the same on every page. The list moves; your place in it doesn't.

Why there's a queue at all. It's not for throughput. 6k messages a second is nothing to a queue. It's there so the person posting doesn't wait. Their post is durable the moment the post store has it, and that's when their request finishes. Delivering it to seventy-five timelines is somebody else's job, and it has five whole seconds to happen. If the worker fleet has a bad minute, delivery lags and nothing is lost, because the queue is holding every message. Loss turns into lag, and lag is the one thing the requirements allow.

That's also what picks the product. At 6k messages a second, throughput decides nothing. All three candidates handle it, so durability picks the winner. Redis pub/sub is fastest and simplest, and it loses immediately: it forgets a message once delivered, so worker downtime loses those posts. That breaks the one promise this queue exists to keep. RabbitMQ is durable and acknowledges each message, but it deletes a message once acknowledged, so there's no going back, and the routing it's good at we don't use. Kafka keeps every message for days after it's read. That buys us what the others can't: if we ship a bug that writes timelines wrong, we fix the worker and replay the last two days. For a box whose whole job is durability, being able to rewind beats routing we'll never use.

How the queue spreads the work. Kafka splits a topic into partitions, and each worker owns some of them. We partition by post id, so consecutive posts land on unrelated partitions and the work spreads flat. The tempting alternative is to partition by author id. That would keep one author's posts in order. It would also send every post from our 50M-follower account to the same partition, which is a hot partition we'd then have to fix. We don't need per-author order anyway. Each post's fan-out is an independent job, and the timeline is sorted when it's read, not when it's written. So post id it is, and the celebrity problem stays where it belongs, for later.

Where the multiplication happens, and it's the most common sizing mistake here. The queue carries about 6k messages a second. The workers emit about 450k writes a second. The amplification is inside the workers, not the queue. Size the queue at 450k and you've misread the write path. One Kafka broker sustains about a hundred thousand appends a second, so our 6k runs at a few percent of one. Now capacity. A message is a post id, an author id, and a little overhead, so call it a hundred bytes. 6k a second is about fifty gigabytes a day. Three days of replay, times three copies, is about half a terabyte across the brokers. That's a rounding error.

What happens when the queue fails. Each message is replicated across brokers, so a broker dying loses nothing that was acknowledged. The harder question is what happens if the queue won't take the message at all. By then the post is already durable in the post store, so the poster still succeeds. The only thing we've lost is the delivery. The post row gets a flag saying its delivery never got queued, and a background sweep re-drives those posts once the brokers are back. That sweep is a few lines of code, not a box on this board. It's the difference between losing a post and delaying one, which is the promise the brief made.

The workers. They're stateless and sized by the work. One worker batches its writes to the cache. It has a whole follower list to write at once, so it pipelines them, and that gets it to roughly 10k timeline writes a second. 450k divided by 10k is forty-five, so call it a few dozen workers, and add more when the number moves. If one dies mid-message, the queue redelivers it, so the same post can be written into the same timeline twice. That's free to handle. A timeline entry is keyed by the post's id, so writing it a second time overwrites the first. Writing it twice and writing it once are the same thing.

The follow graph. Step two read 'the author's follower list' from a box called the follow graph, and that name is a trap worth walking into on purpose. The word graph pulls hard toward a graph database, or something like Meta's TAO, and this is where a lot of good candidates spend five minutes. So ask what we actually do with it. We ask one question: give me the followers of this account. That's it. We never walk the graph. We never ask for friends-of-friends, or the shortest path between two people, or anything else that makes a graph a graph. We read one account's list of followers, by that account's id. A question like that is a row lookup, and a row lookup is answered by a table.

So the follow graph is two plain tables, sharded. One is keyed by the account being followed, so 'who follows this person' is one shard's job. The other is keyed by the follower, so 'who does this person follow' is one shard's job too. Which product? The same one the post store uses: MySQL, sharded. It's a separate cluster, because its shard key and traffic differ, but there's nothing new to learn or operate. That's the point, and it's why this box carries no technology name at all. Every other box earns its name by being chosen. This one is a table, and a table is what you already have. Even Meta's TAO is a cache in front of MySQL, serving the graph walks we never do.

Two operations the follow graph owes, because the brief asked for follow and unfollow. Follow a new account and its old posts aren't in your timeline yet, since fan-out only copies future posts. So a follow kicks off a one-time read of that account's recent posts, merged into your list once. Unfollow is the reverse problem: their posts are already in your list, keyed by you, with no cheap way to pick out just theirs. So we don't hunt them down. The timeline service drops posts from accounts you no longer follow as it serves the page, and they age out of the cache on their own. Both stay cheap, and neither needs a new box.

Size it both ways. Capacity first. Half a billion accounts following about a hundred each is on the order of tens of billions of follow relationships. We store each one twice, once per table, so about a hundred billion rows. A row is two ids and a little overhead, call it twenty bytes. That's about two terabytes, and three copies is six, still a handful of machines. Now throughput. One node serves about 10k point lookups a second. We read this box once per post, 6k reads a second, less than one machine's worth. Take the bigger, the handful. It's the least stressed thing on the board, doing the job people reach for a specialized database to do. Hold on to those two numbers. What comes next changes one of them by fifty times, and the answer moves.

Before we move on, look hard at the number that argument stands on, because the whole next problem is hiding inside it. The average post goes to about seventy-five timelines, which is Twitter's disclosed deliveries a day divided by its disclosed posts a day. Now remember the gap from the brief. Seventy-five is an average, and an average is a summary. This one is summarizing accounts whose followers run to the tens of millions, which is roughly 670k times the average. Everything above says fan-out on write is obviously correct, and for almost every account it is. What comes next is about the accounts where it's a catastrophe.

One account has 50M followers. She posts. Fan-out on write says one post becomes one timeline write per follower, so her one post is 50M timeline writes. The delivery promise is under five seconds, so those 50M writes have to happen in five seconds, which is 10M writes a second from one post. The entire fan-out fleet, serving every account on the platform, does 450k writes a second. So a single post from a single account is asking for roughly twenty times the capacity of the whole system, for five seconds. She posts about ten times a day. She's not the only account like this. What do you do?

The first instinct is buy the capacity: more workers, more cache nodes, size the fleet for the burst. Take this one seriously. 'Add machines' is the correct answer to most problems on this board, and it's what we did for the post store earlier. It fails here, and it fails on arithmetic. The fan-out fleet is a few dozen workers. Sizing it for her worst second means a few hundred workers and the cache tier to match, running twenty times over and sitting idle almost all the time. You'd be buying all that to serve an account that posts ten times today, for five seconds. Buying capacity works when the load is there most of the time. This load is there for fifty seconds a day.

The second instinct is better: let it lag. The queue is already there, it's already durable, and delivery is already allowed to take five seconds. So let her post take a few minutes to work its way through. Nothing is lost. This is genuinely the right answer to a burst, and it's exactly what the queue was built for. But look at who is waiting. It's her 50M followers. The specific 50M people most likely to be refreshing the app right now, because she just posted. The lag lands precisely where it's most visible. And she posts ten times a day, so the backlog from this post is still draining when the next one arrives. This doesn't smooth out. It accumulates.

The third answer is the one that works, and it starts by refusing the premise. Don't fan her out at all. Her posts never get copied into anybody's timeline. Her followers' lists simply don't contain her. Instead, when one of her followers reads their timeline, the timeline service fetches her recent posts and merges them into the list it just read, right there, at read time.

Don't fan her out — merge her in at read time. Refuse the premise. That's the pick, and it's the only one of the three that doesn't accept 50M writes as a thing to be survived. The other two argue about how to absorb the burst. This one stops the burst from happening.

First, one thing about this answer changes what we just decided. Fan-out on write and fan-out on read were argued as alternatives a moment ago. They're not alternatives. They're both in the design, permanently, and an account's follower count decides which one applies. Everybody normal gets fanned out. A few thousand accounts get merged at read time. The system runs both paths forever, and it always did. This isn't a trick invented for interviews. Twitter ran exactly this, and stopped fanning out its biggest accounts once the tail of the follower counts forced a second path.

Here's why the merge is cheap, and it's the idea to keep from this whole problem. Her recent posts are one list, which means one key. 50M people reading her posts are 50M reads of that same key. They all hit the same value in memory, so the machine holding it does the work once and hands out the same answer over and over. Fanning her out asks for the opposite: 50M writes to 50M different keys. No two of those writes have anything in common, so none can share work. Every one is real work that has to happen. Many reads of one thing collapse into one piece of work. Many writes to many things never collapse, because there's nothing to collapse.

There's a second win. Fan-out on write does its work inside five seconds, whether anyone is looking or not. The read-time merge does its work only when somebody reads, spread across the whole day and only for people who open the app. The burst doesn't get absorbed. It stops existing. The cost is real. Every timeline read now does extra work. For each celebrity you follow, fetch their recent list and merge it in. You follow about a hundred accounts and a couple are big, so a read that was one lookup is now one lookup plus about two more, plus a merge of three short lists. That's the trade. A small, bounded, permanent cost on every read, in exchange for never doing 50M writes in five seconds.

So where's the line? It sits at about 10k followers, and this number deserves honesty rather than confidence. It's reconstructed from secondary accounts of a Twitter engineering talk, not a disclosed constant, so treat it as an order of magnitude. What's actually sourced is that the threshold is a tuned value, not a fixed one. Twitter experimented with several. The arithmetic on either side sets it. Below the line, fanning a post out costs at most 10k writes, which the cache tier absorbs without noticing, and it buys every one of those followers a free read. Above the line, the write cost climbs faster than the read savings are worth.

So how does her list get written at all? When a fan-out worker picks a post off the queue, it always adds that post to its author's own list in the timeline cache. Every author, celebrity or not, no exceptions. That's one extra write per post, 6k a second, which is nothing. Then the worker looks at the author's follower count and decides what else to do. Below the line, it fans out to every follower as before. Above the line, it stops. The author's own list is the only copy written, and the read path comes and gets it.

The read path just changed one box's size. The follow graph is now read on every timeline read, not only on every post, because the timeline service has to find which of the accounts you follow are big. 6k reads a second becomes 300k. So go back to the two sizings we did for that box. Capacity didn't move at all: same hundred billion rows, same six terabytes, same handful of machines. Throughput moved by fifty times: 300k reads a second against 10k per node is about thirty nodes. So the answer flips. That box was sized by its capacity earlier, and it's sized by its traffic now. It goes from a handful of machines to about thirty.

So what happens when an account crosses the line? The two directions aren't the same. Going up is the easy one. Her old posts are already sitting in millions of timelines. Nothing chases them down. From the crossing onward, her new posts land only in her own list, and the read path merges them in. The one thing to get right is the dedup rule. The merge asks for her posts newer than the newest post she wrote that's already in your list. Not the newest post in your list overall. That one is almost certainly somebody else's, from ten seconds ago, and asking against it would silently skip everything she wrote in between. Get that predicate right and a post can never be shown twice.

Going down gets waved through, and it doesn't survive a careful look. She loses followers and drops below the line. Fan-out starts again for her new posts, and the merge stops. But every post she made while she was above the line was never copied into anybody's timeline. Those posts aren't lost. The post store has every one. But they quietly disappear from her followers' timelines, which is the kind of bug nobody notices until somebody complains. So the downward crossing needs one extra step. Backfill her: read her recent posts from her own list and fan them out to her followers once. She's below the line now, so it costs at most 10k followers times a handful of posts. A background job nobody feels.

One more thing to be honest about: freshness is now uneven. A celebrity's post appears in your timeline the instant she writes it, because it's merged in live at read time. An ordinary account's post takes a few seconds, because the workers have to deliver it first. So the most-followed accounts on the platform are also the fastest. Nobody set out to build that. It falls straight out of the design, it's harmless, and that's why it stays.

The timeline cache is now the busiest box on this board. Every fan-out write lands in it. Every read is answered from it. It's the reason the read path is four hops instead of a hundred queries. And nobody has said how big it is, so it isn't designed yet. Two parameters decide this box's entire bill, and both are arguable. How much goes in one entry? How many entries does a timeline keep? Answer those two and the size, the node count, and the cost of a read all fall out.

Put the whole post in the timeline. It's a lovely idea. A read becomes one lookup and you're completely finished. No second trip to any other box, the fastest possible read path, and the timeline service barely does anything. Price it. A post is about 500 bytes. 800 posts is 400 kilobytes per person. 500M people, three copies each, is 600 terabytes of memory. That's a preposterous amount of RAM, and it's worse than it looks. The average post is stored 75 separate times, once in each follower's list, and a popular post far more often than that.

Put only the id in the timeline. 8 bytes for the post id, 8 for the author id, 4 for a scrap of metadata: 20 bytes an entry. 800 of those is 16 kilobytes per person. 500M people is 8 terabytes, and three copies is 24 terabytes. That fits. It's a big number but a buildable one, and it's twenty-five times smaller than the alternative.

Only the post id (20 B an entry). Ids only. The memory bill decides it, and it isn't close. 16 kilobytes a person instead of 400, which is twenty-five times smaller. Now name what that choice costs, because it isn't free, and it's the biggest number in this design. The cache holds ids. So the timeline service has to fetch the posts themselves from the post store. A page is 20 posts, and there are 300k timeline reads a second. So the post store is being asked for 6M posts a second. That's a thousand times its write rate.

That 6M is the number the post store dive promised and couldn't compute. Collect on it now. We sized the post store by capacity: 1.4 petabytes of posts, 5 to a machine, 300 machines. The traffic answer waited until now. One machine serves 10k point lookups a second. 6M divided by 10k is 600 machines. Capacity said 300. Traffic says 600. Take the larger. So the post store doubles to 600 shards, because of a decision made here, long after it was first sized. If we'd put whole posts in the timeline, there'd be no second trip, and the store would stay at 300. We didn't buy the smaller cache for free. We paid for it here.

Why can 600 ordinary machines absorb 6M reads a second? It sounds like a lot. But every one of those reads is a single row fetched by its primary key. That's the one job a B+ tree has been the best in the world at for forty years. A B+ tree is the index structure a relational database uses to find one row by key. It's why the boring pick from the post store dive is still standing after the busiest dive in the article.

Now look at the shape of those 6M reads, because it isn't what it first looks like. One read is a page of 20 posts, and those 20 fetches land on 20 different machines, so a timeline read waits for the slowest of 20. That's what the tail of the latency budget is really riding on. And every timeline being read right now holds mostly the same posts: the recent ones, from the popular accounts. So 6M fetches a second land on a small set of rows over and over. Load that skewed is exactly what a memory cache in front of the store would absorb, and it also cuts that slowest-of-twenty tail. A real deployment puts one there. We're not drawing that box on this board. It changes nothing else here, and every decision above stands without it. But naming the load and setting it aside is the honest move.

Why Redis, not Memcached? Look at what a fan-out write does: add one id to the front of a list, and drop the oldest if the list is at 800. Redis has lists built in. Pushing one end and trimming the other is one small operation on 20 bytes. Memcached has no lists. It stores an opaque blob, so adding one id means reading the whole 16-kilobyte blob, adding 20 bytes, and writing all 16 back. Do that 450k times a second and you move seven gigabytes a second to deliver nine megabytes of ids. Worse, two workers writing one timeline at once each read the blob, and the second erases the first's post. Redis wins because it knows the shape of what you store.

How many entries? The cap is 800 posts per timeline, Twitter's real disclosed number. Why 800? Because nobody scrolls past it. The cap isn't a compromise; it's about people. And it's the only real dial on this box's cost. Halve the cap and the memory bill halves with it, and almost nobody would notice. What if somebody does scroll past 800? Then the timeline service builds the older pages the slow way: read the follow graph, query the post store, merge. That's fan-out on read, the design we rejected earlier. So a rejected option isn't a wrong option. It was wrong as the answer. It's exactly right as the fallback for the rare read that goes past the cap.

Both sizings, and they disagree. Throughput first, minding replication and the merge. A write lands on all three replicas while a read touches one. And after the hybrid, one read touches about three cache keys: your own list plus a couple of celebrity lists. Add it up and it's a bit over 2 million operations a second. One Redis node does 100k a second, so about 22 nodes. Now capacity. 24 terabytes at about a hundred gigabytes of usable memory per node is about 240 nodes. Same box, two ways, more than ten times apart. This cache is sized by memory, not traffic. Size it by request rate and you'd build 22 nodes and run out of memory.

One more thing keeps that bill honest. We sized against half a billion active accounts. So eviction isn't what makes the number fit. The number already assumed only people who read. What eviction does is stop the number from growing past that as dormant accounts collect behind it. Timelines are thrown away when they go unread, oldest-unused first. And a fan-out write to a timeline that doesn't exist is dropped, not turned into a new one. So an account that hasn't opened the app in a year costs us nothing. It never starts costing us anything just because somebody it follows keeps posting. If that person comes back, their first read rebuilds their timeline the slow way, once, and everything after that is normal.

It's a cache, and that word carries weight. Every timeline in it can be rebuilt from the post store and the follow graph, so nothing here is the only copy of anything. A node dies, a couple million timelines vanish, and they come back one at a time as people read. Each first read after the death does the slow assembly once, then it's fast again. But price the full rebuild. A whole tier going cold at once is fan-out on read for all 500M timelines, landing on a post store sized for point lookups. That's an outage that makes itself worse. So two rules. The three copies we already counted aren't optional. And rebuilds happen a node at a time, never a tier at a time.