Observed Signal · May 26, 2026 · Technical Explainer · Source: DEV Community · Impact: 2/5 · Sentiment: Neutral
Twitter's Fan-out Pattern and Timeline Architecture
This technical explainer reviews Twitter's timeline scalability challenges (circa 2010–2015) and the architecture choices it used to serve timelines quickly at scale. It describes the problem of high-follower authors (e.g., 50 million followers), contrasts fan-out-on-write and fan-out-on-read approaches, and explains Twitter's hybrid: users with <10,000 followers use fan-out-on-write while celebrity accounts (>10,000) use fan-out-on-read. The post covers Twitter's 2014 migration to its Manhattan distributed key-value store (multi-datacenter replication, <10ms p99 reads) and the Snowflake 64-bit ID format (timestamp, datacenter ID, worker ID, sequence). It also discusses trade-offs (latency, storage, complexity), Redis usage for timelines, and takeaway recommendations for feed systems and ID generation alternatives like ULID and UUID v7.
Technical/historical deep-dive on Twitter’s feed and ID architecture that is relevant for engineers and platform designers but does not represent breaking industry-wide change.
Track Snowflake Signals & Market Shifts in Real-Time
Polaris7 autonomous intelligence agents track regulatory filings, primary sources, executive changes, and deal flow 24/7. Create your free Explorer workspace to monitor these entities.
Key Takeaways & Evidence Grounding
- Twitter used a hybrid feed strategy: accounts with <10,000 followers use fan-out-on-write; accounts with >10,000 followers use fan-out-on-read.
- Fan-out-on-write writes a tweet to each follower's timeline (fast reads, high write/storage cost); fan-out-on-read merges author tweets at read time (fast writes, expensive reads).
- Twitter migrated from MySQL to a custom distributed key-value store called Manhattan in 2014, designed for multi-datacenter replication and low latency (under 10ms p99 reads).
- Twitter uses an ID generation system called Snowflake: a 64-bit ID with a 41-bit timestamp, 5-bit datacenter ID, 5-bit worker ID, and 12-bit sequence number.
- Twitter's timeline implementation leverages Redis (e.g., sorted sets) for pre-computed timelines and per-user tweet stores for celebrities.
Connected Companies & Entities
2 Entities mappedOntology Mapping & Concepts
Related Market Signals & Shifts
Recent verified developments and strategic activity across this market segment.
Asos confirms data breach after rogue app notification
UK fashion retailer Asos has confirmed a data breach involving customers' personal information. Hackers broke into a third-party platform hosting data used for customer communications, stealing names, contact information, home addresses, phone numbers, email addresses, and customer profile notes including search queries. The hackers, calling themselves Xuanye Group, sent an unauthorized push notification through Asos' own app, referencing the compromise of data hosted on Snowflake, a cloud data platform. The notification pressured the company to engage or risk a leak. The attackers reportedly gained access by impersonating a trusted contact to obtain login credentials. Snowflake said its own systems were not breached. Asos has 17 million customers. The incident follows a similar breach at fintech firm Betterment earlier in the year.
Snowflake Completes NZISM Restricted Assessment on Amazon Web Services
Snowflake announces completion of NZISM Restricted Assessment on AWS, a compliance milestone for the public sector in New Zealand.
Graduate Ventures launches €125M second seed fund for Dutch startups
Dutch investor Graduate Ventures has launched its second seed fund of €125 million, with €100 million already committed. Since its founding in 2021 by alumni of Delft University of Technology and Erasmus University Rotterdam, the firm has invested in over 90 tech startups and now partners with eight universities and three university medical centres across the Netherlands. The new fund will focus on sectors including AI, deep tech, healthcare, and climate. Notable backers include Frank Slootman (Snowflake), Michiel Muller (Picnic), Jonas Ruyter (DataSnipper), and Peter Wennink (ASML), with participation from Invest-NL and Oost NL. The firm also operates a pre-seed fund, and its portfolio company QuantWare recently raised €152 million in a Series B round.
Track Real-Time Market Signals & Shifts
Set up custom watchlists to receive automated, evidence-grounded executive digests whenever material signals or shifts occur across your tracked landscape.