These are the notes for System Design episode 29, our third full design problem. Priya, in Andheri, types "Chai at 5?" and taps send; on Rahul's phone it appears in well under a second, Priya watches one tick, two ticks, then blue, and if Rahul is in a train tunnel the message still reaches him later. Today you design that chat system from scratch: you'll write the requirements, watch polling waste 99.7% of its requests, replace it with one open connection and push, route messages between many chat servers with a session registry, keep messages for offline phones in an inbox, build the ticks from acknowledgements, stop duplicates with message ids, keep order with sequence numbers, and send group messages with fan-out. Every demo is plain Python and runs as-is; outputs are real (Python 3.13). The scale numbers (100 million phones online, 12 messages an hour) are our own assumptions for the exercise, not WhatsApp's figures.
A chat system is software that carries short messages between people's devices, in real time when both are online and later when one isn't. Example: WhatsApp, Telegram, Signal, the chat inside Swiggy or Uber.
The problems
| Problem | What it teaches |
|---|---|
| "Chai at 5?" | Requirements: 1:1 and groups, under a second, offline delivery, ticks, never lost, never twice, in order |
| Asking "anything new?" every second | Polling and why it can't win; push over a persistent connection (WebSocket / TCP) |
| Priya and Rahul on different servers | Many chat servers; the session registry (user → server); forwarding |
| Rahul is in a tunnel | The per-user inbox; deliver on reconnect; delete after delivery |
| One tick, two ticks, blue ticks | Acknowledgements (acks) and read receipts |
| "Sent you 500 rupees", twice | Retries, lost acks, message ids, deduplication, at-least-once delivery |
| Messages out of order | Server-assigned sequence numbers per chat |
| The family group | Fan-out to every member; end-to-end encryption |
| Put it all together | The full design, plus push notifications for closed apps |
Problem 0: "Chai at 5?" (requirements)
Before any boxes and arrows, write down what the system must do. In an interview this is the first two or three minutes.
Functional requirements (what it does):
- One-to-one chats and group chats.
- Delivered in under a second when both phones are online.
- If the receiver is offline, deliver when the phone comes back.
- Show the sender three states: sent, delivered, read (the ticks).
Non-functional requirements (how well it does it):
- Never lost: once the sender sees one tick, the message must reach the receiver eventually.
- Never shown twice, even when the network makes phones retry.
- In order within a chat.
- Scale: we assume 100 million phones online at once. That single number decides our first big design choice.
A few words before we go on:
- A server is a computer that runs all the time and answers requests from other computers (phones, browsers). Example: a "chat server" that receives Priya's message and passes it on to Rahul.
- A request is one message from a client to a server asking for something; the server sends back a response. Example: "any new messages?" → "no".
- Online means the phone currently has a working connection to our servers; offline means it doesn't (no signal, switched off, or the app is closed).
Problem 1: asking "anything new?" every second (polling vs push)
Real life: waiting for a parcel
| Waiting for a parcel | Getting new messages |
|---|---|
| Call the shop every minute: "Has it come?" "No." | Polling: the phone asks the server again and again |
| The courier rings your bell, only when it arrives | Push: the side with the news starts the conversation |
| Your doorbell is always wired up | An open connection from phone to server that stays open |
Polling means a client repeatedly asks a server whether something has changed, on a timer. Example: the phone sends "any new messages for Rahul?" every second.
The obvious way, measured
Rahul's phone polls once a second for an hour, and 12 messages arrive at random times in that hour:
import random
random.seed(29)
# Rahul's phone asks the server "anything new?" every second for one hour
arrivals = sorted(random.uniform(0, 3600) for _ in range(12)) # 12 msgs
polls = useful = 0
delays = []
for t in range(1, 3601): # one poll per second
polls += 1
new = [a for a in arrivals if t - 1 < a <= t]
if new:
useful += 1
delays += [t - a for a in new]
print("polls:", polls, " with news:", useful, f" wasted: {1 - useful / polls:.1%}")
print(f"average delay: {sum(delays) / len(delays):.2f} s")
online = 100_000_000
print(f"{online:,} phones polling: {online:,} requests every second")
Output:
polls: 3600 with news: 12 wasted: 99.7%
average delay: 0.38 s
100,000,000 phones polling: 100,000,000 requests every second
- 3,600 polls, 12 with news: 99.7% of the requests were wasted. Each one still costs the phone battery and data, and costs the server CPU and network.
- Messages still waited for the next poll: 0.38 s on average here (between 0 and 1 s).
- Across 100 million phones that's 100 million requests every second, almost all answered "no".
The trap has no way out. Poll less often and messages arrive late; poll more often and the servers melt. (Homework below: poll every 10 s.)
The fix: keep a connection open and push
Each phone opens one connection to a chat server when the app starts, and keeps it open. The server remembers which connection belongs to which user. When a message for Rahul arrives, the server writes it straight into Rahul's open connection. Rahul's phone never asks; it just listens.
This is a real chat server, running on this machine with Python's asyncio (a library for programs that wait on many network connections at once):
import asyncio, time
phones = {} # user -> their open connection
async def chat_server(reader, writer):
user = (await reader.readline()).decode().strip()
phones[user] = writer # connection stays open
while line := await reader.readline():
to, text = line.decode().strip().split(":", 1)
phones[to].write(f"{user}: {text}\n".encode()) # server PUSHES
async def main():
server = await asyncio.start_server(chat_server, "127.0.0.1", 5222)
r_in, r_out = await asyncio.open_connection("127.0.0.1", 5222)
r_out.write(b"rahul\n") # Rahul connects once and waits
p_in, p_out = await asyncio.open_connection("127.0.0.1", 5222)
p_out.write(b"priya\n")
await asyncio.sleep(0.1)
start = time.perf_counter()
p_out.write(b"rahul:Chai at 5?\n") # Priya sends
msg = await r_in.readline() # Rahul's phone just listens
ms = (time.perf_counter() - start) * 1000
print("rahul got ->", msg.decode().strip())
print("pushed in under 50 ms:", ms < 50)
server.close()
asyncio.run(main())
Output:
rahul got -> priya: Chai at 5?
pushed in under 50 ms: True
Line by line:
asyncio.start_server(chat_server, "127.0.0.1", 5222)starts a server on this computer (127.0.0.1means "this machine") at port 5222. A port is a numbered door on a computer that a program listens on.- Each phone connects with
open_connectionand first sends its name; the server storesphones[user] = writer, the open connection to that phone. - When Priya writes
rahul:Chai at 5?, the server looks up Rahul's open connection and writes to it. That's the push. - Rahul's side only does
await r_in.readline(): wait until something arrives. Zero requests.
In real apps the open connection is a WebSocket or a long-lived TCP connection:
- TCP (Transmission Control Protocol) is the basic way two computers keep a reliable, ordered two-way byte stream between them. Example: our demo's
open_connectionis a TCP connection. - A WebSocket is a standard way to keep a two-way connection open between a client and a server, started as an ordinary HTTP request and then "upgraded". Example: web chat apps and live scoreboards in the browser use WebSockets.
- A persistent connection is any connection that stays open across many messages instead of one request per message.
Most of these connections are idle almost all the time, and an idle open connection costs very little memory, so one chat server can hold a very large number of them. That's why push wins: work happens only when there's news.
Problem 2: Priya and Rahul are on different servers (the session registry)
Real life: a hospital with many wards
| A big hospital | Many chat servers |
|---|---|
| Patients in ward 1, ward 2, ward 3 | Phones connected to chat-1, chat-2, chat-3 |
| The reception register: patient → ward | Session registry: user → server |
| Flowers go only to that ward | The message is forwarded to that one server, which pushes it |
One server cannot hold 100 million connections, so we run thousands of chat servers, and a load balancer (episode 6) spreads new connections across them. Priya's phone happens to land on chat-1, Rahul's on chat-2.
The obvious code
# 3 chat servers; each knows only the phones connected to IT
servers = {
"chat-1": {"priya": "socket#17"},
"chat-2": {"rahul": "socket#42"},
"chat-3": {"neha": "socket#08"},
}
def send(on_server, to, text):
conn = servers[on_server][to] # look for the receiver here
print(f"{on_server} -> {conn}: {text}")
send("chat-1", "priya", "note to self") # works: same server
send("chat-1", "rahul", "Chai at 5?") # Rahul is on chat-2...
Output:
chat-1 -> socket#17: note to self
Traceback (most recent call last):
File "route_bad.py", line 13, in <module>
send("chat-1", "rahul", "Chai at 5?") # Rahul is on chat-2...
~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "route_bad.py", line 9, in send
conn = servers[on_server][to] # look for the receiver here
~~~~~~~~~~~~~~~~~~^^^^
KeyError: 'rahul'
KeyError: 'rahul': chat-1 only knows the phones connected to it. Rahul is connected to chat-2. Asking every server "is Rahul with you?" would be thousands of questions for every single message.
The fix: a session registry
A session is the period during which one phone is connected to one chat server. A session registry is a shared, fast key-value store (like Redis, an in-memory key-value database from episode 7) that records which server each user is connected to. A phone's entry is written when it connects and removed when it disconnects.
servers = {
"chat-1": {"priya": "socket#17"},
"chat-2": {"rahul": "socket#42"},
"chat-3": {"neha": "socket#08"},
}
sessions = {} # shared store: user -> server
for name, phones in servers.items():
for user in phones:
sessions[user] = name # written when a phone connects
def send(on_server, to, text):
target = sessions.get(to) # 1. where is the receiver?
if target is None:
print(f"{to} is offline -> nowhere to send")
return
conn = servers[target][to] # 2. forward to that server
print(f"{on_server} -> {target} -> {conn}: {text}")
send("chat-1", "rahul", "Chai at 5?")
send("chat-1", "neha", "Coming?")
del servers["chat-2"]["rahul"], sessions["rahul"] # Rahul enters a tunnel
send("chat-1", "rahul", "Where are you?")
Output:
chat-1 -> chat-2 -> socket#42: Chai at 5?
chat-1 -> chat-3 -> socket#08: Coming?
rahul is offline -> nowhere to send
sessions.get(to)answers "where is the receiver?" in one lookup.- The message goes to that server (one hop: one trip between two servers), which pushes it down the receiver's connection.
- The last line shows the next problem: Rahul went into a tunnel, his entry was removed, and the message has nowhere to go. Right now it would simply be lost.
Real systems also give each registry entry a short expiry (a TTL, time to live) that the phone keeps refreshing with small heartbeat messages ("I'm still here"). If a phone dies without saying goodbye, its entry expires on its own, so the registry doesn't keep pointing at a dead connection.
Problem 3: Rahul is in a tunnel (the inbox and the ticks)
Real life: the building's letterbox
| Post for a flat | Chat messages |
|---|---|
| The letterbox downstairs keeps post while you're away | A per-user inbox on the server keeps undelivered messages |
| The post office receipt: "we have your letter" | One tick: the server has it |
| A signed delivery slip: it reached the flat | Two ticks: the phone acknowledged it |
| "I read your letter" | Blue ticks: a read receipt |
WhatsApp's own FAQ describes the ticks this way (as quoted by MobileSyrup, 2014): one grey check, "message successfully sent"; two grey checks, "message successfully delivered to the recipient's phone"; two blue checks, "the recipient has read your message".
- An inbox (in this design) is a list on the server of messages that haven't reached one user's phone yet.
- An acknowledgement, or ack, is a small reply that says "I received message X". Example: Rahul's phone sends "got m1" back to the server, and the server tells Priya's phone: two ticks.
- A read receipt is a small message the receiver's phone sends when the user actually opens the chat. Example: Rahul opens the chat, his phone sends "read m1", Priya's ticks turn blue.
The code
online = {"priya"} # rahul's phone is off
inbox = {} # user -> undelivered messages
ticks = {} # message id -> what Priya sees
def send(msg_id, frm, to, text):
inbox.setdefault(to, []).append((msg_id, frm, text))
ticks[msg_id] = "✓ sent (server has it)"
if to in online:
deliver(to)
def deliver(user): # push everything waiting
for msg_id, frm, text in inbox.pop(user, []):
print(f" {user}'s phone shows: {frm}: {text}")
ticks[msg_id] = "✓✓ delivered" # the phone sent back an ack
def connect(user):
online.add(user)
deliver(user)
send("m1", "priya", "rahul", "Chai at 5?")
print("offline ->", ticks["m1"], "| waiting:", len(inbox["rahul"]))
connect("rahul") # out of the tunnel
print("back ->", ticks["m1"], "| waiting:", len(inbox.get("rahul", [])))
ticks["m1"] = "✓✓ read (blue)" # Rahul opens the chat
print("opened ->", ticks["m1"])
Output:
offline -> ✓ sent (server has it) | waiting: 1
rahul's phone shows: priya: Chai at 5?
back -> ✓✓ delivered | waiting: 0
opened -> ✓✓ read (blue)
sendalways saves the message in the receiver's inbox first, then sets one tick: the server has it. Saving first is what makes "never lost" possible.- If the receiver is online,
deliverpushes it right away; when the phone acks, two ticks. - When Rahul comes back,
connectempties his inbox to his phone. - Opening the chat sends a read receipt: blue.
WhatsApp's privacy policy describes the same lifecycle: "Once your messages are delivered, they are deleted from our servers." If a message can't be delivered right away, it's kept "in encrypted form on our servers for up to 30 days as we try to deliver it", and deleted if it's still undelivered after that.
Problem 4: "Sent you 500 rupees", twice (message ids)
Real life: the ack that got lost
- Priya sends "Sent you 500 rupees". It reaches the server and is saved.
- On the way back, her signal drops: the tick never arrives. Her phone thinks the send failed.
- So, sensibly, it retries. Rahul sees the message twice and thinks he got 1,000 rupees.
Retrying is right; the network really does lose things. Retrying blindly is what creates the duplicate.
The code
import uuid
def run(with_ids):
shown, seen = [], set()
msg_id = str(uuid.uuid4()) # made ONCE on Priya's phone
for attempt in (1, 2): # the ack was lost, so the
if with_ids: # phone sends it again
if msg_id in seen:
print(f" attempt {attempt}: duplicate id, ignored")
continue
seen.add(msg_id)
shown.append("Sent you 500 rupees")
print(f" attempt {attempt}: shown to Rahul")
return len(shown)
print("without ids -> Rahul sees it", run(False), "times")
print("with ids -> Rahul sees it", run(True), "times")
Output:
attempt 1: shown to Rahul
attempt 2: shown to Rahul
without ids -> Rahul sees it 2 times
attempt 1: shown to Rahul
attempt 2: duplicate id, ignored
with ids -> Rahul sees it 1 times
- A message id is a unique name for one message, made once on the sender's phone before the first attempt. Every retry carries the same id. Here it's a UUID (universally unique identifier, a 128-bit random id like
3f1c...; episode 23). - Deduplication means recognising something you've already seen and ignoring the repeat. The server keeps the ids it has seen.
- At-least-once delivery means "keep retrying until acknowledged", which can deliver a message more than once. Add deduplication by id and the user sees it exactly once. Handling a repeat safely like this is called idempotency (episode 24): doing it twice has the same effect as doing it once.
Keeping messages in order
You can't sort by the time on each phone: phone clocks are often wrong by seconds or minutes. Instead the server gives every chat its own counter, a sequence number: the Priya-Rahul chat's messages get 41, 42, 43, and so on. Phones show messages sorted by that number; if a phone sees 41 and then 43, it knows 42 is missing and asks for it.
The message store is keyed by chat id + sequence number, and sharded by chat id (episode 12), so all of one chat's messages live together on one shard.
Group chats (fan-out)
| A wedding invitation | A group message |
|---|---|
| One card is printed | Priya's phone sends the message once |
| A copy goes to every family | The server puts a copy in every member's inbox: fan-out |
| Sealed envelopes the postman can't read | End-to-end encryption: only the phones can read the text |
- Fan-out means one incoming message becomes many outgoing deliveries. A group of 50 means 49 or 50 deliveries, each with its own delivered and read state.
- End-to-end encryption (E2EE) means messages are scrambled on the sender's phone and unscrambled only on the receivers' phones, so the servers in the middle carry text they can't read. WhatsApp's privacy policy: "End-to-end encryption means that your messages are encrypted to protect against us and third parties from reading them." Our design doesn't change: servers still route, store and fan out; they just handle sealed envelopes.
The full design
phones ──(one open connection each)──▶ chat servers ──▶ session registry (user → server)
│
└──▶ message store (inboxes; chat id + seq → message)
- Connect: the app opens one connection to a chat server (chosen by a load balancer) and writes
user → serverinto the session registry, refreshed by heartbeats. - Send: the chat server saves the message (with the phone's message id; duplicates ignored), gives it the chat's next sequence number, and replies one tick.
- Route: look up the receiver in the registry, forward to that chat server, push down the open connection.
- Ack: the receiver's phone acks → two ticks to the sender; the message is removed from the inbox. Opening the chat sends a read receipt → blue.
- Offline: the message waits in the inbox and is delivered on reconnect.
- Groups: fan-out to every member's inbox.
- App fully closed: there's no open connection at all, so the server asks the phone platform's push notification service (Apple's or Google's) to wake the phone; the app then connects and empties its inbox.
A push notification is a short alert delivered to a phone by the phone maker's own always-on service, even when your app isn't running. Example: "Priya: Chai at 5?" shown on the lock screen.
The whole chain
- Polling wasted 99.7% → keep a connection open, push.
KeyError: 'rahul'→ session registry.- Offline = lost → per-user inbox.
- Ticks = server ack, phone ack, read receipt.
- Shown twice → message ids; order → sequence numbers.
- Groups → fan-out to every member's inbox.
Common mistakes (and the real outputs)
| Mistake | What you see | Why | Fix |
|---|---|---|---|
| The phone polls "anything new?" every second | polls: 3600 with news: 12 wasted: 99.7% |
Almost every request answers "no" | One open connection; the server pushes |
servers[on_server][to] |
KeyError: 'rahul' |
The receiver is connected to another chat server | Look up user → server in a session registry |
| Push only if online, otherwise drop | rahul is offline -> nowhere to send |
Nothing kept the message | Save to the receiver's inbox first, deliver on reconnect |
| Retry without a message id | without ids -> Rahul sees it 2 times |
The ack was lost, not the message | Id made once on the phone + server-side dedupe |
| Sort messages by the phone's clock | Replies appear above the question | Phone clocks drift | Server sequence number per chat |
| Registry entry never removed | Messages forwarded to a dead connection | The phone died without disconnecting | TTL on entries, refreshed by heartbeats |
Mini project: a tiny chat backend in about 100 lines
Everything from the episode in one runnable file: two chat servers, a registry, inboxes, ticks, message ids, sequence numbers and groups.
# Mini project: a tiny WhatsApp-style chat backend (registry, inbox,
# ticks, message ids, sequence numbers, groups), all in one process.
import uuid
from collections import defaultdict
class ChatBackend:
def __init__(self, servers):
self.servers = {s: {} for s in servers} # server -> user -> phone
self.sessions = {} # registry: user -> server
self.inbox = defaultdict(list) # user -> waiting messages
self.seen_ids = set() # for deduplication
self.seq = defaultdict(int) # chat -> last sequence no.
self.ticks = {} # msg id -> status
self.groups = {} # group -> members
def connect(self, user, server, phone):
self.servers[server][user] = phone
self.sessions[user] = server
self.flush(user) # deliver what waited
def disconnect(self, user):
server = self.sessions.pop(user, None)
if server:
del self.servers[server][user]
def send(self, msg_id, frm, to, text):
chat = "-".join(sorted([frm, to])) # one chat per pair
return self.post(msg_id, frm, chat, [to], text)
def send_group(self, msg_id, frm, group, text):
members = [m for m in self.groups[group] if m != frm]
return self.post(msg_id, frm, group, members, text)
def post(self, msg_id, frm, chat, receivers, text):
if msg_id in self.seen_ids: # a retry: already have it
return self.ticks[msg_id]
self.seen_ids.add(msg_id)
self.seq[chat] += 1 # one number per message
msg = {"id": msg_id, "chat": chat, "seq": self.seq[chat],
"from": frm, "text": text}
self.ticks[msg_id] = "sent" # one tick
for to in receivers: # fan-out (1 for a 1:1)
self.inbox[to].append(msg)
self.flush(to)
return self.ticks[msg_id]
def flush(self, user):
server = self.sessions.get(user)
if server is None:
return # offline: keep waiting
phone = self.servers[server][user]
for msg in self.inbox.pop(user, []):
phone.receive(msg)
self.ticks[msg["id"]] = "delivered" # phone acked: two ticks
def read(self, msg_id):
self.ticks[msg_id] = "read" # blue ticks
class Phone:
def __init__(self, owner):
self.owner, self.shown = owner, []
def receive(self, msg):
self.shown.append((msg["chat"], msg["seq"], msg["text"]))
print(f" [{self.owner}'s phone] {msg['chat']} #{msg['seq']}"
f" {msg['from']}: {msg['text']}")
wa = ChatBackend(["chat-1", "chat-2"])
priya, rahul, neha = Phone("priya"), Phone("rahul"), Phone("neha")
wa.connect("priya", "chat-1", priya)
wa.connect("neha", "chat-2", neha)
print("1. Rahul is offline")
m1 = str(uuid.uuid4())
print(" tick:", wa.send(m1, "priya", "rahul", "Chai at 5?"))
print("2. Priya's phone retries (lost ack)")
print(" tick:", wa.send(m1, "priya", "rahul", "Chai at 5?"))
print("3. Rahul connects to chat-2")
wa.connect("rahul", "chat-2", rahul)
print(" tick:", wa.ticks[m1])
wa.read(m1)
print(" after he opens the chat:", wa.ticks[m1])
print("4. A group message")
wa.groups["family"] = ["priya", "rahul", "neha"]
print(" tick:", wa.send_group(str(uuid.uuid4()), "priya", "family",
"Diwali dinner at Dadi's"))
wa.send(str(uuid.uuid4()), "priya", "rahul", "Bring sweets")
print("5. Rahul's phone, in order:", rahul.shown)
Output:
1. Rahul is offline
tick: sent
2. Priya's phone retries (lost ack)
tick: sent
3. Rahul connects to chat-2
[rahul's phone] priya-rahul #1 priya: Chai at 5?
tick: delivered
after he opens the chat: read
4. A group message
[rahul's phone] family #1 priya: Diwali dinner at Dadi's
[neha's phone] family #1 priya: Diwali dinner at Dadi's
tick: delivered
[rahul's phone] priya-rahul #2 priya: Bring sweets
5. Rahul's phone, in order: [('priya-rahul', 1, 'Chai at 5?'), ('family', 1, "Diwali dinner at Dadi's"), ('priya-rahul', 2, 'Bring sweets')]
Notice: the retry in step 2 returns the existing tick and creates nothing new; the group message gets one sequence number in the family chat even though it went to two phones; and Rahul's 1:1 chat numbers go 1, then 2.
Try extending it: add typing(user, chat) that pushes "typing..." to the other members without saving it to any inbox (if it's late, it's useless), and a last_seen time per user updated on every heartbeat.
Practice
- Homework: in the POLL demo, poll every 10 seconds instead of every second. What happens to the waste and the delay?
- Homework: in the PUSH demo, add a third user, Neha, and send her a message from Rahul.
- Homework: in the ROUTE demo, what goes wrong if the registry still says
rahul → chat-2after his phone died? How do you prevent it? - Homework: how would you build the "typing..." indicator?
- Why must the server save the message before sending one tick?
- Priya sees one grey tick for an hour. List two possible reasons.
- A group has 200 members and Priya sends one message. How many inbox entries does the server write? How many acks can come back?
- Why can't the phones use their own clocks to order messages?
Solutions and hints
1. Change the loop to for t in range(10, 3601, 10): and the window to t - 10 < a <= t. Output:
polls: 360 with news: 12 wasted: 96.7%
average delay: 4.38 s
100,000,000 phones polling: 100,000,000 requests every second
Ten times fewer polls, still 96.7% wasted, and messages now wait 4.38 s on average: far outside "under a second". (The last line is hard-coded in the demo; with 10-second polls it would really be 10 million requests a second, still huge.) Polling trades load for delay; it can't give you both.
2. After Priya's send, add n_in, n_out = await asyncio.open_connection("127.0.0.1", 5222), n_out.write(b"neha\n"), a short sleep, then r_out.write(b"neha:Running late\n") and print(await n_in.readline()). Output ends with:
neha got -> rahul: Running late
3. chat-1 forwards to chat-2, which has no connection for Rahul any more: the push fails (in our demo, a KeyError on chat-2). Fixes: remove the entry on disconnect; give entries a TTL that the phone refreshes with heartbeats so dead phones expire; and if forwarding fails, keep the message in the inbox instead of dropping it.
4. The phone sends a small "typing" event over its open connection; the server routes it like a message (registry lookup, push) but doesn't store it in any inbox and needs no ack. If the receiver is offline it's simply dropped, because a late "typing..." is meaningless. Phones send it at most every few seconds, and the receiver hides it after a few seconds without a new one.
5. One tick is a promise: "the server has your message, it won't be lost". If the server replied first and then crashed before saving, the message would vanish while Priya believes it's safe.
6. Rahul's phone is offline (no signal, switched off, app closed and not woken yet), or his phone received it but the ack hasn't reached the server yet.
7. 199 inbox entries (everyone except Priya), and up to 199 delivery acks and 199 read receipts, each tracked separately.
8. Phone clocks can be minutes off, and two phones' clocks disagree. A reply could sort above the question. The server's per-chat counter gives one agreed order.
Quick quiz
Priya sees: ✓✓ (grey)
Rahul's phone: in his pocket
Rahul: hasn't opened the chat
What do the two grey ticks tell Priya?
- A. The server has it, Rahul's phone doesn't
- B. It reached Rahul's phone, not read yet
- C. Rahul has read it
- D. It failed and will retry
Answer
B. Two grey ticks mean Rahul's phone sent an acknowledgement: the message is on his phone. It turns blue only when he opens the chat and his phone sends a read receipt. One tick would mean only the server has it.
Interview questions: "Design WhatsApp" / "Design a chat system"
1. Clarify requirements (2-3 minutes). 1:1 and groups? Media? Ticks and read receipts? Last seen, typing? How many users online? Must offline users get messages later? Then state: real-time delivery under a second, offline delivery, sent/delivered/read, never lost, never duplicated, ordered per chat, groups.
2. Estimate. "100 million online. If each polled once a second, that's 100 million requests a second, nearly all empty. So: persistent connections and push." Messages a day × average size gives storage for undelivered messages; since delivered messages are deleted from servers, storage stays modest.
3. Connections. Each phone keeps one WebSocket/TCP connection to a chat server behind a load balancer. Chat servers are stateful (they hold connections), so scaling means adding servers and spreading new connections.
4. Routing. A session registry (Redis-like) maps user → chat server, written on connect, removed on disconnect, with TTL + heartbeats. The receiving chat server looks up the receiver and forwards; the receiver's server pushes.
5. Storage and delivery guarantees. Save before ack (one tick). Per-user inbox for undelivered messages. Phone acks → two ticks → remove from inbox. Read receipts → blue. Message ids from the phone for dedupe; server sequence numbers per chat for order; messages keyed by chat id + seq, sharded by chat id.
6. Groups. Fan-out on the server to each member's inbox; per-member delivery state. Very large groups may be read from a shared group log instead of copied per member.
7. Offline and closed apps. Inbox + push notifications (APNs for Apple, FCM for Google) to wake the app.
8. Security. End-to-end encryption: servers route and store ciphertext only.
Common follow-ups:
- Why not HTTP polling, or long polling? (Polling wastes requests and adds delay. Long polling, where the server holds a request open until there's news, is better, but a persistent two-way connection is simpler and cheaper at this scale.)
- What happens when a chat server crashes? (Its phones reconnect to other servers and update the registry; undelivered messages are safe in the inbox because they were saved before the first tick.)
- How do you guarantee exactly-once? (You can't over a lossy network; you do at-least-once delivery plus dedupe by message id, which looks exactly-once to the user.)
- How do you implement "last seen"? (Update a timestamp on heartbeats and disconnects; show it to contacts, respecting privacy settings.)
- How are messages ordered in a group with many senders? (The server assigns the group's sequence number when it saves each message.)
Glossary
| Term | Meaning |
|---|---|
| Chat system | Software carrying short messages between devices, live or later, e.g. WhatsApp |
| Requirements (functional / non-functional) | What the system must do / how well it must do it (speed, scale, reliability) |
| Server | A computer that runs all the time and answers requests, e.g. a chat server |
| Request / response | A client asking a server for something / the server's answer |
| Online / offline | The phone has / doesn't have a working connection to our servers |
| Polling | Asking a server again and again on a timer whether anything changed |
| Push | The server sends data to the client as soon as there's news, without being asked |
| Long polling | A poll the server holds open until there's news or a timeout |
| Persistent connection | A connection kept open across many messages |
| TCP | The standard reliable, ordered two-way byte stream between two computers |
| WebSocket | A standard two-way connection between client and server that stays open, started as an HTTP request |
asyncio |
Python's library for programs that wait on many connections at once |
| Port | A numbered door on a computer that a program listens on, e.g. 5222 in the demo |
127.0.0.1 |
The address meaning "this same computer" |
| Load balancer | Spreads incoming connections or requests across many servers (episode 6) |
| Session | The period one phone is connected to one chat server |
| Session registry | A shared store mapping each online user to the chat server holding their connection |
| Redis | A fast in-memory key-value store (episode 7), often used for registries and counters |
| Hop | One trip of a message from one server to another |
| TTL (time to live) | An expiry on a stored entry, after which it disappears on its own |
| Heartbeat | A small regular "I'm still here" message that keeps a connection or entry alive |
| Inbox | A server-side list of messages not yet delivered to one user's phone |
| Acknowledgement (ack) | A small reply saying "I received message X" |
| Read receipt | A message the receiver's phone sends when the user opens the chat; turns ticks blue |
| Retry | Sending the same thing again because no ack came back |
| Message id | A unique name for one message, made once on the sender's phone and reused on retries |
| UUID | A 128-bit random unique id (episode 23) |
| Deduplication | Recognising a repeat and ignoring it |
| At-least-once delivery | Keep retrying until acked; may deliver more than once |
| Idempotency | Doing something twice has the same effect as once (episode 24) |
| Sequence number | A per-chat counter the server assigns to keep messages in order |
| Shard / sharding | Splitting data across servers by a key, e.g. chat id (episode 12) |
| Fan-out | One incoming message becoming many deliveries, e.g. to every group member |
| End-to-end encryption | Only the sender's and receivers' devices can read the message; servers carry ciphertext |
| Push notification | An alert delivered by Apple's or Google's always-on service to wake an app |
Summary
- Write requirements first: 1:1 and groups, under a second, offline delivery, ticks, never lost, never twice, in order, at our assumed scale of 100 million phones online.
- Polling wasted 99.7% of its requests and still delayed messages; at scale it's 100 million requests a second. Use one persistent connection per phone (WebSocket/TCP) and push.
- Many chat servers mean the receiver may be elsewhere (
KeyError: 'rahul'): a session registry maps user → server, with TTLs and heartbeats. - Save every message to the receiver's inbox first; deliver on reconnect; delete after delivery (WhatsApp: delivered messages are deleted from its servers; undelivered ones are kept encrypted for up to 30 days).
- Ticks: one = the server saved it; two = the phone acked it; blue = a read receipt.
- Lost acks cause retries; message ids + dedupe turn at-least-once delivery into "shown once". Server sequence numbers keep each chat in order.
- Groups are fan-out to every member's inbox; end-to-end encryption keeps servers from reading the text.
- Closed apps are woken with push notifications.
Next episode: you open Instagram and a feed of posts from hundreds of accounts appears instantly. What happens when a celebrity with millions of followers posts? Design the Instagram feed.
Sources: WhatsApp Privacy Policy, whatsapp.com/legal/privacy-policy ("Once your messages are delivered, they are deleted from our servers"; undelivered messages kept "in encrypted form on our servers for up to 30 days as we try to deliver it"; "End-to-end encryption means that your messages are encrypted to protect against us and third parties from reading them"); MobileSyrup, "WhatsApp update brings confirmation of read messages" (6 Nov 2014), quoting the WhatsApp FAQ on check marks. Scale numbers are our own stated assumptions.