SyntaxPilot
Courses/System Design/Lesson 29
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

System Design · Lesson 29 of 40

Design WhatsApp chat

queued

These are the notes for System Design episode 29, our third full design problem. Priya, in Andheri, types "Chai at 5?" and taps send; on Rahul's phone it appears in well under a second, Priya watches one tick, two ticks, then blue, and if Rahul is in a train tunnel the message still reaches him later. Today you design that chat system from scratch: you'll write the requirements, watch polling waste 99.7% of its requests, replace it with one open connection and push, route messages between many chat servers with a session registry, keep messages for offline phones in an inbox, build the ticks from acknowledgements, stop duplicates with message ids, keep order with sequence numbers, and send group messages with fan-out. Every demo is plain Python and runs as-is; outputs are real (Python 3.13). The scale numbers (100 million phones online, 12 messages an hour) are our own assumptions for the exercise, not WhatsApp's figures.

A chat system is software that carries short messages between people's devices, in real time when both are online and later when one isn't. Example: WhatsApp, Telegram, Signal, the chat inside Swiggy or Uber.

The problems

Problem What it teaches
"Chai at 5?" Requirements: 1:1 and groups, under a second, offline delivery, ticks, never lost, never twice, in order
Asking "anything new?" every second Polling and why it can't win; push over a persistent connection (WebSocket / TCP)
Priya and Rahul on different servers Many chat servers; the session registry (user → server); forwarding
Rahul is in a tunnel The per-user inbox; deliver on reconnect; delete after delivery
One tick, two ticks, blue ticks Acknowledgements (acks) and read receipts
"Sent you 500 rupees", twice Retries, lost acks, message ids, deduplication, at-least-once delivery
Messages out of order Server-assigned sequence numbers per chat
The family group Fan-out to every member; end-to-end encryption
Put it all together The full design, plus push notifications for closed apps

Problem 0: "Chai at 5?" (requirements)

Before any boxes and arrows, write down what the system must do. In an interview this is the first two or three minutes.

Functional requirements (what it does):

  • One-to-one chats and group chats.
  • Delivered in under a second when both phones are online.
  • If the receiver is offline, deliver when the phone comes back.
  • Show the sender three states: sent, delivered, read (the ticks).

Non-functional requirements (how well it does it):

  • Never lost: once the sender sees one tick, the message must reach the receiver eventually.
  • Never shown twice, even when the network makes phones retry.
  • In order within a chat.
  • Scale: we assume 100 million phones online at once. That single number decides our first big design choice.

A few words before we go on:

  • A server is a computer that runs all the time and answers requests from other computers (phones, browsers). Example: a "chat server" that receives Priya's message and passes it on to Rahul.
  • A request is one message from a client to a server asking for something; the server sends back a response. Example: "any new messages?" → "no".
  • Online means the phone currently has a working connection to our servers; offline means it doesn't (no signal, switched off, or the app is closed).

Problem 1: asking "anything new?" every second (polling vs push)

Real life: waiting for a parcel

Waiting for a parcel Getting new messages
Call the shop every minute: "Has it come?" "No." Polling: the phone asks the server again and again
The courier rings your bell, only when it arrives Push: the side with the news starts the conversation
Your doorbell is always wired up An open connection from phone to server that stays open

Polling means a client repeatedly asks a server whether something has changed, on a timer. Example: the phone sends "any new messages for Rahul?" every second.

The obvious way, measured

Rahul's phone polls once a second for an hour, and 12 messages arrive at random times in that hour:

import random
random.seed(29)
# Rahul's phone asks the server "anything new?" every second for one hour
arrivals = sorted(random.uniform(0, 3600) for _ in range(12))  # 12 msgs
polls = useful = 0
delays = []
for t in range(1, 3601):                 # one poll per second
    polls += 1
    new = [a for a in arrivals if t - 1 < a <= t]
    if new:
        useful += 1
        delays += [t - a for a in new]
print("polls:", polls, " with news:", useful, f" wasted: {1 - useful / polls:.1%}")
print(f"average delay: {sum(delays) / len(delays):.2f} s")
online = 100_000_000
print(f"{online:,} phones polling: {online:,} requests every second")

Output:

polls: 3600  with news: 12  wasted: 99.7%
average delay: 0.38 s
100,000,000 phones polling: 100,000,000 requests every second
  • 3,600 polls, 12 with news: 99.7% of the requests were wasted. Each one still costs the phone battery and data, and costs the server CPU and network.
  • Messages still waited for the next poll: 0.38 s on average here (between 0 and 1 s).
  • Across 100 million phones that's 100 million requests every second, almost all answered "no".

The trap has no way out. Poll less often and messages arrive late; poll more often and the servers melt. (Homework below: poll every 10 s.)

The fix: keep a connection open and push

Each phone opens one connection to a chat server when the app starts, and keeps it open. The server remembers which connection belongs to which user. When a message for Rahul arrives, the server writes it straight into Rahul's open connection. Rahul's phone never asks; it just listens.

This is a real chat server, running on this machine with Python's asyncio (a library for programs that wait on many network connections at once):

import asyncio, time

phones = {}                     # user -> their open connection

async def chat_server(reader, writer):
    user = (await reader.readline()).decode().strip()
    phones[user] = writer                 # connection stays open
    while line := await reader.readline():
        to, text = line.decode().strip().split(":", 1)
        phones[to].write(f"{user}: {text}\n".encode())   # server PUSHES

async def main():
    server = await asyncio.start_server(chat_server, "127.0.0.1", 5222)
    r_in, r_out = await asyncio.open_connection("127.0.0.1", 5222)
    r_out.write(b"rahul\n")               # Rahul connects once and waits
    p_in, p_out = await asyncio.open_connection("127.0.0.1", 5222)
    p_out.write(b"priya\n")
    await asyncio.sleep(0.1)
    start = time.perf_counter()
    p_out.write(b"rahul:Chai at 5?\n")    # Priya sends
    msg = await r_in.readline()           # Rahul's phone just listens
    ms = (time.perf_counter() - start) * 1000
    print("rahul got ->", msg.decode().strip())
    print("pushed in under 50 ms:", ms < 50)
    server.close()

asyncio.run(main())

Output:

rahul got -> priya: Chai at 5?
pushed in under 50 ms: True

Line by line:

  • asyncio.start_server(chat_server, "127.0.0.1", 5222) starts a server on this computer (127.0.0.1 means "this machine") at port 5222. A port is a numbered door on a computer that a program listens on.
  • Each phone connects with open_connection and first sends its name; the server stores phones[user] = writer, the open connection to that phone.
  • When Priya writes rahul:Chai at 5?, the server looks up Rahul's open connection and writes to it. That's the push.
  • Rahul's side only does await r_in.readline(): wait until something arrives. Zero requests.

In real apps the open connection is a WebSocket or a long-lived TCP connection:

  • TCP (Transmission Control Protocol) is the basic way two computers keep a reliable, ordered two-way byte stream between them. Example: our demo's open_connection is a TCP connection.
  • A WebSocket is a standard way to keep a two-way connection open between a client and a server, started as an ordinary HTTP request and then "upgraded". Example: web chat apps and live scoreboards in the browser use WebSockets.
  • A persistent connection is any connection that stays open across many messages instead of one request per message.

Most of these connections are idle almost all the time, and an idle open connection costs very little memory, so one chat server can hold a very large number of them. That's why push wins: work happens only when there's news.

Problem 2: Priya and Rahul are on different servers (the session registry)

Real life: a hospital with many wards

A big hospital Many chat servers
Patients in ward 1, ward 2, ward 3 Phones connected to chat-1, chat-2, chat-3
The reception register: patient → ward Session registry: user → server
Flowers go only to that ward The message is forwarded to that one server, which pushes it

One server cannot hold 100 million connections, so we run thousands of chat servers, and a load balancer (episode 6) spreads new connections across them. Priya's phone happens to land on chat-1, Rahul's on chat-2.

The obvious code

# 3 chat servers; each knows only the phones connected to IT
servers = {
    "chat-1": {"priya": "socket#17"},
    "chat-2": {"rahul": "socket#42"},
    "chat-3": {"neha": "socket#08"},
}

def send(on_server, to, text):
    conn = servers[on_server][to]         # look for the receiver here
    print(f"{on_server} -> {conn}: {text}")

send("chat-1", "priya", "note to self")   # works: same server
send("chat-1", "rahul", "Chai at 5?")     # Rahul is on chat-2...

Output:

chat-1 -> socket#17: note to self
Traceback (most recent call last):
  File "route_bad.py", line 13, in <module>
    send("chat-1", "rahul", "Chai at 5?")     # Rahul is on chat-2...
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "route_bad.py", line 9, in send
    conn = servers[on_server][to]         # look for the receiver here
           ~~~~~~~~~~~~~~~~~~^^^^
KeyError: 'rahul'

KeyError: 'rahul': chat-1 only knows the phones connected to it. Rahul is connected to chat-2. Asking every server "is Rahul with you?" would be thousands of questions for every single message.

The fix: a session registry

A session is the period during which one phone is connected to one chat server. A session registry is a shared, fast key-value store (like Redis, an in-memory key-value database from episode 7) that records which server each user is connected to. A phone's entry is written when it connects and removed when it disconnects.

servers = {
    "chat-1": {"priya": "socket#17"},
    "chat-2": {"rahul": "socket#42"},
    "chat-3": {"neha": "socket#08"},
}
sessions = {}                             # shared store: user -> server
for name, phones in servers.items():
    for user in phones:
        sessions[user] = name             # written when a phone connects

def send(on_server, to, text):
    target = sessions.get(to)             # 1. where is the receiver?
    if target is None:
        print(f"{to} is offline -> nowhere to send")
        return
    conn = servers[target][to]            # 2. forward to that server
    print(f"{on_server} -> {target} -> {conn}: {text}")

send("chat-1", "rahul", "Chai at 5?")
send("chat-1", "neha", "Coming?")
del servers["chat-2"]["rahul"], sessions["rahul"]   # Rahul enters a tunnel
send("chat-1", "rahul", "Where are you?")

Output:

chat-1 -> chat-2 -> socket#42: Chai at 5?
chat-1 -> chat-3 -> socket#08: Coming?
rahul is offline -> nowhere to send
  • sessions.get(to) answers "where is the receiver?" in one lookup.
  • The message goes to that server (one hop: one trip between two servers), which pushes it down the receiver's connection.
  • The last line shows the next problem: Rahul went into a tunnel, his entry was removed, and the message has nowhere to go. Right now it would simply be lost.

Real systems also give each registry entry a short expiry (a TTL, time to live) that the phone keeps refreshing with small heartbeat messages ("I'm still here"). If a phone dies without saying goodbye, its entry expires on its own, so the registry doesn't keep pointing at a dead connection.

Problem 3: Rahul is in a tunnel (the inbox and the ticks)

Real life: the building's letterbox

Post for a flat Chat messages
The letterbox downstairs keeps post while you're away A per-user inbox on the server keeps undelivered messages
The post office receipt: "we have your letter" One tick: the server has it
A signed delivery slip: it reached the flat Two ticks: the phone acknowledged it
"I read your letter" Blue ticks: a read receipt

WhatsApp's own FAQ describes the ticks this way (as quoted by MobileSyrup, 2014): one grey check, "message successfully sent"; two grey checks, "message successfully delivered to the recipient's phone"; two blue checks, "the recipient has read your message".

  • An inbox (in this design) is a list on the server of messages that haven't reached one user's phone yet.
  • An acknowledgement, or ack, is a small reply that says "I received message X". Example: Rahul's phone sends "got m1" back to the server, and the server tells Priya's phone: two ticks.
  • A read receipt is a small message the receiver's phone sends when the user actually opens the chat. Example: Rahul opens the chat, his phone sends "read m1", Priya's ticks turn blue.

The code

online = {"priya"}                        # rahul's phone is off
inbox = {}                                # user -> undelivered messages
ticks = {}                                # message id -> what Priya sees

def send(msg_id, frm, to, text):
    inbox.setdefault(to, []).append((msg_id, frm, text))
    ticks[msg_id] = "✓  sent (server has it)"
    if to in online:
        deliver(to)

def deliver(user):                        # push everything waiting
    for msg_id, frm, text in inbox.pop(user, []):
        print(f"  {user}'s phone shows: {frm}: {text}")
        ticks[msg_id] = "✓✓ delivered"    # the phone sent back an ack

def connect(user):
    online.add(user)
    deliver(user)

send("m1", "priya", "rahul", "Chai at 5?")
print("offline ->", ticks["m1"], "| waiting:", len(inbox["rahul"]))
connect("rahul")                          # out of the tunnel
print("back    ->", ticks["m1"], "| waiting:", len(inbox.get("rahul", [])))
ticks["m1"] = "✓✓ read (blue)"            # Rahul opens the chat
print("opened  ->", ticks["m1"])

Output:

offline -> ✓  sent (server has it) | waiting: 1
  rahul's phone shows: priya: Chai at 5?
back    -> ✓✓ delivered | waiting: 0
opened  -> ✓✓ read (blue)
  • send always saves the message in the receiver's inbox first, then sets one tick: the server has it. Saving first is what makes "never lost" possible.
  • If the receiver is online, deliver pushes it right away; when the phone acks, two ticks.
  • When Rahul comes back, connect empties his inbox to his phone.
  • Opening the chat sends a read receipt: blue.

WhatsApp's privacy policy describes the same lifecycle: "Once your messages are delivered, they are deleted from our servers." If a message can't be delivered right away, it's kept "in encrypted form on our servers for up to 30 days as we try to deliver it", and deleted if it's still undelivered after that.

Problem 4: "Sent you 500 rupees", twice (message ids)

Real life: the ack that got lost

  1. Priya sends "Sent you 500 rupees". It reaches the server and is saved.
  2. On the way back, her signal drops: the tick never arrives. Her phone thinks the send failed.
  3. So, sensibly, it retries. Rahul sees the message twice and thinks he got 1,000 rupees.

Retrying is right; the network really does lose things. Retrying blindly is what creates the duplicate.

The code

import uuid

def run(with_ids):
    shown, seen = [], set()
    msg_id = str(uuid.uuid4())            # made ONCE on Priya's phone
    for attempt in (1, 2):                # the ack was lost, so the
        if with_ids:                      # phone sends it again
            if msg_id in seen:
                print(f"  attempt {attempt}: duplicate id, ignored")
                continue
            seen.add(msg_id)
        shown.append("Sent you 500 rupees")
        print(f"  attempt {attempt}: shown to Rahul")
    return len(shown)

print("without ids -> Rahul sees it", run(False), "times")
print("with ids    -> Rahul sees it", run(True), "times")

Output:

  attempt 1: shown to Rahul
  attempt 2: shown to Rahul
without ids -> Rahul sees it 2 times
  attempt 1: shown to Rahul
  attempt 2: duplicate id, ignored
with ids    -> Rahul sees it 1 times
  • A message id is a unique name for one message, made once on the sender's phone before the first attempt. Every retry carries the same id. Here it's a UUID (universally unique identifier, a 128-bit random id like 3f1c...; episode 23).
  • Deduplication means recognising something you've already seen and ignoring the repeat. The server keeps the ids it has seen.
  • At-least-once delivery means "keep retrying until acknowledged", which can deliver a message more than once. Add deduplication by id and the user sees it exactly once. Handling a repeat safely like this is called idempotency (episode 24): doing it twice has the same effect as doing it once.

Keeping messages in order

You can't sort by the time on each phone: phone clocks are often wrong by seconds or minutes. Instead the server gives every chat its own counter, a sequence number: the Priya-Rahul chat's messages get 41, 42, 43, and so on. Phones show messages sorted by that number; if a phone sees 41 and then 43, it knows 42 is missing and asks for it.

The message store is keyed by chat id + sequence number, and sharded by chat id (episode 12), so all of one chat's messages live together on one shard.

Group chats (fan-out)

A wedding invitation A group message
One card is printed Priya's phone sends the message once
A copy goes to every family The server puts a copy in every member's inbox: fan-out
Sealed envelopes the postman can't read End-to-end encryption: only the phones can read the text
  • Fan-out means one incoming message becomes many outgoing deliveries. A group of 50 means 49 or 50 deliveries, each with its own delivered and read state.
  • End-to-end encryption (E2EE) means messages are scrambled on the sender's phone and unscrambled only on the receivers' phones, so the servers in the middle carry text they can't read. WhatsApp's privacy policy: "End-to-end encryption means that your messages are encrypted to protect against us and third parties from reading them." Our design doesn't change: servers still route, store and fan out; they just handle sealed envelopes.

The full design

 phones ──(one open connection each)──▶ chat servers ──▶ session registry (user → server)
                                            │
                                            └──▶ message store (inboxes; chat id + seq → message)
  1. Connect: the app opens one connection to a chat server (chosen by a load balancer) and writes user → server into the session registry, refreshed by heartbeats.
  2. Send: the chat server saves the message (with the phone's message id; duplicates ignored), gives it the chat's next sequence number, and replies one tick.
  3. Route: look up the receiver in the registry, forward to that chat server, push down the open connection.
  4. Ack: the receiver's phone acks → two ticks to the sender; the message is removed from the inbox. Opening the chat sends a read receipt → blue.
  5. Offline: the message waits in the inbox and is delivered on reconnect.
  6. Groups: fan-out to every member's inbox.
  7. App fully closed: there's no open connection at all, so the server asks the phone platform's push notification service (Apple's or Google's) to wake the phone; the app then connects and empties its inbox.

A push notification is a short alert delivered to a phone by the phone maker's own always-on service, even when your app isn't running. Example: "Priya: Chai at 5?" shown on the lock screen.

The whole chain

  1. Polling wasted 99.7% → keep a connection open, push.
  2. KeyError: 'rahul' → session registry.
  3. Offline = lost → per-user inbox.
  4. Ticks = server ack, phone ack, read receipt.
  5. Shown twice → message ids; order → sequence numbers.
  6. Groups → fan-out to every member's inbox.

Common mistakes (and the real outputs)

Mistake What you see Why Fix
The phone polls "anything new?" every second polls: 3600 with news: 12 wasted: 99.7% Almost every request answers "no" One open connection; the server pushes
servers[on_server][to] KeyError: 'rahul' The receiver is connected to another chat server Look up user → server in a session registry
Push only if online, otherwise drop rahul is offline -> nowhere to send Nothing kept the message Save to the receiver's inbox first, deliver on reconnect
Retry without a message id without ids -> Rahul sees it 2 times The ack was lost, not the message Id made once on the phone + server-side dedupe
Sort messages by the phone's clock Replies appear above the question Phone clocks drift Server sequence number per chat
Registry entry never removed Messages forwarded to a dead connection The phone died without disconnecting TTL on entries, refreshed by heartbeats

Mini project: a tiny chat backend in about 100 lines

Everything from the episode in one runnable file: two chat servers, a registry, inboxes, ticks, message ids, sequence numbers and groups.

# Mini project: a tiny WhatsApp-style chat backend (registry, inbox,
# ticks, message ids, sequence numbers, groups), all in one process.
import uuid
from collections import defaultdict

class ChatBackend:
    def __init__(self, servers):
        self.servers = {s: {} for s in servers}   # server -> user -> phone
        self.sessions = {}                        # registry: user -> server
        self.inbox = defaultdict(list)            # user -> waiting messages
        self.seen_ids = set()                     # for deduplication
        self.seq = defaultdict(int)               # chat -> last sequence no.
        self.ticks = {}                           # msg id -> status
        self.groups = {}                          # group -> members

    def connect(self, user, server, phone):
        self.servers[server][user] = phone
        self.sessions[user] = server
        self.flush(user)                          # deliver what waited

    def disconnect(self, user):
        server = self.sessions.pop(user, None)
        if server:
            del self.servers[server][user]

    def send(self, msg_id, frm, to, text):
        chat = "-".join(sorted([frm, to]))        # one chat per pair
        return self.post(msg_id, frm, chat, [to], text)

    def send_group(self, msg_id, frm, group, text):
        members = [m for m in self.groups[group] if m != frm]
        return self.post(msg_id, frm, group, members, text)

    def post(self, msg_id, frm, chat, receivers, text):
        if msg_id in self.seen_ids:               # a retry: already have it
            return self.ticks[msg_id]
        self.seen_ids.add(msg_id)
        self.seq[chat] += 1                       # one number per message
        msg = {"id": msg_id, "chat": chat, "seq": self.seq[chat],
               "from": frm, "text": text}
        self.ticks[msg_id] = "sent"               # one tick
        for to in receivers:                      # fan-out (1 for a 1:1)
            self.inbox[to].append(msg)
            self.flush(to)
        return self.ticks[msg_id]

    def flush(self, user):
        server = self.sessions.get(user)
        if server is None:
            return                                # offline: keep waiting
        phone = self.servers[server][user]
        for msg in self.inbox.pop(user, []):
            phone.receive(msg)
            self.ticks[msg["id"]] = "delivered"   # phone acked: two ticks

    def read(self, msg_id):
        self.ticks[msg_id] = "read"               # blue ticks

class Phone:
    def __init__(self, owner):
        self.owner, self.shown = owner, []
    def receive(self, msg):
        self.shown.append((msg["chat"], msg["seq"], msg["text"]))
        print(f"  [{self.owner}'s phone] {msg['chat']} #{msg['seq']}"
              f" {msg['from']}: {msg['text']}")

wa = ChatBackend(["chat-1", "chat-2"])
priya, rahul, neha = Phone("priya"), Phone("rahul"), Phone("neha")
wa.connect("priya", "chat-1", priya)
wa.connect("neha", "chat-2", neha)

print("1. Rahul is offline")
m1 = str(uuid.uuid4())
print("   tick:", wa.send(m1, "priya", "rahul", "Chai at 5?"))
print("2. Priya's phone retries (lost ack)")
print("   tick:", wa.send(m1, "priya", "rahul", "Chai at 5?"))
print("3. Rahul connects to chat-2")
wa.connect("rahul", "chat-2", rahul)
print("   tick:", wa.ticks[m1])
wa.read(m1)
print("   after he opens the chat:", wa.ticks[m1])
print("4. A group message")
wa.groups["family"] = ["priya", "rahul", "neha"]
print("   tick:", wa.send_group(str(uuid.uuid4()), "priya", "family",
                             "Diwali dinner at Dadi's"))
wa.send(str(uuid.uuid4()), "priya", "rahul", "Bring sweets")
print("5. Rahul's phone, in order:", rahul.shown)

Output:

1. Rahul is offline
   tick: sent
2. Priya's phone retries (lost ack)
   tick: sent
3. Rahul connects to chat-2
  [rahul's phone] priya-rahul #1 priya: Chai at 5?
   tick: delivered
   after he opens the chat: read
4. A group message
  [rahul's phone] family #1 priya: Diwali dinner at Dadi's
  [neha's phone] family #1 priya: Diwali dinner at Dadi's
   tick: delivered
  [rahul's phone] priya-rahul #2 priya: Bring sweets
5. Rahul's phone, in order: [('priya-rahul', 1, 'Chai at 5?'), ('family', 1, "Diwali dinner at Dadi's"), ('priya-rahul', 2, 'Bring sweets')]

Notice: the retry in step 2 returns the existing tick and creates nothing new; the group message gets one sequence number in the family chat even though it went to two phones; and Rahul's 1:1 chat numbers go 1, then 2.

Try extending it: add typing(user, chat) that pushes "typing..." to the other members without saving it to any inbox (if it's late, it's useless), and a last_seen time per user updated on every heartbeat.

Practice

  1. Homework: in the POLL demo, poll every 10 seconds instead of every second. What happens to the waste and the delay?
  2. Homework: in the PUSH demo, add a third user, Neha, and send her a message from Rahul.
  3. Homework: in the ROUTE demo, what goes wrong if the registry still says rahul → chat-2 after his phone died? How do you prevent it?
  4. Homework: how would you build the "typing..." indicator?
  5. Why must the server save the message before sending one tick?
  6. Priya sees one grey tick for an hour. List two possible reasons.
  7. A group has 200 members and Priya sends one message. How many inbox entries does the server write? How many acks can come back?
  8. Why can't the phones use their own clocks to order messages?
Solutions and hints

1. Change the loop to for t in range(10, 3601, 10): and the window to t - 10 < a <= t. Output:

polls: 360  with news: 12  wasted: 96.7%
average delay: 4.38 s
100,000,000 phones polling: 100,000,000 requests every second

Ten times fewer polls, still 96.7% wasted, and messages now wait 4.38 s on average: far outside "under a second". (The last line is hard-coded in the demo; with 10-second polls it would really be 10 million requests a second, still huge.) Polling trades load for delay; it can't give you both.

2. After Priya's send, add n_in, n_out = await asyncio.open_connection("127.0.0.1", 5222), n_out.write(b"neha\n"), a short sleep, then r_out.write(b"neha:Running late\n") and print(await n_in.readline()). Output ends with:

neha got -> rahul: Running late

3. chat-1 forwards to chat-2, which has no connection for Rahul any more: the push fails (in our demo, a KeyError on chat-2). Fixes: remove the entry on disconnect; give entries a TTL that the phone refreshes with heartbeats so dead phones expire; and if forwarding fails, keep the message in the inbox instead of dropping it.

4. The phone sends a small "typing" event over its open connection; the server routes it like a message (registry lookup, push) but doesn't store it in any inbox and needs no ack. If the receiver is offline it's simply dropped, because a late "typing..." is meaningless. Phones send it at most every few seconds, and the receiver hides it after a few seconds without a new one.

5. One tick is a promise: "the server has your message, it won't be lost". If the server replied first and then crashed before saving, the message would vanish while Priya believes it's safe.

6. Rahul's phone is offline (no signal, switched off, app closed and not woken yet), or his phone received it but the ack hasn't reached the server yet.

7. 199 inbox entries (everyone except Priya), and up to 199 delivery acks and 199 read receipts, each tracked separately.

8. Phone clocks can be minutes off, and two phones' clocks disagree. A reply could sort above the question. The server's per-chat counter gives one agreed order.

Quick quiz

Priya sees: ✓✓ (grey)
Rahul's phone: in his pocket
Rahul: hasn't opened the chat

What do the two grey ticks tell Priya?

  • A. The server has it, Rahul's phone doesn't
  • B. It reached Rahul's phone, not read yet
  • C. Rahul has read it
  • D. It failed and will retry
Answer

B. Two grey ticks mean Rahul's phone sent an acknowledgement: the message is on his phone. It turns blue only when he opens the chat and his phone sends a read receipt. One tick would mean only the server has it.

Interview questions: "Design WhatsApp" / "Design a chat system"

1. Clarify requirements (2-3 minutes). 1:1 and groups? Media? Ticks and read receipts? Last seen, typing? How many users online? Must offline users get messages later? Then state: real-time delivery under a second, offline delivery, sent/delivered/read, never lost, never duplicated, ordered per chat, groups.

2. Estimate. "100 million online. If each polled once a second, that's 100 million requests a second, nearly all empty. So: persistent connections and push." Messages a day × average size gives storage for undelivered messages; since delivered messages are deleted from servers, storage stays modest.

3. Connections. Each phone keeps one WebSocket/TCP connection to a chat server behind a load balancer. Chat servers are stateful (they hold connections), so scaling means adding servers and spreading new connections.

4. Routing. A session registry (Redis-like) maps user → chat server, written on connect, removed on disconnect, with TTL + heartbeats. The receiving chat server looks up the receiver and forwards; the receiver's server pushes.

5. Storage and delivery guarantees. Save before ack (one tick). Per-user inbox for undelivered messages. Phone acks → two ticks → remove from inbox. Read receipts → blue. Message ids from the phone for dedupe; server sequence numbers per chat for order; messages keyed by chat id + seq, sharded by chat id.

6. Groups. Fan-out on the server to each member's inbox; per-member delivery state. Very large groups may be read from a shared group log instead of copied per member.

7. Offline and closed apps. Inbox + push notifications (APNs for Apple, FCM for Google) to wake the app.

8. Security. End-to-end encryption: servers route and store ciphertext only.

Common follow-ups:

  • Why not HTTP polling, or long polling? (Polling wastes requests and adds delay. Long polling, where the server holds a request open until there's news, is better, but a persistent two-way connection is simpler and cheaper at this scale.)
  • What happens when a chat server crashes? (Its phones reconnect to other servers and update the registry; undelivered messages are safe in the inbox because they were saved before the first tick.)
  • How do you guarantee exactly-once? (You can't over a lossy network; you do at-least-once delivery plus dedupe by message id, which looks exactly-once to the user.)
  • How do you implement "last seen"? (Update a timestamp on heartbeats and disconnects; show it to contacts, respecting privacy settings.)
  • How are messages ordered in a group with many senders? (The server assigns the group's sequence number when it saves each message.)

Glossary

Term Meaning
Chat system Software carrying short messages between devices, live or later, e.g. WhatsApp
Requirements (functional / non-functional) What the system must do / how well it must do it (speed, scale, reliability)
Server A computer that runs all the time and answers requests, e.g. a chat server
Request / response A client asking a server for something / the server's answer
Online / offline The phone has / doesn't have a working connection to our servers
Polling Asking a server again and again on a timer whether anything changed
Push The server sends data to the client as soon as there's news, without being asked
Long polling A poll the server holds open until there's news or a timeout
Persistent connection A connection kept open across many messages
TCP The standard reliable, ordered two-way byte stream between two computers
WebSocket A standard two-way connection between client and server that stays open, started as an HTTP request
asyncio Python's library for programs that wait on many connections at once
Port A numbered door on a computer that a program listens on, e.g. 5222 in the demo
127.0.0.1 The address meaning "this same computer"
Load balancer Spreads incoming connections or requests across many servers (episode 6)
Session The period one phone is connected to one chat server
Session registry A shared store mapping each online user to the chat server holding their connection
Redis A fast in-memory key-value store (episode 7), often used for registries and counters
Hop One trip of a message from one server to another
TTL (time to live) An expiry on a stored entry, after which it disappears on its own
Heartbeat A small regular "I'm still here" message that keeps a connection or entry alive
Inbox A server-side list of messages not yet delivered to one user's phone
Acknowledgement (ack) A small reply saying "I received message X"
Read receipt A message the receiver's phone sends when the user opens the chat; turns ticks blue
Retry Sending the same thing again because no ack came back
Message id A unique name for one message, made once on the sender's phone and reused on retries
UUID A 128-bit random unique id (episode 23)
Deduplication Recognising a repeat and ignoring it
At-least-once delivery Keep retrying until acked; may deliver more than once
Idempotency Doing something twice has the same effect as once (episode 24)
Sequence number A per-chat counter the server assigns to keep messages in order
Shard / sharding Splitting data across servers by a key, e.g. chat id (episode 12)
Fan-out One incoming message becoming many deliveries, e.g. to every group member
End-to-end encryption Only the sender's and receivers' devices can read the message; servers carry ciphertext
Push notification An alert delivered by Apple's or Google's always-on service to wake an app

Summary

  • Write requirements first: 1:1 and groups, under a second, offline delivery, ticks, never lost, never twice, in order, at our assumed scale of 100 million phones online.
  • Polling wasted 99.7% of its requests and still delayed messages; at scale it's 100 million requests a second. Use one persistent connection per phone (WebSocket/TCP) and push.
  • Many chat servers mean the receiver may be elsewhere (KeyError: 'rahul'): a session registry maps user → server, with TTLs and heartbeats.
  • Save every message to the receiver's inbox first; deliver on reconnect; delete after delivery (WhatsApp: delivered messages are deleted from its servers; undelivered ones are kept encrypted for up to 30 days).
  • Ticks: one = the server saved it; two = the phone acked it; blue = a read receipt.
  • Lost acks cause retries; message ids + dedupe turn at-least-once delivery into "shown once". Server sequence numbers keep each chat in order.
  • Groups are fan-out to every member's inbox; end-to-end encryption keeps servers from reading the text.
  • Closed apps are woken with push notifications.

Next episode: you open Instagram and a feed of posts from hundreds of accounts appears instantly. What happens when a celebrity with millions of followers posts? Design the Instagram feed.

Sources: WhatsApp Privacy Policy, whatsapp.com/legal/privacy-policy ("Once your messages are delivered, they are deleted from our servers"; undelivered messages kept "in encrypted form on our servers for up to 30 days as we try to deliver it"; "End-to-end encryption means that your messages are encrypted to protect against us and third parties from reading them"); MobileSyrup, "WhatsApp update brings confirmation of read messages" (6 Nov 2014), quoting the WhatsApp FAQ on check marks. Scale numbers are our own stated assumptions.