SyntaxPilot
Courses/System Design/Lesson 39
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

System Design · Lesson 39 of 40

The system design interview framework

queued

These are the notes for System Design episode 39. Arjun is in his first system design round. The interviewer says: "Design a live cricket score app. You have 45 minutes." Arjun knows every part from this course, so he starts drawing Kafka, Redis and Cassandra at once. At minute 45 the interviewer asks "how many users was this for?" and Arjun has no answer. Today you learn how to do it with a clock running and someone watching. You'll replay Arjun's round in code and watch it end in TimeoutError: time's up, never covered: requirements, estimate, API, data model, deep dive, trade-offs, then invent a 7-step plan with a time budget (5, 5, 5, 5, 10, 10 and 5 minutes). Then you fill each step with the cricket question: requirements (ask before you draw, so you don't design video for nothing), a back-of-the-envelope estimate (1,000,000 reads a second against 0.017 writes a second, a 60,000,000 : 1 ratio, 4 Gbit/s), the API and the data model (an append-only ball log and a since cursor), the high-level design, a deep dive into the one hot key (a CDN with a 2-second TTL makes the database see 10,000 times fewer requests), and the trade-offs (a score up to 12 s old). Every demo is plain Python and runs as-is; outputs are real (Python 3.13). Arjun and the interviewer are fictional. This episode states no facts about real systems or real interviews: the 45-minute round, 1 crore (10 million) viewers, the 10-second refresh, the 4-hour match, the 500-byte score, the 200 CDN edge servers and the 2-second TTL are all our own assumptions. The cricket rules (a T20 innings is 20 overs of 6 balls, so 240 legal balls in a full match) are general knowledge.

A system design interview is a conversation, usually 45 to 60 minutes, where you are given a big vague product ("design a live cricket score app") and asked to sketch how you would build it, while the interviewer watches how you think (general knowledge; the 45 minutes here is our assumption). Example: Arjun at a whiteboard, talking out loud, with someone taking notes.

The problems

Problem What it teaches
Arjun draws boxes for 45 minutes No plan means no coverage: a 7-step plan with a time budget
He almost designs the wrong thing Step 1, requirements: ask first, write down scope
He has no idea how big it is Step 2, estimate: reads, writes, ratio, bandwidth
Boxes with nothing inside Steps 3 and 4: the API and the data model (ball log, since)
A wall of boxes Step 5, high-level design: start simple
One hot key, a million reads a second Step 6, deep dive: a CDN and a TTL
"My design is perfect" Step 7, trade-offs, bottlenecks and failures
What does the interviewer want? The scorecard: clarify, numbers, simple design, depth, trade-offs, talking

Problem 0: Arjun's interview (the question)

The interviewer says: "Design a live cricket score app." That is the whole question. Look how much is missing:

  • Just the score, or ball-by-ball commentary too? Live video? Chat?
  • How fresh must the score be?
  • How many people, in one country or the world?

Nobody will tell you unless you ask. A system design question is vague on purpose: the interviewer wants to see what you do with a vague problem, as you would in a real job.

A few words first:

  • Requirements are the list of what the system must do and how well. Example: "show the live score, within about 10 seconds, for 1 crore viewers".
  • A functional requirement says what the system does. Example: "show the score and ball-by-ball updates for any match".
  • A non-functional requirement says how well it must do it: speed, size, reliability. Example: "never go down during a final".
  • Scope is the line around what you will and will not design. Example: "no video, no chat, no login".
  • A box (or component) is one part of the design drawn as a rectangle on the board. Example: a "database" box, a "cache" box.
  • Kafka, Redis and Cassandra are real tools: Kafka moves streams of messages between programs, Redis is a very fast in-memory store often used as a cache, Cassandra is a database built to spread across many machines (general knowledge). Arjun drew them before he knew whether he needed them.

Problem 1: no plan (a 7-step plan with a time budget)

Arjun spends 20 minutes drawing boxes, 15 arguing SQL versus NoSQL, and the last 10 answering "how many users?". We replay his round. must_cover is the list of parts an interviewer hopes to hear; he only touched "high-level design". If the clock hits 45 and parts are missing, we raise an error:

# Arjun's 45 minutes: "Design a live cricket score app"
arjun = [("draw boxes: Kafka, Redis, Cassandra", 20),
         ("argue SQL vs NoSQL", 15),
         ("answer: how many users?", 10)]
must_cover = ["requirements", "estimate", "API", "data model",
              "high-level design", "deep dive", "trade-offs"]

clock = 0
for step, minutes in arjun:
    clock += minutes
    print(f"minute {clock:2}: {step}")

covered = ["high-level design"]          # only the boxes
missing = [s for s in must_cover if s not in covered]
if clock >= 45 and missing:
    raise TimeoutError(f"time's up, never covered: {', '.join(missing)}")

Output (file path in the traceback shortened to main.py):

minute 20: draw boxes: Kafka, Redis, Cassandra
minute 35: argue SQL vs NoSQL
minute 45: answer: how many users?
Traceback (most recent call last):
  File "main.py", line 16, in <module>
    raise TimeoutError(f"time's up, never covered: {', '.join(missing)}")
TimeoutError: time's up, never covered: requirements, estimate, API, data model, deep dive, trade-offs
  • The clock reaches minute 45 and six of the seven parts were never covered: requirements, estimate, API, data model, deep dive, trade-offs.
  • A TimeoutError is Python's error for "the time allowed ran out". Here it is our way to show the round ending. A traceback is Python's report of where the error happened, innermost call last.
  • [s for s in must_cover if s not in covered] is a list comprehension: it builds a new list of every s from must_cover that is not in covered. ', '.join(missing) glues the list into one text with commas.
  • Why did it happen? Arjun had the knowledge, but no plan. He spent the time on whatever came to mind first, and the interviewer never saw how he thinks.

Real life: the board exam

You already know the fix from exams.

In a 3-hour exam In a 45-minute round
Read the whole paper first Requirements first: clarify the question
Minutes per question, by the marks A time budget: minutes per step
Attempt every question, then add detail Cover every step, then go deep

The fix: seven steps, one clock

plan = [("1. Requirements: what, for whom", 5),
        ("2. Estimate: users, reads, writes", 5),
        ("3. API: the few calls that matter", 5),
        ("4. Data model: what we store", 5),
        ("5. High-level design: the boxes", 10),
        ("6. Deep dive: the hardest part", 10),
        ("7. Trade-offs, bottlenecks, wrap-up", 5)]

clock = 0
for step, minutes in plan:
    print(f"{clock:2}-{clock + minutes:2} min  {step}")
    clock += minutes
print("total:", clock, "minutes, with every step covered")

Output:

 0- 5 min  1. Requirements: what, for whom
 5-10 min  2. Estimate: users, reads, writes
10-15 min  3. API: the few calls that matter
15-20 min  4. Data model: what we store
20-30 min  5. High-level design: the boxes
30-40 min  6. Deep dive: the hardest part
40-45 min  7. Trade-offs, bottlenecks, wrap-up
total: 45 minutes, with every step covered
  • Seven steps add up to exactly 45 minutes: 5 + 5 + 5 + 5 + 10 + 10 + 5.
  • {clock:2} prints the number in at least 2 characters, so the columns line up (0, 5, 10).
  • The minutes are a guide, not a law. If the interviewer wants to go deep somewhere, follow them. But you always know where you are on the clock.
  • The split is our own suggestion (our assumption), not an official rule. Early steps are short because they are quick to do and they decide everything after them.

You just invented an interview framework. The next sections fill in each step with Arjun's question.

Step 1: requirements (ask before you draw), 5 minutes

A live cricket score app could mean almost anything. Just the score? Ball-by-ball commentary? Live video? Chat? Arjun was about to design video streaming for nothing.

So ask. Say your questions out loud, and write the answers on the board:

You ask The interviewer says (our assumed answers)
Live score only, or commentary, video and chat too? Score and ball-by-ball only
How fresh must the score be? Seconds? A few seconds late is fine, about 10 s
How many viewers at the peak? One big match? A final: 1 crore (10 million) fans, all on the same match

Then write down what you agreed:

  • Functional: show the score and ball-by-ball updates, for any match.
  • Non-functional: updates within about 10 seconds; 1 crore viewers; it must never go down during a final.
  • Out of scope: video, chat, login.

  • A crore is 10 million (an Indian number word); a lakh is 100,000. 1 crore = 10,000,000.

  • Availability means the system keeps answering when parts fail. "Never down during a final" is an availability requirement.
  • Now the problem is small enough for 45 minutes. If the interviewer says "also video", you can say "then we need another 45 minutes", and you have shown you know what is big.

Step 2: estimate (put numbers on it), 5 minutes

A back-of-the-envelope estimate is a quick rough calculation, done by hand, to learn the size of a problem before designing it. The name comes from doing it on the back of an envelope. Example: "1 crore viewers asking every 10 seconds is about a million requests a second". It does not need to be exact; it needs to get the order of magnitude (is it 100 or 1,000,000?) right.

Terms used below:

  • A read is a request that asks for data without changing it. Example: the app asking "what is the score?".
  • A write is a request that changes data. Example: the scorer recording a ball.
  • Bandwidth is how much data per second moves through a connection, measured in bits per second (1 byte = 8 bits). Example: 4 Gbit/s (gigabits a second, a gigabit is a billion bits).
  • JSON is a plain-text format for data like {"runs": 12}. One score update is about 500 bytes of it (our guess).
  • 1_000_000 and 4 * 3600 in Python: underscores in a number are just for reading; 4 * 3600 is 4 hours in seconds.

Our numbers: 1 crore viewers, each app asks every 10 seconds; a T20 match is 2 innings of 20 overs of 6 balls, so 240 balls, over about 4 hours; one score update is 500 bytes.

viewers = 10_000_000           # 1 crore fans in a final (our guess)
refresh = 10                   # the app asks for the score every 10 s
balls = 240                    # a T20 match: 2 innings x 20 overs x 6
match_seconds = 4 * 3600       # about 4 hours (our guess)
score_bytes = 500              # one score update as JSON (our guess)

reads = viewers / refresh
writes = balls / match_seconds
print(f"reads:  {reads:,.0f} per second")
print(f"writes: {writes:.3f} per second (one ball every {1 / writes:.0f} s)")
print(f"read:write ratio = {reads / writes:,.0f} : 1")
print(f"bandwidth: {reads * score_bytes * 8 / 1e9:.0f} Gbit/s")

Output:

reads:  1,000,000 per second
writes: 0.017 per second (one ball every 60 s)
read:write ratio = 60,000,000 : 1
bandwidth: 4 Gbit/s
  • 10,000,000 ÷ 10 = 1,000,000 reads a second.
  • 240 ÷ 14,400 seconds = 0.0167, shown as 0.017 writes a second: one ball every 60 s. (Real cricket is not that even, but a rough average is all we need.)
  • The read:write ratio is how many reads for each write: 1,000,000 ÷ 0.01667 = 60,000,000 : 1.
  • Bandwidth: 1,000,000 × 500 bytes × 8 bits ÷ 1e9 = 4 Gbit/s. 1e9 is a billion. :,.0f means "thousands separator, no decimals".

That one number shapes the whole design. Writes are trivial; reads are huge, and they all want the same tiny piece of data. So this is a caching problem, not a database problem. Arjun spent 15 minutes on SQL versus NoSQL, which barely matters here.

  • A cache is a fast store that keeps a copy of data that is slow or costly to get, so repeated requests don't reach the real source. Example: keeping the latest score in memory instead of asking the database each time.
  • A database is the durable store where data lives. Example: the ball log below.

Estimation shortcuts (round it and move on)

Interviews have no calculator, so use shortcuts and say the numbers out loud:

  • A day has 86,400 seconds. Round it to 100,000. So 1 million things a day is about 10 a second by the shortcut; the exact answer is about 12.
  • Traffic is never flat. Peak = average x 2 to 4, and say that it is your guess.
  • Storage = items x size x days kept x copies. A copy (replica) is an extra saved version of the data, for safety. Example: 3 copies means 3 times the disk.
  • Don't chase exact answers. The point is the size: is it 10 a second or a million?
per_day = 1_000_000
print(f"exact:   {per_day / 86_400:.1f} a second")
print(f"rounded: {per_day / 100_000:.1f} a second  (day = 100,000 s)")
print(f"peak at 4x: {per_day / 86_400 * 4:.0f} a second")
print(f"storage: {per_day} x 500 B x 365 days x 3 copies = {per_day * 500 * 365 * 3 / 1e9:,.0f} GB")

Output:

exact:   11.6 a second
rounded: 10.0 a second  (day = 100,000 s)
peak at 4x: 46 a second
storage: 1000000 x 500 B x 365 days x 3 copies = 548 GB
  • The shortcut (10) and the exact value (11.6) differ by about 14%, far less than the guesses behind them (500 bytes, 4x), so rounding is safe. (The 500 bytes and 3 copies here are our assumptions.)

Steps 3 and 4: the API and the data model (before any boxes), 10 minutes

Before drawing boxes, decide what the boxes must do.

  • An API (application programming interface) here means the list of calls that programs can make to your system. Example: GET /matches/42/score.
  • A data model is what you store and how it is shaped. Example: one row per ball with an id, the over and the runs.
  • An append-only log is a list you only ever add to at the end and never edit. Example: a notebook where each ball is a new line. It is simple, easy to copy, and you can rebuild the score from it (like the change log in the file storage episode).
  • A cursor is a bookmark that says "I have everything up to here". Here, the id of the last ball the app has: since=4.

The data model: one row per ball, never edited. Each ball has an id, the over (like 0.4 = over 0, ball 4) and the runs. The API: one main call, GET /matches/42/score?since=. The app says "I already have up to ball 4", and the server sends only the newer balls:

# data model: one row per ball, never edited (an append-only log)
balls = [(1, "0.1", 1), (2, "0.2", 4), (3, "0.3", 0),
         (4, "0.4", 6), (5, "0.5", 1), (6, "0.6", 0)]   # (id, over, runs)

def get_score(since=0):           # API: GET /matches/42/score?since=
    new = [b for b in balls if b[0] > since]
    total = sum(runs for _, _, runs in balls)
    return {"runs": total, "last_ball": balls[-1][0], "new_balls": new}

first = get_score()
print("first open:", first["runs"], "runs,", len(first["new_balls"]), "balls sent")
later = get_score(since=4)
print("app had ball 4:", later["runs"], "runs, new:", later["new_balls"])

Output:

first open: 12 runs, 6 balls sent
app had ball 4: 12 runs, new: [(5, '0.5', 1), (6, '0.6', 0)]
  • First open: no since, so since=0: all 6 balls are sent, and the total is 1 + 4 + 0 + 6 + 1 + 0 = 12 runs.
  • Later, the app had ball 4: the total is still 12 runs, but only balls 5 and 6 come back: [(5, '0.5', 1), (6, '0.6', 0)].
  • sum(runs for _, _, runs in balls) adds up the third number of each tuple; the _ marks parts we ignore.
  • b[0] > since keeps only balls whose id is higher than the bookmark.
  • Why this is good: responses stay tiny (only new balls), the same request from a million apps with the same since gets the same answer (easy to cache!), and the log can never be corrupted by an edit.

Now every box you draw has a job: something writes balls, something serves get_score. And the interviewer has seen you think before drawing.

Step 5: the high-level design (the boxes, now with a reason), 10 minutes

A high-level design is the first picture of the whole system: the main boxes and the arrows between them, without inner detail. Example: scorer app, score service, database on one side; apps, CDN, cache and read service on the other.

 WRITE PATH (tiny)
   scorer app ──▶ score service ──▶ database (ball log, replicated)
   (at the ground)  (write a ball)

 READ PATH (huge)
   1 crore apps ──▶ CDN + cache ──▶ read service ──▶ database
   (every 10 s)     (latest score)   (GET score?since=)   (only on a miss)
  • Writes: tiny traffic, one path, so a replicated database (a database with copies for safety) is plenty.
  • Reads: huge, served from caches near users. Only on a cache miss (the cache doesn't have it) does the read service ask the database.
  • Notice what is missing: no Kafka, no Cassandra. Start simple; scale only what the numbers say needs scaling. Interviewers like that much more than a wall of boxes.
  • Every box has one sentence: what it does, and why it is there.

Step 6: a deep dive (the hardest part), 10 minutes

A deep dive is going into detail on one hard part of the design, instead of covering everything thinly. The interviewer often picks it, or you pick it. Example: "how do we serve a million reads a second for one match?".

Here the hard part is obvious: a million reads a second, all for the same key, the score of one match.

  • A key is the name you use to look up a piece of data. Example: match 42 score.
  • A hot key is one key that gets a huge share of all requests. Even a single cache server can melt under it, because the data lives on one machine.
  • A CDN (content delivery network) is a set of servers spread across many places, near users, that keep copies of content so requests are answered nearby (the earlier CDN episode, episode 21). Example: an edge server in Mumbai answering Mumbai users.
  • A TTL (time to live) here is how long an edge keeps a copy before asking again. Example: 2 seconds.
  • The origin is the real source behind the CDN; here, our read service and database.

Say we use 200 edge servers near users (our guess), each keeping the score for 2 seconds. Each edge then asks the origin once per TTL: 200 ÷ 2 = 100 requests a second:

reads = 1_000_000              # score requests a second (our estimate)
edges = 200                    # CDN edge servers near users (our guess)
ttl = 2                        # each edge keeps the score for 2 seconds

print(f"no cache: the database gets {reads:,} requests a second")
origin = edges / ttl           # each edge asks once per TTL
print(f"CDN, TTL {ttl} s: the database gets {origin:,.0f} a second")
print(f"that's {reads / origin:,.0f} times fewer")
print(f"cost: a score can be up to {ttl} s + 10 s refresh = {ttl + 10} s old")

Output:

no cache: the database gets 1,000,000 requests a second
CDN, TTL 2 s: the database gets 100 a second
that's 10,000 times fewer
cost: a score can be up to 2 s + 10 s refresh = 12 s old
  • No cache: 1,000,000 requests a second hit the database.
  • With the CDN: 100 a second, 10,000 times fewer (1,000,000 ÷ 100).
  • The last line is the price: a score can be up to 2 s + 10 s = 12 s old (the edge may have kept it 2 s, and the app may ask only every 10 s). That fits the "about 10 seconds" we agreed in step 1, give or take, so say it out loud. See how the steps connect?
  • This is a best-case, back-of-the-envelope figure: it assumes each edge sends exactly one request per TTL and ignores the network time. It is for sizing, not a promise.

Step 7: trade-offs, bottlenecks and failures, 5 minutes

A trade-off is a choice where you gain one thing and give up another. Example: a longer TTL means less load but older scores. A bottleneck is the part that limits the whole system. A failure is a part breaking. Say all three out loud:

Trade-off The choice and its price
Freshness vs load A 2 s TTL = up to 12 s old, but 10,000x fewer origin requests
Polling vs push Polling (apps ask every 10 s) is simple. Push (the server sends updates over a WebSocket, a connection kept open) is fresher, but means 1 crore open connections
Bottleneck: the scorer app Two scorers might send the same ball twice. Ball ids make a repeat harmless (idempotent: doing it twice has the same effect as once)
Failure: the CDN is down The cache and read service still answer, just with more load

The sentence interviewers love: "I chose X, because the numbers said Y." Every choice has a price, and saying it shows you understand your own design.

What interviewers look for (the scorecard)

An interviewer does not check whether you drew the same picture as theirs. These are the usual things they look for (our summary, not a rule from any company):

They check What it looks like
Did you clarify the problem? Questions and a written scope in step 1
Did numbers drive the design? "Reads are 60 million times writes, so it is a caching problem"
Is it simple and complete? Every box has a job; nothing extra
Can you go deep? One hard part, done properly
Do you know the trade-offs? Prices and failures named
Did you think out loud? Talked the whole time, and took hints

Treat it like designing with a colleague: explain, listen, and when they give a hint, use it.

The whole chain

  1. TimeoutError at minute 45 → a 7-step plan with a time budget.
  2. Almost designed video → requirements first.
  3. "How many users?" → an estimate: 60,000,000 : 1.
  4. Empty boxes → API and data model (ball log, since).
  5. A hot key → CDN, 10,000x fewer reads.
  6. "It's perfect" → trade-offs and failures.

Framework cheat sheet (one page)

All times are for a 45-minute round (our assumption). Scale them for 60 minutes (see the mini project).

# Step Minutes What to say Questions to ask
1 Requirements 0-5 "Let me clarify first. In scope: ... Out of scope: ..." Which features? Who uses it? How fresh or fast? How many users at the peak? What must never fail?
2 Estimate 5-10 "About X reads and Y writes a second, ratio Z:1, so this is a ... problem" How often does each user ask? How big is one item? How long does the data live?
3 API 10-15 "The main calls are ... with these inputs and outputs" Is the user always logged in? Any call that must be safe to repeat?
4 Data model 15-20 "We store ... one row per ..., keyed by ..." Do we edit data or only add? What do we look up by?
5 High-level design 20-30 "Write path ..., read path ... Start simple, scale where the numbers say" Is this level of detail OK, or shall I go into one part?
6 Deep dive 30-40 "The hardest part is ... Here is how it works and what it costs" Which part would you like me to go deeper on?
7 Trade-offs 40-45 "I chose X because Y. It gives up Z. If W fails, ..." Anything you would like me to revisit?

Habits for all seven: think out loud; write the numbers on the board; check the clock at each step; if stuck, say what you know and ask for a hint.

The framework on an earlier design: the URL shortener (episode 27)

The same plan works for any question. Take the URL shortener from episode 27 (a service that turns a long link into a short one and redirects). These numbers are our own assumptions.

  1. Requirements: turn a long link into a short one, and redirect. Ask about custom names and expiry.
  2. Estimate: say 1 crore (10 million) new links a day, about 116 a second, with reads 100 times more.
  3. API: POST /links (long link in, short code out) and GET /{code} (redirect).
  4. Data model: short code to long URL, plus owner, created and expires.
  5. Boxes: app servers, a key-value store (a database that finds a value by its key) and a cache.
  6. Deep dive: generating unique short codes, using the ID ideas from episode 23.
  7. Trade-offs: 301 vs 302 and how stale a cached link may be. A 301 is a permanent redirect: browsers may remember it, so repeat clicks skip our server (less load) but we lose click counts and can't change the target easily. A 302 is a temporary redirect: every click comes to us (counts and changes work) but with more load (general knowledge).

Step 2 as code:

links_a_day = 10_000_000      # 1 crore new short links a day (our assumption)
read_write = 100              # reads are 100x more (our assumption)
bytes_each = 500              # one stored link (our assumption)
writes = links_a_day / 86_400
reads = writes * read_write
print(f"writes: {writes:,.0f} per second")
print(f"reads:  {reads:,.0f} per second")
print(f"storage: {links_a_day * bytes_each * 365 / 1e12:,.1f} TB a year")

Output:

writes: 116 per second
reads:  11,574 per second
storage: 1.8 TB a year
  • 10,000,000 / 86,400 = 115.7, printed as 116 a second; x 100 = 11,574 reads a second; 10,000,000 x 500 bytes x 365 / 1e12 = 1.8 TB a year.
  • Reads are 100 times writes: a cache matters, but far less extreme than the cricket app's 60,000,000 : 1. Different numbers, different deep dive.

Common mistakes (and the real outputs)

Mistake What you see Why Fix
Start drawing boxes at minute 1 TimeoutError: time's up, never covered: requirements, estimate, API, data model, deep dive, trade-offs No plan, no clarifying Follow the 7 steps with a time budget
Skip the estimate read:write ratio = 60,000,000 : 1 (never computed) You spend 15 minutes on SQL vs NoSQL, the wrong problem Estimate reads, writes and storage first
Say "my design has no downsides" cost: a score can be up to 2 s + 10 s refresh = 12 s old Every choice has a price Name the trade-off and why it is OK
Forget * 8 (bytes to bits) bandwidth: 0 Gbit/s Network speed is in bits, sizes are in bytes Multiply bytes by 8
Ignore the since filter app had ball 4: 12 runs, new: [(1, '0.1', 1), (2, '0.2', 4), ...] (all 6 balls) The API resends everything every time Return only ids greater than since
Set a TTL of 0 ZeroDivisionError: division by zero (Practice 2) edges / ttl divides by zero; 0 means "no cache" Use a TTL above 0, or no CDN

The "forgot * 8" row, with the multiplication by 8 removed:

viewers = 10_000_000
refresh = 10
score_bytes = 500
reads = viewers / refresh
print(f"bandwidth: {reads * score_bytes / 1e9:.0f} Gbit/s")      # forgot x 8 (bytes to bits)

Output:

bandwidth: 0 Gbit/s

(1,000,000 × 500 ÷ 1e9 = 0.5, which prints as 0.) The "ignore since" row, with new = balls:

balls = [(1, "0.1", 1), (2, "0.2", 4), (3, "0.3", 0),
         (4, "0.4", 6), (5, "0.5", 1), (6, "0.6", 0)]

def get_score(since=0):
    new = balls                   # forgot to filter by since
    total = sum(runs for _, _, runs in balls)
    return {"runs": total, "last_ball": balls[-1][0], "new_balls": new}

later = get_score(since=4)
print("app had ball 4:", later["runs"], "runs, new:", later["new_balls"])

Output:

app had ball 4: 12 runs, new: [(1, '0.1', 1), (2, '0.2', 4), (3, '0.3', 0), (4, '0.4', 6), (5, '0.5', 1), (6, '0.6', 0)]

Mini project: an interview helper

A tool you can run before a mock interview. You give it the numbers you agreed in step 1 (watchers, how often each asks, writes, the time window, bytes per item, how fresh the data must be). It prints a time budget for a 45 or 60 minute round, then the step 2 estimates: reads and writes a second, the ratio, bandwidth, new data a year, a verdict (read-heavy means a cache or CDN is the deep dive), and the largest cache TTL that still meets your freshness target. It reuses the earlier ideas: the plan (Problem 1), the estimate (Step 2) and the TTL arithmetic (Step 6). All numbers in apps are assumptions you would agree with the interviewer.

# Mini project: an interview helper. Give it the numbers you agreed in step 1
# and it prints a time budget for your round and the step 2 estimates.
STEPS = [("Requirements", 5), ("Estimate", 5), ("API", 5), ("Data model", 5),
         ("High-level design", 10), ("Deep dive", 10), ("Trade-offs", 5)]  # 45 min

def plan(total=45):
    scale = total / 45                   # a 60-minute round? scale every step
    clock = 0
    for i, (name, mins) in enumerate(STEPS):
        mins = round(mins * scale)
        if i == len(STEPS) - 1:
            mins = total - clock         # the last step takes what is left
        print(f"  {clock:2}-{clock + mins:2} min  {name}")
        clock += mins
    print(f"  total {clock} minutes")

def estimate(app):
    reads = app["watchers"] / app["ask_every"]        # requests per second
    writes = app["writes"] / app["window"]            # writes per second
    mbit = reads * app["bytes"] * 8 / 1e6
    gb_day = app["writes"] * (86_400 / app["window"]) * app["bytes"] / 1e9
    print(f"  reads:  {reads:,.1f} per second")
    print(f"  writes: {writes:,.3f} per second")
    print(f"  ratio:  {reads / writes:,.0f} : 1")
    print(f"  bandwidth: {mbit:,.1f} Mbit/s")
    print(f"  new data: {gb_day:,.2f} GB a day, {gb_day * 365 / 1000:,.2f} TB a year")
    if reads / writes > 1000:
        print("  verdict: read-heavy and hot -> cache / CDN is the deep dive")
    else:
        print("  verdict: not extreme -> database and indexes need care")
    ttl = app["fresh"] - app["ask_every"]
    if ttl <= 0:
        print(f"  freshness: {app['fresh']} s is impossible when apps ask every "
              f"{app['ask_every']} s; ask more often or relax it")
    else:
        print(f"  freshness: use a cache TTL of at most {ttl} s")

apps = {   # every number is an assumption you agreed with the interviewer
 "Live cricket score": dict(watchers=10_000_000, ask_every=10, writes=240,
         window=4 * 3600, bytes=500, fresh=12),
 "Parking app": dict(watchers=50_000, ask_every=30, writes=200_000,
         window=86_400, bytes=300, fresh=60),
 "Parking app, fresh=20": dict(watchers=50_000, ask_every=30, writes=200_000,
         window=86_400, bytes=300, fresh=20),
}
print("A 45-minute round:"); plan(45)
print("A 60-minute round:"); plan(60)
for name, app in apps.items():
    print(f"\n{name}:"); estimate(app)

Output:

A 45-minute round:
   0- 5 min  Requirements
   5-10 min  Estimate
  10-15 min  API
  15-20 min  Data model
  20-30 min  High-level design
  30-40 min  Deep dive
  40-45 min  Trade-offs
  total 45 minutes
A 60-minute round:
   0- 7 min  Requirements
   7-14 min  Estimate
  14-21 min  API
  21-28 min  Data model
  28-41 min  High-level design
  41-54 min  Deep dive
  54-60 min  Trade-offs
  total 60 minutes

Live cricket score:
  reads:  1,000,000.0 per second
  writes: 0.017 per second
  ratio:  60,000,000 : 1
  bandwidth: 4,000.0 Mbit/s
  new data: 0.00 GB a day, 0.00 TB a year
  verdict: read-heavy and hot -> cache / CDN is the deep dive
  freshness: use a cache TTL of at most 2 s

Parking app:
  reads:  1,666.7 per second
  writes: 2.315 per second
  ratio:  720 : 1
  bandwidth: 4.0 Mbit/s
  new data: 0.06 GB a day, 0.02 TB a year
  verdict: not extreme -> database and indexes need care
  freshness: use a cache TTL of at most 30 s

Parking app, fresh=20:
  reads:  1,666.7 per second
  writes: 2.315 per second
  ratio:  720 : 1
  bandwidth: 4.0 Mbit/s
  new data: 0.06 GB a day, 0.02 TB a year
  verdict: not extreme -> database and indexes need care
  freshness: 20 s is impossible when apps ask every 30 s; ask more often or relax it

- **Time budget**: for 60 minutes every step is scaled by 60 ÷ 45 and rounded; the last step takes what is left, so the total is exactly 60 (without that fix, rounding gave 61 in our first try).
- **Cricket**: 60,000,000 : 1 and `TTL of at most 2 s`: the helper reaches the same answers as the video. New data is tiny: 240 × 500 bytes = 120,000 bytes per match, which rounds to `0.00 GB`.
- **Parking app** (our assumed numbers): 50,000 drivers looking at once, each app asking every 30 s = 1,666.7 reads a second; 200,000 bookings a day = 2.315 writes a second; ratio 720 : 1, so not extreme.
- **Freshness**: the maximum age is TTL + refresh time. With `fresh=20` and an app that asks only every 30 s, no TTL can work (even a TTL of 0 gives 30 s), so the helper says to ask more often or relax the requirement.

Try extending it: ask for the numbers with `input()`; add a peak multiplier (evenings 4x); add a storage line for data kept for years; print the questions to ask for each step.

## Practice

1. Homework: in the ESTIMATE demo, change the refresh to 3 seconds. What changes?
2. Homework: in the CDN demo, what TTL keeps scores at most 5 seconds fresh? (Also try TTL 0.)
3. Homework: do a full 45-minute mock for "design a parking app". Write the outline.
4. In the MODEL demo, the app already has ball 6 (the last one). What does `get_score(since=6)` return?
5. The interviewer says "now 10 crore viewers, 10 times more". What changes in the estimate, and in the CDN deep dive?
6. The interviewer is excited about caching and wants 20 minutes on it. Which steps do you shorten, and what must you never skip?
7. Why is "start simple" better than drawing Kafka and Cassandra?

<details markdown="1"><summary>Solution</summary>

**1.** Change `refresh = 10` to `refresh = 3`. Output:

reads: 3,333,333 per second writes: 0.017 per second (one ball every 60 s) read:write ratio = 200,000,000 : 1 bandwidth: 13 Gbit/s


Asking more often means more reads: 10,000,000 ÷ 3 = **3,333,333 a second**, 3.3 times more. Writes do not change, so the ratio grows to **200,000,000 : 1**, and bandwidth to **13 Gbit/s** (3,333,333 × 500 × 8 ÷ 1e9 = 13.3). The score is fresher for users but the load on the read side is 3.3 times bigger: a trade-off you can name in step 7.

**2.** The age of a score is at most `ttl + refresh`. With a 10 s refresh, even a TTL of 0 gives 10 s, so **no TTL can give 5 s**: the app itself asks too rarely. You must change the refresh as well (see Practice 1). This script tries both:

```python
reads = 1_000_000              # score requests a second (our estimate)
edges = 200                    # CDN edge servers near users (our guess)

for refresh in (10, 3):        # how often the app asks, in seconds
    for ttl in (2, 1, 0.5):    # how long each edge keeps the score
        age = ttl + refresh
        ok = "ok" if age <= 5 else "too old"
        print(f"refresh {refresh:2} s, TTL {ttl} s: up to {age} s old ({ok}), database gets {edges / ttl:,.0f} a second")

Output:

refresh 10 s, TTL 2 s: up to 12 s old (too old), database gets 100 a second
refresh 10 s, TTL 1 s: up to 11 s old (too old), database gets 200 a second
refresh 10 s, TTL 0.5 s: up to 10.5 s old (too old), database gets 400 a second
refresh  3 s, TTL 2 s: up to 5 s old (ok), database gets 100 a second
refresh  3 s, TTL 1 s: up to 4 s old (ok), database gets 200 a second
refresh  3 s, TTL 0.5 s: up to 3.5 s old (ok), database gets 400 a second

With a 3 s refresh, a TTL of 2 s gives exactly 5 s (any TTL up to 2 s works; a shorter TTL means fresher scores but more origin requests: 200 ÷ 0.5 = 400 a second). TTL 0 means "no cache", and the formula breaks:

reads = 1_000_000
edges = 200
ttl = 0
origin = edges / ttl
print(f"CDN, TTL {ttl} s: the database gets {origin:,.0f} a second")

Output:

Traceback (most recent call last):
  File "main.py", line 4, in <module>
    origin = edges / ttl
             ~~~~~~^~~~~
ZeroDivisionError: division by zero

Dividing by zero is an error in Python. In the real system a TTL of 0 means every request goes to the origin: all 1,000,000 (or 3,333,333) a second, so the cache would do nothing.

3. A full 45-minute mock outline for "Design a parking app" (finding and booking a parking spot near you). All numbers are our assumptions, to be agreed with the interviewer; the helper's numbers (50,000 drivers looking at once, refresh every 30 s, 200,000 bookings a day) are used below.

Minutes Step What you say and write
0-5 1. Requirements Ask: find free spots near me, book one, pay? Only for car parks that report free spots live, or any street? How fresh must "free spots" be (about a minute)? In scope: search, view free spots, book, cancel. Out of scope: payments, navigation, reviews. Non-functional: one spot is never booked twice; search is fast; 50,000 drivers looking at once at the peak.
5-10 2. Estimate 50,000 drivers ÷ 30 s = about 1,667 searches a second; 200,000 bookings a day ÷ 86,400 = about 2.3 bookings a second; ratio about 720 : 1. Data is small (about 0.06 GB of new bookings a day). "Reads are 700 times writes, so a cache helps, but the real danger is two people booking the same spot."
10-15 3. API GET /spots?lat=&lng=&radius= (free spots nearby); POST /bookings with spot_id, user_id and a booking id (safe to repeat); DELETE /bookings/{id}.
15-20 4. Data model lots (id, name, location), spots (id, lot_id, status), bookings (id, spot_id, user_id, from, to, status). Search by location needs an index on location. A booking is a new row, not an edit (like the ball log).
20-30 5. High-level design Apps → load balancer → search service (reads) with a cache → database; booking service (writes) → database with a transaction (all-or-nothing). Sensors or lot staff update spot status through a small write path.
30-40 6. Deep dive The hardest part is double booking: two drivers press "book" on the last spot at the same moment. Solution: a database transaction that checks the spot is free and marks it booked in one step, so only one wins; the loser sees "just taken". Cache the search results for a few seconds (the freshness we agreed was about a minute), but never cache the booking step.
40-45 7. Trade-offs A cached search may show a spot that was just taken (the price of speed): the booking step re-checks. Holding a spot for 5 minutes during payment helps the driver but locks the spot for others. Failure: if the booking service is down, search still works. Bottleneck: one busy lot at 9 am; we could shard by city.

Notice how the cricket app's deep dive (a hot read key) and this app's (correct writes) differ: the estimate decided it. Cricket was read-heavy, parking is balanced, so the interesting problem moved to correctness.

4. Output of get_score(since=6):

app had ball 6: {'runs': 12, 'last_ball': 6, 'new_balls': []}
app had ball 4: [(5, '0.5', 1), (6, '0.6', 0)]

(The script prints both calls.) With nothing newer than ball 6, new_balls is an empty list: the response is tiny and the app just keeps what it has. This is the common case between balls: most requests find nothing new.

5. Change viewers = 10_000_000 to 100_000_000. Output:

reads:  10,000,000 per second
writes: 0.017 per second (one ball every 60 s)
read:write ratio = 600,000,000 : 1
bandwidth: 40 Gbit/s

Reads grow 10 times, to 10,000,000 a second, and bandwidth to 40 Gbit/s. With the CDN, the origin does not feel it, because each edge still asks once per TTL (200 edges, 2 s):

reads = 10_000_000             # score requests a second with 10x the viewers
edges = 200
ttl = 2
origin = edges / ttl
print(f"CDN, TTL {ttl} s: the database gets {origin:,.0f} a second")
print(f"that's {reads / origin:,.0f} times fewer")

Output:

CDN, TTL 2 s: the database gets 100 a second
that's 100,000 times fewer

The database still gets 100 a second (100,000 times fewer than the reads). What grows is the CDN's own work, so you would add more edge servers and bandwidth. That is why a cache layer in front of a hot key scales so well.

6. Follow the interviewer: they are telling you where to go deep. Shorten step 5 (the boxes) and steps 3 and 4 (say them briefly), but never skip requirements (a minute is enough if you are sure), the estimate (one minute, it is what justifies the cache), and the trade-offs (even two sentences: the TTL's price and a failure case). A rule of thumb: you can compress a step, but you should not drop it.

7. Every extra box costs time to explain, adds more things that can fail, and must be justified by a number. At 0.017 writes a second, the write path needs nothing more than a replicated database; Kafka or Cassandra would be extra boxes with no reason behind them. Start simple, say which number would make you add them ("if writes reached thousands a second, I would add a queue"), and scale only what the numbers say.

Quick quiz

interviewer: Design a URL shortener.
you: ???

The interviewer says "Design a URL shortener". What is your first move?

  • A. Draw the database schema
  • B. Ask: how many links a day? custom names? expiry?
  • C. Pick Cassandra
  • D. Write the hash function
Answer

B. Clarify requirements and scale first: they decide everything that comes after. A schema, a database choice or a hash function before you know the numbers is exactly Arjun's mistake.

Interview questions: "How do you approach a system design question?"

1. Clarify requirements. Functional (what features), non-functional (how fast, how fresh, how many users, how reliable), and scope (what is out). Write them on the board and confirm. "If I understand correctly, we need ... and we will not cover ...".

2. Estimate. "Users × how often each asks ÷ seconds = reads a second; writes a day ÷ 86,400 = writes a second; the ratio tells me whether I am designing for reads or writes. Item size × reads gives bandwidth; item size × writes × time gives storage." For the cricket app, with our assumptions: 1,000,000 reads a second, 0.017 writes a second, 60,000,000 : 1, 4 Gbit/s.

3. Design. API and data model first; then the high-level design in a write path and a read path; every box with one sentence of why; start simple.

4. Deep dive. Pick the part the numbers say is hardest (the hot key; double booking), or the part the interviewer points to. Show how it works, what it costs and how it fails.

5. Trade-offs. Freshness vs load, polling vs push, consistency vs availability, simple vs scalable. "I chose X because the numbers said Y; it gives up Z."

Common follow-ups:

  • What if the number of users grows 10 times? (Redo the estimate; find which part grows: here only the CDN, the origin stays at 100 a second; see Practice 5.)
  • What if the interviewer disagrees with an assumption? (Change the number and redo the estimate out loud; the method matters more than the value.)
  • How do you handle a question you don't know? (Say what you do know, state an assumption, and keep going; ask for a hint.)
  • What would you do with more time? (Name the next deep dives: for the cricket app, push with WebSockets, multiple scorers, monitoring and alerts.)
  • Why not just use technology X? (Say what problem X solves, whether the numbers have that problem, and what X would cost.)
  • How do you know when to stop? (When the interviewer is satisfied with a step and the clock says move on; cover every step, then go deep.)
  • What would you monitor? (Origin requests a second, cache hit rate, how old the served score is, error rate.)

Glossary

Term Meaning
System design interview A 45 to 60 minute conversation where you sketch how to build a big product while the interviewer watches your thinking
Requirements What the system must do and how well
Functional requirement What the system does, e.g. "show the score and ball-by-ball updates"
Non-functional requirement How well it does it: speed, size, reliability, e.g. "never down during a final"
Scope / out of scope The line around what you will design / what you will not, e.g. no video, no chat
Availability The system keeps answering even when parts fail
Box / component One part of the design drawn as a rectangle, e.g. "database"
Kafka / Redis / Cassandra Tools for streaming messages / very fast in-memory storage and caching / a database spread across many machines
TimeoutError Python's error for "the time allowed ran out"
Traceback Python's report of where an error happened, innermost call last
List comprehension A short Python form that builds a list from another, e.g. [s for s in a if s not in b]
7-step plan Requirements, estimate, API, data model, high-level design, deep dive, trade-offs, with a time budget
Time budget Minutes assigned to each step, e.g. 5, 5, 5, 5, 10, 10, 5
Back-of-the-envelope estimate A quick rough calculation to learn the size of a problem
Order of magnitude The rough size of a number as a power of ten: 100 vs 1,000,000
Read / write A request that only gets data / one that changes data
Read:write ratio How many reads for each write, e.g. 60,000,000 : 1
Bandwidth Data moved per second, in bits per second (1 byte = 8 bits), e.g. 4 Gbit/s
JSON A plain-text format for data like {"runs": 12}
Crore / lakh 10 million / 100,000
T20 A cricket match of 20 overs per innings; an over is 6 balls, so a full match has 240 legal balls
Cache A fast store with a copy of data that is slow to get, so repeated requests skip the source
Database The durable store where the data lives
API The list of calls other programs can make to your system, e.g. GET /matches/42/score
Data model What you store and how it is shaped, e.g. one row per ball
Append-only log A list you only add to at the end and never edit
Cursor A bookmark meaning "I have everything up to here", e.g. since=4
High-level design The first picture of the whole system: main boxes and arrows
Write path / read path The route a change takes into the system / the route a request takes to get data out
Replicated database A database with copies, for safety
Cache miss The cache does not have the data, so the request goes on to the source
Deep dive Going into detail on one hard part of the design
Key / hot key The name used to look up data / one key that gets a huge share of all requests
CDN Content delivery network: servers spread near users that keep copies of content
Edge server A CDN server near users that answers requests nearby
Origin The real source behind a CDN or cache, e.g. the read service and database
TTL (time to live) How long a cache keeps a copy before asking again, e.g. 2 s
Trade-off A choice where you gain one thing and give up another
Bottleneck The part that limits the whole system
Polling / push The app asks again and again / the server sends updates when they happen
WebSocket A connection kept open so the server can send updates at any time
Idempotent Doing it twice has the same effect as doing it once, e.g. sending the same ball id twice
Transaction A group of database steps that all happen or none do, e.g. check a spot is free and book it
Index An extra lookup structure that makes searches on a column fast
301 / 302 redirect A permanent / a temporary redirect: 301 may be remembered by browsers, 302 sends every click to us
Key-value store A database that finds a value by its key, e.g. short code to long URL
Copy (replica) An extra saved version of the data, for safety
Scorecard What interviewers check: clarify, numbers, simple design, depth, trade-offs, thinking out loud

Summary

  • Knowledge without a plan runs out of time: Arjun's round ended in TimeoutError: time's up, never covered: requirements, estimate, API, data model, deep dive, trade-offs. Use a 7-step plan for 45 minutes: 5, 5, 5, 5, 10, 10, 5 (our suggested split; follow the interviewer if they want depth).
  • Step 1, requirements: ask what, for whom, how fresh, how many; write functional, non-functional and out of scope. Do not design video for nothing.
  • Step 2, estimate: 1 crore viewers asking every 10 s = 1,000,000 reads a second against 0.017 writes a second: 60,000,000 : 1, 4 Gbit/s. One number says: a caching problem, not a SQL-vs-NoSQL problem.
  • Steps 3 and 4: the API (GET /matches/42/score?since=) and the data model (an append-only ball log) before any box. First open: 12 runs, 6 balls; with since=4: 12 runs and only balls 5 and 6.
  • Step 5: write path small, read path through CDN and cache; start simple.
  • Step 6, deep dive: a hot key; a CDN with 200 edges and a 2 s TTL gives the database 100 a second, 10,000 times fewer, for a score up to 12 s old (TTL + refresh). For 5 s you need a 3 s refresh and a TTL of at most 2 s.
  • Step 7: name the trade-offs, bottlenecks and failures: "I chose X because the numbers said Y".
  • Interviewers look for: clarifying, numbers driving the design, simple and complete, depth, trade-offs, and thinking out loud.

Next episode: the final one, a capstone where we design one last system using the whole course, and a roadmap for what to learn next.

Sources: none. No external sources were used: this episode states no facts about real systems, companies or interviews. Our own stated assumptions: the 45-minute round and the time split, 1 crore (10 million) viewers, the 10-second refresh, the 4-hour match, the 500-byte score, 200 CDN edge servers, the 2-second TTL, the URL shortener and parking app numbers, and all names. The cricket rules (T20 = 20 overs per innings, 6 balls per over) and what CDNs, caches, TTLs, WebSockets, transactions, indexes, Kafka, Redis and Cassandra are is general knowledge.