These are the notes for AI episode 4. Inside supervised learning, one simple question decides almost everything: what kind of answer does the model give, a choice or a number? A choice from a fixed menu is classification; a number is regression. You will also see why right/wrong counting fails for numbers, invent mean absolute error, train a straight-line regression model in plain Python, and learn how classifiers use scores and a threshold. Every Python example below runs as-is in the SyntaxPilot compiler, and the outputs shown are real.
Classification: answering from a menu
Raju's model from episode 2 answered from his menu board.
| At Raju's stall | In machine learning |
|---|---|
| The menu board: chai · cold coffee · lassi | Classes: the fixed set of possible answers |
| "Chai or cold coffee?" (two options) | Binary classification: exactly 2 classes |
| "Chai, coffee or lassi?" (three or more) | Multi-class classification: 3 or more classes |
| Your inbox: spam or not spam | Classification: pick one class |
Classification = pick one answer from a menu.
The problem: the milkman's question
Every evening Raju's milkman asks: how much milk do you need tomorrow? To answer, Raju needs to know how many cups of chai he will sell. That answer is not on any menu: it is a number, like 56, 84 or 96 cups.
- Guess too high, and the extra milk goes sour.
- Guess too low, and customers leave without their chai.
So the guess should be as close as possible.
Demo 1: menu thinking fails
First instinct: what is the "average order" of the day?
orders = ["chai", "cold coffee", "chai", "lassi", "chai"]
# "Average order" of the day?
print(sum(orders) / len(orders))
Output:
Traceback (most recent call last):
File "main.py", line 4, in <module>
print(sum(orders) / len(orders))
~~~^^^^^^^^
TypeError: unsupported operand type(s) for +: 'int' and 'str'
sum starts from the number 0 and tries 0 + "chai". Python can't add chai to cold coffee: classes are names, not amounts. There is no average of chai and lassi. For cups, the answer must be a number. But how do we judge a model whose answers are numbers?
Demo 2: right/wrong scoring fails for numbers
# How many cups of chai were sold (the truth)
actual = [60, 60, 60, 60]
model_a = [59, 61, 61, 59] # always off by about 1
model_b = [20, 100, 5, 140] # wildly off
def accuracy(pred, true): # right/wrong counting
right = sum(p == t for p, t in zip(pred, true))
return right / len(true)
print("model A accuracy:", accuracy(model_a, actual))
print("model B accuracy:", accuracy(model_b, actual))
Output:
model A accuracy: 0.0
model B accuracy: 0.0
A model that is off by one cup gets the same score as one that is off by a hundred. Right/wrong ignores how far off a guess is. For numbers, being close matters.
The fix: measure the gap (mean absolute error)
| A quiz question | A darts board |
|---|---|
| Right or wrong, nothing in between | A throw close to the centre still scores |
| Works for classes | Works for numbers: closeness counts |
- For every guess, measure the gap:
|guess − truth|(ignore the minus sign). - Average all the gaps.
That average is the mean absolute error (MAE). Smaller = better.
Demo 3: mean absolute error
actual = [60, 60, 60, 60]
model_a = [59, 61, 61, 59]
model_b = [20, 100, 5, 140]
def mae(pred, true): # mean absolute error
gaps = [abs(p - t) for p, t in zip(pred, true)]
return sum(gaps) / len(gaps)
print("model A is off by", mae(model_a, actual), "cups on average")
print("model B is off by", mae(model_b, actual), "cups on average")
Output:
model A is off by 1.0 cups on average
model B is off by 53.75 cups on average
Model B's gaps are 40, 40, 55 and 80, which average to 53.75. Now the score tells the truth: A is far better. This is the loss from episode 2, rebuilt for numbers: count mistakes for classes, measure gaps for numbers.
Regression: hot day, fewer cups
Raju pulls six days from his notebook: the temperature and how many cups of chai he sold, from (14 °C, 96 cups) to (34 °C, 57 cups). The hotter the day, the fewer the cups. A good guess for the model is a straight line:
cups = start − drop × temp
| Part | Role |
|---|---|
| Temperature | Feature |
The line (start, drop) |
Model |
| Number of cups | Target: in regression the label is often called the target |
Demo 4: train a regression model
# (temperature in °C, cups of chai sold that day)
days = [(14, 96), (18, 88), (22, 81), (26, 72), (30, 63), (34, 57)]
def predict(temp, start, drop): # a straight line
return start - drop * temp
def mae(start, drop):
return sum(abs(predict(t, start, drop) - cups)
for t, cups in days) / len(days)
# Training: try many lines, keep the one with the smallest error
best = min(((s, d / 10) for s in range(100, 141) for d in range(0, 41)),
key=lambda line: mae(*line))
print("learned line: cups =", best[0], "-", best[1], "x temp")
print("average error:", round(mae(*best), 2), "cups")
# Inference: tomorrow will be 20 °C
print("20 °C ->", round(predict(20, *best)), "cups")
Output:
learned line: cups = 124 - 2.0 x temp
average error: 0.5 cups
20 °C -> 84 cups
How it works:
- The loss is the mean absolute error over all six days.
- Training is exactly like episode 2: try many lines (every
startfrom 100 to 140, everydropfrom 0.0 to 4.0 in steps of 0.1) and keep the one with the smallest error.min(..., key=...)returns the line with the lowestmae. mae(*line)unpacks the pair(start, drop)into the two arguments.- Inference: for tomorrow's 20 °C forecast the line gives 84 cups, so Raju orders milk for 84 cups.
Predicting a number from features is called regression.
Problem: negative cups (extrapolation)
A model is just maths, and maths doesn't know about chai.
def predict(temp): # the line Raju learned
return 124 - 2.0 * temp
print("20 °C ->", predict(20), "cups")
print("40 °C ->", predict(40), "cups")
# The weather app switched to Fahrenheit: 70 °F (about 21 °C)
print("70 ->", predict(70), "cups")
Output:
20 °C -> 84.0 cups
40 °C -> 44.0 cups
70 -> -16.0 cups
One morning the weather app quietly switches to Fahrenheit and sends 70 (only about 21 °C). The model orders negative milk, with no error and no crash. A regression line keeps going for ever. It never saw anything above 34 °C, yet it answers with full confidence. Predicting far outside your data is called extrapolation, and it is risky. Even the 40 °C answer is a guess, because the hottest day in the notebook was 34 °C.
A line never says "I don't know".
The fix: guard the input and the output
def predict(temp):
if not 0 <= temp <= 50: # outside anything Raju has seen
raise ValueError(f"{temp} doesn't look like °C")
return max(0, round(124 - 2.0 * temp)) # never below 0 cups
print("20 °C ->", predict(20), "cups")
print("70 ->", predict(70), "cups")
Output:
20 °C -> 84 cups
Traceback (most recent call last):
File "main.py", line 7, in <module>
print("70 ->", predict(70), "cups")
~~~~~~~^^^^
File "main.py", line 3, in predict
raise ValueError(f"{temp} doesn't look like °C")
ValueError: 70 doesn't look like °C
- Refuse inputs that don't look like anything Raju has seen (below 0 or above 50).
- Round to whole cups and never go below zero.
A loud error beats a silent wrong answer: Raju can fix the app before ordering milk. Real AI systems do the same: the model makes the guess, and normal code around it checks that the input and the output make sense.
Choice or number? Side by side
| Classification | Regression | |
|---|---|---|
| Answer | One class from a fixed menu | A number |
| Examples | chai / cold coffee, spam / not spam | 84 cups, a flat's monthly rent (₹18,500) |
| Loss | Count mistakes | Measure the gap (e.g. MAE) |
| Question word | "Which one?" | "How much?" / "How many?" |
Quick trick: if the question starts with how much or how many, it is usually regression.
The same notebook can feed both. Is tomorrow a busy day or a quiet day? is a class. How many cups exactly? is a number. The data doesn't decide the task; the question you ask does.
Scores and thresholds
Many classifiers don't jump straight to a class. They first output a score between 0 and 1, like how sure they are that the answer is cold coffee. Then a threshold turns the score into a class:
| Customer | Score | Threshold | Class |
|---|---|---|---|
| Priya, 31 °C | 0.91 | > 0.5 ? yes | cold coffee |
| Amit, 16 °C | 0.12 | > 0.5 ? no | chai |
The score is a number, but the final answer is still a class, so this is still classification. What matters is the kind of answer you need at the end.
Demo 5: multi-class scores
# A multi-class model gives one score per class
scores = {"chai": 0.15, "cold coffee": 0.70, "lassi": 0.15}
# The answer is the class with the highest score
answer = max(scores, key=scores.get)
print("scores:", scores)
print("answer:", answer)
print("total:", sum(scores.values()))
Output:
scores: {'chai': 0.15, 'cold coffee': 0.7, 'lassi': 0.15}
answer: cold coffee
total: 1.0
With more than two classes the model gives one score per class, and the highest score wins. Scores like these are often probabilities that add up to 1. Numbers inside, a class out: still classification.
Glossary
| Term | Meaning |
|---|---|
| Class | One possible answer from a fixed set (chai, cold coffee, lassi) |
| Classification | Predicting which class an input belongs to |
| Binary classification | Exactly 2 classes (spam / not spam) |
| Multi-class classification | 3 or more classes |
| Regression | Predicting a number from features |
| Target | The number a regression model predicts (its label) |
| Accuracy | Fraction of predictions that are exactly right |
| Mean absolute error (MAE) | The average of |guess − truth| |
| Extrapolation | Predicting far outside the range of the training data |
| Score | A number (often 0 to 1) a classifier gives before choosing a class |
| Threshold | The cut-off that turns a score into a class (e.g. 0.5) |
Common mistakes
| Mistake | Code | Error |
|---|---|---|
| Doing maths on classes | sum(["chai", "cold coffee", "lassi"]) |
TypeError: unsupported operand type(s) for +: 'int' and 'str' |
| Feeding a class name into a regression | float("chai") |
ValueError: could not convert string to float: 'chai' |
| Judging numbers as right/wrong | accuracy([59, 61, 61, 59], [60, 60, 60, 60]) |
no error, just 0.0 |
| Trusting the line outside the data | predict(70) |
no error, just -16.0 cups |
Fixes: count classes (the most common one), don't add them. Classes need a classifier, or must be turned into numbers first. Use a gap-based error such as MAE for numbers. Guard inputs and outputs around a regression model.
Misconception: "If a model outputs numbers, it's regression." Not always: a classifier's scores are numbers too, but its final answer is a class.
Practice
- Use the learned line
cups = 124 - 2.0 x tempto predict 10 °C and 40 °C. Do the answers make sense? - Add a day
(38, 49)to the data in Demo 4 and retrain. Does the line change? - Write the most common order of the day with
max()andcount(). - A model C guesses
[60, 60, 60, 100]foractual = [60, 60, 60, 60]. Compare its accuracy and MAE with model A. Which metric would you trust, and why? - Name two classification and two regression tasks from apps you use.
Solutions and hints
1. 10 °C gives 104.0 cups and 40 °C gives 44.0 cups. Both are outside the training range (14 to 34 °C), so both are extrapolations. They look reasonable, but the model has no evidence for them.
2. It stays cups = 124 - 2.0 x temp; the average error becomes 0.57 cups. The new day (38 °C, 49 cups) is close to what the line predicts (48), so the best line doesn't change.
3.
orders = ["chai", "cold coffee", "chai", "lassi", "chai"]
print(max(set(orders), key=orders.count))
print(orders.count("chai"), "of", len(orders))
chai
3 of 5
4.
actual = [60, 60, 60, 60]
model_a = [59, 61, 61, 59]
model_c = [60, 60, 60, 100]
def mae(pred, true):
gaps = [abs(p - t) for p, t in zip(pred, true)]
return sum(gaps) / len(gaps)
def accuracy(pred, true):
return sum(p == t for p, t in zip(pred, true)) / len(true)
print("A:", accuracy(model_a, actual), mae(model_a, actual))
print("C:", accuracy(model_c, actual), mae(model_c, actual))
A: 0.0 1.0
C: 0.75 10.0
Accuracy says C is better; MAE says A is better. For cups, a 40-cup miss is a real problem, so MAE tells the truth.
5. Classification: spam filter, face unlock (you or not you). Regression: delivery time in minutes, a taxi fare estimate.
Quick quiz 1
Four apps, four models. Which one is a regression task?
- A. Spam or not spam
- B. Next month's flat rent
- C. Cat or dog photo
- D. Movie genre
Answer
B, flat rent. It is a how much question, so the answer is a number. The other three pick a class from a fixed set.
Quick quiz 2
Same weather data: humidity, clouds, wind. Q1: Will it rain tomorrow? (yes / no). Q2: How many mm of rain tomorrow?
- A. Q1 and Q2 are both classification
- B. Q1 classification, Q2 regression
- C. Q1 regression, Q2 classification
- D. Both are regression
Answer
B. Yes/no is a class, so Q1 is binary classification. Millimetres of rain is a number, so Q2 is regression. Same data, but the kind of answer decides the task.
Interview questions
- What is the difference between classification and regression? Give two examples of each.
- What is the difference between binary and multi-class classification?
- Why is accuracy a poor metric for a regression model? What would you use instead?
- What is mean absolute error, and how do you compute it?
- What is extrapolation, and why is it risky? How would you protect a production model from bad inputs?
- A classifier outputs 0.91. How does that become a class? (A threshold, e.g. 0.5.)
Summary
- Classification picks one class from a fixed menu: binary = 2 classes, multi-class = 3 or more.
- Classes can't be added (
TypeError), and right/wrong scoring gave a near-perfect number model 0. - Mean absolute error = the average gap between guess and truth.
- Regression predicts a number (the target); we trained a line
cups = 124 - 2.0 x temp. - Lines extrapolate for ever: guard regression inputs and outputs (no negative cups).
- Classifiers often output scores, then a threshold or the highest score picks the class.
Next episode: counting mistakes worked fine for classes. But what if one kind of mistake is much worse than the other, like a medical test that misses a sick person? Measuring a model: accuracy, precision, recall, F1 and the confusion matrix.