SyntaxPilot
Courses/AI: Zero to Agents/Lesson 4

AI: Zero to Agents · Lesson 4 of 40 · 10 min

Classification vs regression

scheduled (upload queued)

These are the notes for AI episode 4. Inside supervised learning, one simple question decides almost everything: what kind of answer does the model give, a choice or a number? A choice from a fixed menu is classification; a number is regression. You will also see why right/wrong counting fails for numbers, invent mean absolute error, train a straight-line regression model in plain Python, and learn how classifiers use scores and a threshold. Every Python example below runs as-is in the SyntaxPilot compiler, and the outputs shown are real.

Classification: answering from a menu

Raju's model from episode 2 answered from his menu board.

At Raju's stall In machine learning
The menu board: chai · cold coffee · lassi Classes: the fixed set of possible answers
"Chai or cold coffee?" (two options) Binary classification: exactly 2 classes
"Chai, coffee or lassi?" (three or more) Multi-class classification: 3 or more classes
Your inbox: spam or not spam Classification: pick one class

Classification = pick one answer from a menu.

The problem: the milkman's question

Every evening Raju's milkman asks: how much milk do you need tomorrow? To answer, Raju needs to know how many cups of chai he will sell. That answer is not on any menu: it is a number, like 56, 84 or 96 cups.

  • Guess too high, and the extra milk goes sour.
  • Guess too low, and customers leave without their chai.

So the guess should be as close as possible.

Demo 1: menu thinking fails

First instinct: what is the "average order" of the day?

orders = ["chai", "cold coffee", "chai", "lassi", "chai"]

# "Average order" of the day?
print(sum(orders) / len(orders))

Output:

Traceback (most recent call last):
  File "main.py", line 4, in <module>
    print(sum(orders) / len(orders))
          ~~~^^^^^^^^
TypeError: unsupported operand type(s) for +: 'int' and 'str'

sum starts from the number 0 and tries 0 + "chai". Python can't add chai to cold coffee: classes are names, not amounts. There is no average of chai and lassi. For cups, the answer must be a number. But how do we judge a model whose answers are numbers?

Demo 2: right/wrong scoring fails for numbers

# How many cups of chai were sold (the truth)
actual = [60, 60, 60, 60]
model_a = [59, 61, 61, 59]    # always off by about 1
model_b = [20, 100, 5, 140]   # wildly off

def accuracy(pred, true):     # right/wrong counting
    right = sum(p == t for p, t in zip(pred, true))
    return right / len(true)

print("model A accuracy:", accuracy(model_a, actual))
print("model B accuracy:", accuracy(model_b, actual))

Output:

model A accuracy: 0.0
model B accuracy: 0.0

A model that is off by one cup gets the same score as one that is off by a hundred. Right/wrong ignores how far off a guess is. For numbers, being close matters.

The fix: measure the gap (mean absolute error)

A quiz question A darts board
Right or wrong, nothing in between A throw close to the centre still scores
Works for classes Works for numbers: closeness counts
  1. For every guess, measure the gap: |guess − truth| (ignore the minus sign).
  2. Average all the gaps.

That average is the mean absolute error (MAE). Smaller = better.

Demo 3: mean absolute error

actual = [60, 60, 60, 60]
model_a = [59, 61, 61, 59]
model_b = [20, 100, 5, 140]

def mae(pred, true):          # mean absolute error
    gaps = [abs(p - t) for p, t in zip(pred, true)]
    return sum(gaps) / len(gaps)

print("model A is off by", mae(model_a, actual), "cups on average")
print("model B is off by", mae(model_b, actual), "cups on average")

Output:

model A is off by 1.0 cups on average
model B is off by 53.75 cups on average

Model B's gaps are 40, 40, 55 and 80, which average to 53.75. Now the score tells the truth: A is far better. This is the loss from episode 2, rebuilt for numbers: count mistakes for classes, measure gaps for numbers.

Regression: hot day, fewer cups

Raju pulls six days from his notebook: the temperature and how many cups of chai he sold, from (14 °C, 96 cups) to (34 °C, 57 cups). The hotter the day, the fewer the cups. A good guess for the model is a straight line:

cups = start − drop × temp
Part Role
Temperature Feature
The line (start, drop) Model
Number of cups Target: in regression the label is often called the target

Demo 4: train a regression model

# (temperature in °C, cups of chai sold that day)
days = [(14, 96), (18, 88), (22, 81), (26, 72), (30, 63), (34, 57)]

def predict(temp, start, drop):    # a straight line
    return start - drop * temp

def mae(start, drop):
    return sum(abs(predict(t, start, drop) - cups)
               for t, cups in days) / len(days)

# Training: try many lines, keep the one with the smallest error
best = min(((s, d / 10) for s in range(100, 141) for d in range(0, 41)),
           key=lambda line: mae(*line))
print("learned line: cups =", best[0], "-", best[1], "x temp")
print("average error:", round(mae(*best), 2), "cups")

# Inference: tomorrow will be 20 °C
print("20 °C ->", round(predict(20, *best)), "cups")

Output:

learned line: cups = 124 - 2.0 x temp
average error: 0.5 cups
20 °C -> 84 cups

How it works:

  • The loss is the mean absolute error over all six days.
  • Training is exactly like episode 2: try many lines (every start from 100 to 140, every drop from 0.0 to 4.0 in steps of 0.1) and keep the one with the smallest error. min(..., key=...) returns the line with the lowest mae.
  • mae(*line) unpacks the pair (start, drop) into the two arguments.
  • Inference: for tomorrow's 20 °C forecast the line gives 84 cups, so Raju orders milk for 84 cups.

Predicting a number from features is called regression.

Problem: negative cups (extrapolation)

A model is just maths, and maths doesn't know about chai.

def predict(temp):              # the line Raju learned
    return 124 - 2.0 * temp

print("20 °C ->", predict(20), "cups")
print("40 °C ->", predict(40), "cups")

# The weather app switched to Fahrenheit: 70 °F (about 21 °C)
print("70 ->", predict(70), "cups")

Output:

20 °C -> 84.0 cups
40 °C -> 44.0 cups
70 -> -16.0 cups

One morning the weather app quietly switches to Fahrenheit and sends 70 (only about 21 °C). The model orders negative milk, with no error and no crash. A regression line keeps going for ever. It never saw anything above 34 °C, yet it answers with full confidence. Predicting far outside your data is called extrapolation, and it is risky. Even the 40 °C answer is a guess, because the hottest day in the notebook was 34 °C.

A line never says "I don't know".

The fix: guard the input and the output

def predict(temp):
    if not 0 <= temp <= 50:     # outside anything Raju has seen
        raise ValueError(f"{temp} doesn't look like °C")
    return max(0, round(124 - 2.0 * temp))   # never below 0 cups

print("20 °C ->", predict(20), "cups")
print("70 ->", predict(70), "cups")

Output:

20 °C -> 84 cups
Traceback (most recent call last):
  File "main.py", line 7, in <module>
    print("70 ->", predict(70), "cups")
                   ~~~~~~~^^^^
  File "main.py", line 3, in predict
    raise ValueError(f"{temp} doesn't look like °C")
ValueError: 70 doesn't look like °C
  • Refuse inputs that don't look like anything Raju has seen (below 0 or above 50).
  • Round to whole cups and never go below zero.

A loud error beats a silent wrong answer: Raju can fix the app before ordering milk. Real AI systems do the same: the model makes the guess, and normal code around it checks that the input and the output make sense.

Choice or number? Side by side

Classification Regression
Answer One class from a fixed menu A number
Examples chai / cold coffee, spam / not spam 84 cups, a flat's monthly rent (₹18,500)
Loss Count mistakes Measure the gap (e.g. MAE)
Question word "Which one?" "How much?" / "How many?"

Quick trick: if the question starts with how much or how many, it is usually regression.

The same notebook can feed both. Is tomorrow a busy day or a quiet day? is a class. How many cups exactly? is a number. The data doesn't decide the task; the question you ask does.

Scores and thresholds

Many classifiers don't jump straight to a class. They first output a score between 0 and 1, like how sure they are that the answer is cold coffee. Then a threshold turns the score into a class:

Customer Score Threshold Class
Priya, 31 °C 0.91 > 0.5 ? yes cold coffee
Amit, 16 °C 0.12 > 0.5 ? no chai

The score is a number, but the final answer is still a class, so this is still classification. What matters is the kind of answer you need at the end.

Demo 5: multi-class scores

# A multi-class model gives one score per class
scores = {"chai": 0.15, "cold coffee": 0.70, "lassi": 0.15}

# The answer is the class with the highest score
answer = max(scores, key=scores.get)
print("scores:", scores)
print("answer:", answer)
print("total:", sum(scores.values()))

Output:

scores: {'chai': 0.15, 'cold coffee': 0.7, 'lassi': 0.15}
answer: cold coffee
total: 1.0

With more than two classes the model gives one score per class, and the highest score wins. Scores like these are often probabilities that add up to 1. Numbers inside, a class out: still classification.

Glossary

Term Meaning
Class One possible answer from a fixed set (chai, cold coffee, lassi)
Classification Predicting which class an input belongs to
Binary classification Exactly 2 classes (spam / not spam)
Multi-class classification 3 or more classes
Regression Predicting a number from features
Target The number a regression model predicts (its label)
Accuracy Fraction of predictions that are exactly right
Mean absolute error (MAE) The average of |guess − truth|
Extrapolation Predicting far outside the range of the training data
Score A number (often 0 to 1) a classifier gives before choosing a class
Threshold The cut-off that turns a score into a class (e.g. 0.5)

Common mistakes

Mistake Code Error
Doing maths on classes sum(["chai", "cold coffee", "lassi"]) TypeError: unsupported operand type(s) for +: 'int' and 'str'
Feeding a class name into a regression float("chai") ValueError: could not convert string to float: 'chai'
Judging numbers as right/wrong accuracy([59, 61, 61, 59], [60, 60, 60, 60]) no error, just 0.0
Trusting the line outside the data predict(70) no error, just -16.0 cups

Fixes: count classes (the most common one), don't add them. Classes need a classifier, or must be turned into numbers first. Use a gap-based error such as MAE for numbers. Guard inputs and outputs around a regression model.

Misconception: "If a model outputs numbers, it's regression." Not always: a classifier's scores are numbers too, but its final answer is a class.

Practice

  1. Use the learned line cups = 124 - 2.0 x temp to predict 10 °C and 40 °C. Do the answers make sense?
  2. Add a day (38, 49) to the data in Demo 4 and retrain. Does the line change?
  3. Write the most common order of the day with max() and count().
  4. A model C guesses [60, 60, 60, 100] for actual = [60, 60, 60, 60]. Compare its accuracy and MAE with model A. Which metric would you trust, and why?
  5. Name two classification and two regression tasks from apps you use.
Solutions and hints

1. 10 °C gives 104.0 cups and 40 °C gives 44.0 cups. Both are outside the training range (14 to 34 °C), so both are extrapolations. They look reasonable, but the model has no evidence for them.

2. It stays cups = 124 - 2.0 x temp; the average error becomes 0.57 cups. The new day (38 °C, 49 cups) is close to what the line predicts (48), so the best line doesn't change.

3.

orders = ["chai", "cold coffee", "chai", "lassi", "chai"]
print(max(set(orders), key=orders.count))
print(orders.count("chai"), "of", len(orders))
chai
3 of 5

4.

actual = [60, 60, 60, 60]
model_a = [59, 61, 61, 59]
model_c = [60, 60, 60, 100]

def mae(pred, true):
    gaps = [abs(p - t) for p, t in zip(pred, true)]
    return sum(gaps) / len(gaps)

def accuracy(pred, true):
    return sum(p == t for p, t in zip(pred, true)) / len(true)

print("A:", accuracy(model_a, actual), mae(model_a, actual))
print("C:", accuracy(model_c, actual), mae(model_c, actual))
A: 0.0 1.0
C: 0.75 10.0

Accuracy says C is better; MAE says A is better. For cups, a 40-cup miss is a real problem, so MAE tells the truth.

5. Classification: spam filter, face unlock (you or not you). Regression: delivery time in minutes, a taxi fare estimate.

Quick quiz 1

Four apps, four models. Which one is a regression task?

  • A. Spam or not spam
  • B. Next month's flat rent
  • C. Cat or dog photo
  • D. Movie genre
Answer

B, flat rent. It is a how much question, so the answer is a number. The other three pick a class from a fixed set.

Quick quiz 2

Same weather data: humidity, clouds, wind. Q1: Will it rain tomorrow? (yes / no). Q2: How many mm of rain tomorrow?

  • A. Q1 and Q2 are both classification
  • B. Q1 classification, Q2 regression
  • C. Q1 regression, Q2 classification
  • D. Both are regression
Answer

B. Yes/no is a class, so Q1 is binary classification. Millimetres of rain is a number, so Q2 is regression. Same data, but the kind of answer decides the task.

Interview questions

  • What is the difference between classification and regression? Give two examples of each.
  • What is the difference between binary and multi-class classification?
  • Why is accuracy a poor metric for a regression model? What would you use instead?
  • What is mean absolute error, and how do you compute it?
  • What is extrapolation, and why is it risky? How would you protect a production model from bad inputs?
  • A classifier outputs 0.91. How does that become a class? (A threshold, e.g. 0.5.)

Summary

  • Classification picks one class from a fixed menu: binary = 2 classes, multi-class = 3 or more.
  • Classes can't be added (TypeError), and right/wrong scoring gave a near-perfect number model 0.
  • Mean absolute error = the average gap between guess and truth.
  • Regression predicts a number (the target); we trained a line cups = 124 - 2.0 x temp.
  • Lines extrapolate for ever: guard regression inputs and outputs (no negative cups).
  • Classifiers often output scores, then a threshold or the highest score picks the class.

Next episode: counting mistakes worked fine for classes. But what if one kind of mistake is much worse than the other, like a medical test that misses a sick person? Measuring a model: accuracy, precision, recall, F1 and the confusion matrix.