SyntaxPilot
Courses/AI: Zero to Agents/Lesson 2

AI: Zero to Agents · Lesson 2 of 40 · 8 min

How machines learn

scheduled

These are the notes for AI episode 2. In episode 1 we said that in machine learning the computer learns from examples. Here you find out how that learning actually happens, at Raju's chai stall. On the way you invent data, features, labels, a model, loss, training and inference, and see why bad data gives confidently wrong answers. Every Python example below runs as-is in the SyntaxPilot compiler, and the outputs shown are real.

The problem: Raju's chai stall

Raju runs a chai stall at a college gate. Amit always has chai, Priya always has cold coffee, and Raju remembers everyone's order.

Then the college grows to 500 students, with new faces every day. And today Neha arrives on her first day. She is not in Raju's memory at all. What should he get ready?

A computer runs into exactly this problem. The obvious approach: make it memorise everyone's order.

Demo 1: memorising fails

# Raju memorises every regular's order
orders = {"Amit": "chai", "Priya": "cold coffee", "Ravi": "chai"}

print(orders["Amit"])
print(orders["Neha"])

Output:

chai
Traceback (most recent call last):
  File "main.py", line 5, in <module>
    print(orders["Neha"])
          ~~~~~~^^^^^^^^
KeyError: 'Neha'

Amit gets chai, but Neha crashes the program because she isn't in the dictionary. Memorising is not learning. Memory only knows old cases. Learning means catching a pattern that also works on new cases; this is called generalising.

The fix: spot a pattern (Raju's notebook)

Raju starts writing every order in a notebook, one line per customer.

Raju's notebook In machine learning
Every page of orders, one line per customer Data / dataset: many examples (rows)
Weather and time: things he can see before the order Features: the inputs we measure
What the customer ordered Label: the answer we want to predict
His rule of thumb: hot day → cold coffee Model: the learned pattern

Within a few weeks Raju sees the pattern hot day means cold coffee, cool day means chai. Nobody told him this rule: he got it from the data.

Features and labels in code

One row of data looks like this:

(31, 13, "cold coffee")
Value Role
31 feature: temperature in °C
13 feature: hour of day (1 PM)
"cold coffee" label
  • Features = what you know before: the inputs, often written as capital X in ML books.
  • Label = the answer you want to predict: the output, often written as small y.
  • Model = a pattern that maps features → label.

Next problem: how hot is "hot"?

Raju's rule is: if the temperature is above a cutoff, cold coffee; otherwise chai.

if temp > cutoff:  cold coffee
else:             chai

But which cutoff? 15 °C? 25? 30? The obvious way is to guess, but then how do you know whether a guess is good? You can't improve what you can't measure. We need to turn the mistakes into a number.

Demo 2: count the mistakes (loss)

data = [
    (14, 8, "chai"), (18, 9, "chai"), (22, 17, "chai"),
    (31, 13, "cold coffee"), (35, 15, "cold coffee"),
    (28, 8, "chai"), (33, 12, "cold coffee"), (26, 18, "chai"),
]

def predict(temp, cutoff):
    return "cold coffee" if temp > cutoff else "chai"

for cutoff in [15, 25, 30]:
    mistakes = sum(predict(t, cutoff) != label for t, h, label in data)
    print("cutoff", cutoff, "-> mistakes:", mistakes)

Output:

cutoff 15 -> mistakes: 4
cutoff 25 -> mistakes: 2
cutoff 30 -> mistakes: 0
  • Each row is (temperature, hour, label); for t, h, label in data unpacks it.
  • predict(t, cutoff) != label is True (counted as 1) when the model is wrong, so sum(...) counts the mistakes.

The number that says how wrong a model is is called the loss. Lower loss = better model. Here the loss is simply the number of mistakes.

The fix: training

We only tried three guesses. What if the right cutoff is 27? Think of how Raju gets the sugar right:

  1. Add one spoon (try a value).
  2. Taste it: how far from perfect? (measure the loss)
  3. Two spoons, three spoons (adjust and repeat).
  4. Keep the amount that tastes best (keep the smallest loss).

Try, measure the loss, keep the best. That loop is called training, and a computer can run it far faster than us.

Demo 3: train, then predict

# Data: each row = features -> label
# (temperature in °C, hour of day) -> what they ordered
data = [
    (14, 8, "chai"), (18, 9, "chai"), (22, 17, "chai"),
    (31, 13, "cold coffee"), (35, 15, "cold coffee"),
    (28, 8, "chai"), (33, 12, "cold coffee"), (26, 18, "chai"),
]

def predict(temp, cutoff):
    return "cold coffee" if temp > cutoff else "chai"

# Training: try every cutoff, count mistakes, keep the best
best, best_mistakes = None, len(data)
for cutoff in range(10, 40):
    mistakes = sum(predict(t, cutoff) != label for t, h, label in data)
    if mistakes < best_mistakes:
        best, best_mistakes = cutoff, mistakes
print("learned cutoff:", best, "mistakes:", best_mistakes)

# Inference: a brand-new customer on a 34 °C afternoon
print("prediction:", predict(34, best))

Output:

learned cutoff: 28 mistakes: 0
prediction: cold coffee

We never wrote 28 anywhere. The computer found it from the data. Then it answered for a brand-new customer (34 °C) who is not in the data at all.

Note: the hour feature is in the data, but this tiny model only uses temperature. Using several features together comes in later episodes.

Why 28 and not 29 or 30? Cutoffs 28, 29 and 30 all give zero mistakes. The loop only replaces best when it finds strictly fewer mistakes (<), so the first zero-mistake cutoff, 28, is kept.

Training vs inference

Phase Input What happens Output Exam analogy
Training Data with labels Try, measure loss, adjust A model (cutoff = 28) Practising with the answer key
Inference New features, no label The model predicts A prediction (34 °C → cold coffee) The real exam: questions you've never seen

When you ask a chatbot a question, that is inference too.

Data (features + labels) -> Training (minimise loss) -> Model (cutoff = 28) -> Inference (34 °C -> cold coffee)

That whole flow is the machine learning pipeline.

Next problem: one-sided data

Suppose Raju's new helper learned only from the May notebook, when everyone ordered cold coffee. Then a cold December morning arrives: 10 °C.

Demo 4: bad data, wrong answer

# Trained only on summer days: every row is cold coffee
summer = [(34, "cold coffee"), (36, "cold coffee"), (38, "cold coffee")]

labels = [label for t, label in summer]
guess = max(set(labels), key=labels.count)

print("10 °C winter morning ->", guess)

Output:

10 °C winter morning -> cold coffee

This model learns the most common label. For a freezing morning it says cold coffee: no error, no crash, just confidently wrong. The bug is not in the code, it is in the data. A model learns only what the data shows: garbage in, garbage out. Only when the data covers every kind of real case does the model become trustworthy.

Glossary

Term Meaning
Data / dataset All the examples; each row is one example
Example (row) One record, e.g. (31, 13, "cold coffee")
Feature (X) An input you know beforehand: temperature, hour
Label (y) The answer you want to predict: the order
Model The learned pattern from features → label (here, a cutoff)
Loss A number for how wrong the model is (here, mistakes counted)
Training Adjusting the model to make the loss as small as possible
Inference Using the trained model on new data with no label
Generalisation Working correctly on new cases, not just memorised ones
Garbage in, garbage out Bad or one-sided data gives a bad model

Common mistakes

A row with no label

data = [(14, "chai"), (31, "cold coffee"), (18)]
for temp, label in data:
    print(temp, label)
14 chai
31 cold coffee
Traceback (most recent call last):
  File "main.py", line 2, in <module>
    for temp, label in data:
        ^^^^^^^^^^^
TypeError: cannot unpack non-iterable int object

(18) is just the number 18 (brackets without a comma don't make a tuple), and it has no label, so it can't be split into (temp, label). Every training row needs its features and its label: clean the data first.

Memorising instead of learning

orders["Neha"] gave KeyError: 'Neha'. Storing answers only covers old cases; a model learns a pattern from features instead.

One-sided data (no error at all)

Training on May and predicting in December gave cold coffee at 10 °C. This is the most dangerous mistake because nothing crashes. Collect data that covers all the situations the model will face.

Testing on the training data only

A model that is perfect on its own training rows may still fail on new ones (like the memorising dictionary, which is "perfect" on Amit). Always check a model on examples it has not seen.

Practice

  1. Add four winter rows such as (10, "chai") to the summer list in Demo 4. Does the answer change? Is the model now good?
  2. Change the training range in Demo 3 to range(0, 50). Is the learned cutoff the same? Why?
  3. Add the row (29, 14, "cold coffee") to the data in Demo 3 and retrain. What cutoff is learned now?
  4. Pick an app you use. What are its features, and what is its label?
Solutions and hints

1. With four winter rows added there are 4 chai rows and 3 cold coffee rows, so the answer becomes chai:

summer = [(34, "cold coffee"), (36, "cold coffee"), (38, "cold coffee"),
          (10, "chai"), (12, "chai"), (9, "chai"), (11, "chai")]
labels = [label for t, label in summer]
guess = max(set(labels), key=labels.count)
print("10 °C winter morning ->", guess)
print("40 °C summer afternoon ->", guess)
10 °C winter morning -> chai
40 °C summer afternoon -> chai

Better for winter, but now it says chai at 40 °C too. This "most common label" model ignores the temperature completely. Better data helps, but the model also has to use the features.

2. Yes, still 28 with 0 mistakes. Cutoffs 28, 29 and 30 all make zero mistakes, and the loop keeps the first one it finds. Starting the range lower only adds worse cutoffs.

3. Still 28 with 0 mistakes: 29 > 28, so the new row is predicted as cold coffee correctly. But now only 28 gives zero mistakes (29 and 30 would get the new row wrong). More data narrowed down the answer.

4. Example: a food delivery app predicting delivery time. Features: distance, time of day, restaurant, weather. Label: the actual minutes taken.

Quick quiz

A model predicts the price of a flat from its area (sq ft), bedrooms and city. One row is (950, 2, "Pune", 6500000). Which one is the label?

  • A. area
  • B. bedrooms
  • C. price
  • D. city
Answer

C, price. The label is the answer we want to predict. Area, bedrooms and city are known beforehand, so they are features.

Interview questions

  • What is the difference between a feature and a label? Give an example.
  • What is a model, in one sentence?
  • What is a loss function, and why do we need one?
  • What is the difference between training and inference?
  • Why is memorising the training data not the same as learning? (Generalisation.)
  • What does "garbage in, garbage out" mean for ML? Give an example of biased or one-sided data.

Summary

  • Memorising crashes on new cases (KeyError), so we learn a pattern instead.
  • Data = examples; features = inputs (X); label = the answer (y).
  • Model = the learned pattern from features → label.
  • Loss = how wrong the model is; training = adjusting to minimise the loss.
  • Inference = predicting for brand-new data with no label.
  • Garbage in, garbage out: the data must cover real life.

Next episode: Raju always had labels: he knew who ordered what. What if there are no labels at all, or the machine has to learn by trying things itself? Types of learning: supervised, unsupervised and reinforcement.