SyntaxPilot
Courses/AI: Zero to Agents/Lesson 3

AI: Zero to Agents · Lesson 3 of 40 · 10 min

Types of learning

scheduled

These are the notes for AI episode 3. In episode 2 Raju always knew the right answer for every row of his notebook. Here that assumption breaks three times, and each time you invent a new way for a machine to learn: supervised, unsupervised (clustering), reinforcement (rewards, explore vs exploit) and self-supervised learning, which is how large language models pretrain. Every Python example below runs as-is in the SyntaxPilot compiler, and the outputs shown are real.

Supervised learning: learning with an answer key

Think of a student with a practice worksheet:

In a classroom In machine learning
Practice questions (the worksheet) Features: the inputs
The answer key at the back of the book Labels: the right outputs
The student answers, checks the key, corrects themself Supervised learning: learning from labelled examples

Raju's notebook in episode 2 was exactly this: weather and time (features) plus what each customer actually ordered (label).

Supervised = learning with the answer key.

Problem 1: no labels

Business is good, and Raju wants to send different offers to different kinds of customers: a loyalty card for some, a party discount for others. His new notebook has two numbers per customer: cups per week and average bill in rupees. But nobody ever wrote down which kind of customer each person is. There is no answer column: no labels.

The obvious move is to reuse the supervised trainer from episode 2.

Demo 1: supervised code crashes

# Raju's new notebook: (cups per week, average bill in Rs)
# No label column: nobody wrote down "which group"
customers = [(14, 20), (12, 18), (15, 22), (2, 160),
             (3, 150), (13, 21), (1, 170), (2, 140)]

# Try the supervised trainer from episode 2
for cups, bill, label in customers:
    print(cups, bill, label)

Output:

Traceback (most recent call last):
  File "main.py", line 7, in <module>
    for cups, bill, label in customers:
        ^^^^^^^^^^^^^^^^^
ValueError: not enough values to unpack (expected 3, got 2)

Python looked for the label and found nothing. Supervised learning needs the answer key, and this data has none. We need a way to learn without any answers.

The fix: find groups (unsupervised learning)

Think of sorting a pile of laundry:

  1. No shirt has a tag saying which pile it belongs in (data with no labels).
  2. You put similar things together: whites here, darks there (group by similarity).
  3. Piles appear on their own; nobody named them first. These piles are clusters.
  4. Only afterwards do you name the piles: "office shirts", "gym wear" (meaning comes after).

Learning patterns from data with no labels is called unsupervised learning.

Demo 2: clustering in code

customers = [(14, 20), (12, 18), (15, 22), (2, 160),
             (3, 150), (13, 21), (1, 170), (2, 140)]
def dist(p, q):
    return ((p[0] - q[0]) ** 2 + (p[1] - q[1]) ** 2) ** 0.5

def mean(group):
    return (sum(p[0] for p in group) / len(group),
            sum(p[1] for p in group) / len(group))

centres = [customers[0], customers[1]]   # 2 starting guesses
for step in range(5):
    groups = [[], []]
    for c in customers:   # each customer joins the nearest centre
        nearest = min([0, 1], key=lambda i: dist(c, centres[i]))
        groups[nearest].append(c)
    centres = [mean(g) for g in groups]   # move centres to the middle

for g in groups:
    print(len(g), "customers:", g)

Output:

4 customers: [(2, 160), (3, 150), (1, 170), (2, 140)]
4 customers: [(14, 20), (12, 18), (15, 22), (13, 21)]

How it works:

  • dist measures how far apart two customers are, like measuring a straight line on a graph.
  • Start with two rough guesses for the middle of each pile (centres).
  • Every customer joins the nearest centre; then each centre moves to the middle (mean) of its group. Repeat five times.

One group comes rarely but spends around ₹150 each time; the other buys about 13 cups a week with a ₹20 bill. The code never knew the words regulars or party orders. It only found the groups. Raju looks at them and decides: daily regulars get a loyalty card, bulk buyers get a party discount. (This "join the nearest centre, move the centre, repeat" loop is the idea behind the well-known k-means algorithm, here with k = 2.)

What unsupervised learning gives you

Input Data with features only, no labels
Output Structure, such as groups (clusters)
Who gives meaning The machine finds the groups; a human decides what they mean
Used for Customer segments, similar songs in a music app, transactions that don't fit any normal group (fraud checks)

Problem 2: no data at all

Raju opens a second stall at the railway station. This time the notebook is completely empty: no data, no labels. He can make one snack per customer: samosa, vada pav or poha. Which sells best here? He has to find out by doing.

The obvious way: samosa sold on day one, so stick with samosa forever.

Demo 3: sticking with one choice fails

import random
random.seed(7)
# True chance that a customer buys each snack (Raju can't see this)
chance = {"samosa": 0.3, "vada pav": 0.8, "poha": 0.5}

def sell(snack):          # reward: 1 if it sold, 0 if not
    return 1 if random.random() < chance[snack] else 0

# Obvious way: samosa sold on day 1? Stick with it forever.
pick = "samosa"
sold = sum(sell(pick) for day in range(100))
print("always", pick, "-> sold", sold, "of 100")

Output:

always samosa -> sold 33 of 100
  • chance is hidden from Raju: it is how the station's customers behave. Vada pav is secretly the favourite.
  • sell returns a reward: 1 if the customer buys, 0 if not.
  • random.seed(7) makes the random numbers the same on every run, so you get the same output.

No error, no crash: Raju just never found out that vada pav sells far better, because he never tried it. To learn without data, you have to act, see the result, and sometimes try something new.

The fix: rewards (reinforcement learning)

Think of training a dog with treats:

Training a dog Reinforcement learning
The dog decides what to do Agent: the learner that acts
Sit, jump, bark Actions: the choices it can make
A treat (or nothing) after each try Reward: a score for the action
Repeats what earned treats, but still tries new tricks Explore vs exploit

Nobody shows the dog the right answer; it is only told how well it did. Using the best-known choice is exploiting. Trying something new is exploring. Learning by acting and collecting rewards is reinforcement learning (RL).

Demo 4: reinforcement learning in code

import random
random.seed(7)
chance = {"samosa": 0.3, "vada pav": 0.8, "poha": 0.5}
def sell(snack):
    return 1 if random.random() < chance[snack] else 0
value = {s: 0.0 for s in chance}   # Raju's guess per snack
tries = {s: 0 for s in chance}
sold = 0
for day in range(100):
    if random.random() < 0.1:              # explore: try anything
        snack = random.choice(list(chance))
    else:                                  # exploit: best so far
        snack = max(value, key=value.get)
    reward = sell(snack)
    tries[snack] += 1
    value[snack] += (reward - value[snack]) / tries[snack]
    sold += reward
print("tries:", tries)
print("sold", sold, "of 100")

Output:

tries: {'samosa': 12, 'vada pav': 82, 'poha': 6}
sold 78 of 100

How it works:

  • value is Raju's current guess of how often each snack sells; tries counts how often he has made it.
  • One time in ten (random.random() < 0.1) he explores and picks any snack; otherwise he exploits the snack with the best guess so far.
  • After each sale, value[snack] += (reward - value[snack]) / tries[snack] nudges the guess towards what just happened. (It keeps a running average of the rewards.)

78 sales instead of 33, and vada pav was made 82 times. Raju found the favourite himself, with no data and no labels, just rewards.

At a giant scale: AlphaGo

The same idea, at a giant scale, is behind AlphaGo. In March 2016 it beat the Go champion Lee Sedol four games to one. It first studied human games, then improved by playing against copies of itself and learning from wins and losses.

Problem 3: labels cost too much

Raju's nephew wants a keyboard app that guesses your next word. The supervised way: for every phrase, a person types the correct next word.

"one cutting"  -> chai
"cutting chai" -> please
... a person types millions of these

For millions of sentences that is impossibly slow and expensive. But look at any sentence: the answer is already in it. After one cutting, the next word is chai. The text labels itself. Hide the next word and ask the model to guess it.

Demo 5: self-supervised learning in code

text = ("one cutting chai please . one cutting chai please . "
        "one vada pav please . two cutting chai please .")
words = text.split()
# The labels come free: every next word is the answer
pairs = [(words[i], words[i + 1]) for i in range(len(words) - 1)]
print(len(pairs), "training pairs, 0 typed by hand")
print(pairs[:4])

# Learn: count which word follows which
follows = {}
for word, nxt in pairs:
    follows.setdefault(word, []).append(nxt)

def predict(word):
    options = follows[word]
    return max(set(options), key=options.count)

print("cutting ->", predict("cutting"))
print("one ->", predict("one"))

Output:

19 training pairs, 0 typed by hand
[('one', 'cutting'), ('cutting', 'chai'), ('chai', 'please'), ('please', '.')]
cutting -> chai
one -> cutting

The code pairs every word with the word that follows it: the next word is the label, and nobody typed it. Creating the labels from the data itself is called self-supervised learning. Large language models learn this way in pretraining: guess the next word in huge amounts of text, check it against the real next word, and update.

All four types side by side

Type Where the answers come from Example from this episode
Supervised An answer key: labelled examples Chai or cold coffee predictor (episode 2)
Unsupervised No answers: find groups and structure Raju's customer segments, laundry piles
Reinforcement No data to start: act, get rewards, explore vs exploit The station snack stall, AlphaGo
Self-supervised Labels made from the data itself Next-word keyboard, LLM pretraining

The question to ask is always: where do the answers come from?

Glossary

Term Meaning
Supervised learning Learning from examples that come with the right answers (labels)
Unsupervised learning Finding structure in data that has no labels
Cluster A group of similar data points found without labels
Clustering Putting similar items into groups (here: nearest centre, move centre, repeat)
Centre (centroid) The middle point of a cluster
Reinforcement learning Learning by acting and receiving rewards
Agent The learner that chooses actions
Action A choice the agent can make (samosa, vada pav, poha)
Reward A score after each action (1 if sold, 0 if not)
Explore Try something new to learn about it
Exploit Use the best-known choice so far
Self-supervised learning The data creates its own labels (the next word is the answer)
Pretraining An LLM's first training stage: predicting the next word in huge amounts of text

Common mistakes

Mistake Code Error
Supervised code on data with no label column for cups, bill, label in customers: ValueError: not enough values to unpack (expected 3, got 2)
More starting cluster centres than data points random.sample(customers, 5) with 3 customers ValueError: Sample larger than population or is negative
An agent that never explores pick = "samosa" forever no error, just sold 33 of 100

The second one in full:

import random
customers = [(14, 20), (12, 18), (3, 150)]
centres = random.sample(customers, 5)
Traceback (most recent call last):
  File "main.py", line 3, in <module>
    centres = random.sample(customers, 5)
  File ".../random.py", line 434, in sample
    raise ValueError("Sample larger than population or is negative")
ValueError: Sample larger than population or is negative

The number of clusters must be at most the number of rows. For the first mistake: either label the data first, or switch to unsupervised learning. For the third: keep some exploration, for example try something random one time in ten.

Misconceptions

  • "Unsupervised learning names the groups." It only finds them. A human gives them meaning.
  • "Reinforcement learning is shown the right answer." It is only told how good its action was (a reward).
  • "Self-supervised means no data is needed." It needs lots of raw data; it just doesn't need humans to label it.

Practice

  1. Add a customer (8, 60) to the clustering demo. Which group does it join?
  2. In Demo 4, change the explore chance from 0.1 to 0.0. What happens?
  3. Add a fourth snack "idli" with chance 0.9 to Demo 4 and rerun. Does Raju find it?
  4. Add your own sentences to the text in Demo 5 and call predict("chai").
  5. Name the type of learning: (a) a bank flags transactions that don't fit any normal group, (b) a game bot gets +1 for winning, (c) an email filter trained on emails marked spam/not spam.
Solutions and hints

1. It joins the frequent, low-bill group (it is much closer to bills around ₹20 than ₹150):

4 customers: [(2, 160), (3, 150), (1, 170), (2, 140)]
5 customers: [(14, 20), (12, 18), (15, 22), (13, 21), (8, 60)]

That group's centre moves to about (12.4, 28.2).

2. With 0.0 Raju never explores. All guesses start at 0.0, so max picks the first snack, samosa, and it stays the best guess for ever:

tries: {'samosa': 100, 'vada pav': 0, 'poha': 0}
sold 35 of 100

(35 rather than 33 because this version calls random.random() once more per day for the explore check, so the random sequence is different.)

3. Yes. With chance = {"samosa": 0.3, "vada pav": 0.8, "poha": 0.5, "idli": 0.9} and explore 0.1:

tries: {'samosa': 5, 'vada pav': 3, 'poha': 1, 'idli': 91}
sold 87 of 100

4. Example: add "chai and bun please . chai and bun please .". Now follows["chai"] is ['please', 'please', 'please', 'and', 'and'] and predict("chai") returns please. If two words tie for most common, the pick can change between runs, because the order of a set of strings is not fixed in Python.

5. (a) unsupervised, (b) reinforcement, (c) supervised.

Quick quiz

A music app has 1 crore (10 million) listeners. Nobody labels them. The app puts listeners with similar playlists into taste groups. Which type of learning is this?

  • A. Supervised
  • B. Unsupervised
  • C. Reinforcement
  • D. Self-supervised
Answer

B, unsupervised. There are no labels and the goal is to find groups of similar listeners: that is clustering, just like Raju's customers.

Interview questions

  • What is the difference between supervised and unsupervised learning? Give one example of each.
  • Explain how a simple clustering algorithm like k-means works.
  • What are agent, action and reward in reinforcement learning?
  • What is the explore vs exploit trade-off? What happens if an agent never explores?
  • What is self-supervised learning, and why is it so useful for training LLMs?
  • How would you decide which type of learning to use for a new problem? (Ask where the answers come from.)

Summary

  • Supervised: learn from labelled examples (an answer key).
  • No labels crashed the trainer, so we used unsupervised learning: clustering joins the nearest centre, moves it, and repeats.
  • No data at all: reinforcement learning learns from actions and rewards, balancing explore and exploit.
  • Labels too costly to type: self-supervised learning uses the next word as a free label, which is how LLMs pretrain.

Next episode: so far Raju's models answered with a choice: chai or cold coffee. But his milk supplier asks "how many cups will you sell tomorrow?" That's a number, not a choice. Classification versus regression.