Machine Learning Full Course · Episode 3. Every code sample below was run with Python 3.13 and NumPy 2.5.3; the output shown is the real output. No maths background is assumed beyond adding, multiplying and squaring numbers.
The problem: "show me similar flats"
Neha likes one flat on a rental app: 900 sq ft, 4 km from her office, Rs 25,000 a month. The app has 2,000 other flats, and she taps show me similar. A computer has no idea what "similar" means, so we must turn the idea into numbers and a formula. That translation is at the heart of machine learning.
Scalars, vectors, components and dimension
Describe each flat with three numbers, always in the same order: size (× 100 sq ft), distance (km), rent (× Rs 1,000).
liked = [ 9, 4, 25]
A = [12, 1, 25]
B = [11, 6, 21]
| Term | Meaning | Example |
|---|---|---|
| Scalar | one plain number | 25 |
| Vector | an ordered list of numbers describing one thing | [9, 4, 25] |
| Component | one entry of a vector | the 4 (distance) |
| Dimension | how many components a vector has | 3 for our flats; a 1-megapixel greyscale photo: 1,000,000 |
Order matters: position 0 is always size, position 1 always distance. Swap them and the numbers mean nothing.
The obvious similarity measure fails
Add up the differences, component by component:
# flat = [size (100 sq ft), distance (km), rent (thousand Rs)]
liked = [9, 4, 25]
a = [12, 1, 25]
b = [11, 6, 21]
def diff(p, q):
total = 0
for i in range(3):
total += p[i] - q[i]
return total
print("A:", diff(liked, a))
print("B:", diff(liked, b))
Output:
A: 0
B: 0
Both flats look "identical" to the liked one, which is nonsense. For A, the differences are −3 (size), +3 (distance), 0 (rent): positive and negative differences cancel.
And plain Python lists aren't vectors:
liked = [9, 4, 25]
a = [12, 1, 25]
print(liked + a)
print(liked - a)
[9, 4, 25, 12, 1, 25]
TypeError: unsupported operand type(s) for -: 'list' and 'list'
+ glues lists together; - isn't defined at all. We need NumPy arrays and a difference measure where signs can't cancel.
Vector addition and scalar multiplication
Real-life example: chai stall orders as vectors [chai, samosa, cake].
- Vector addition: two friends' orders on one bill: add matching components. Both vectors must have the same dimension.
- Scalar multiplication: "same order again, for 3 tables": multiply every component by the scalar.
import numpy as np
order1 = np.array([2, 1, 0]) # chai, samosa, cake
order2 = np.array([1, 2, 1])
print(order1 + order2) # vector addition
print(3 * order1) # scalar multiplication
[3 3 1]
[6 3 0]
Norms: the length of a vector
Real-life example: getting across a city of square blocks, 3 blocks east and 3 north.
| City picture | Formula for v = (v₁, v₂, …) | (3, 4) | (−3, 3, 0) | |
|---|---|---|---|---|
| L1 norm (Manhattan, taxicab) | an auto rickshaw driving along the blocks | |v₁| + |v₂| + … | 7 | 6 |
| L2 norm (Euclidean) | a pigeon flying straight | √(v₁² + v₂² + …) | 5 | √18 ≈ 4.24 |
- Absolute value |x| = the size of a number without its sign: |−3| = 3.
- The L2 formula is Pythagoras' theorem extended to any number of dimensions.
- A norm is the length (size) of a vector, written ‖v‖. Plain "norm" usually means L2.
- Both remove signs: absolute values directly, squares because (−3)² = 9.
- L2 squares the differences, so one big difference counts more than several small ones.
import numpy as np
v = np.array([3, 4])
print(np.sqrt((v ** 2).sum()))
print(np.linalg.norm(v))
5.0
5.0
np.linalg is NumPy's linear algebra module (the maths of vectors and matrices).
Distance between vectors
Distance(p, q) = norm of (p − q): subtract, then measure the length of the result.
import numpy as np
liked = np.array([9, 4, 25])
a = np.array([12, 1, 25])
b = np.array([11, 6, 21])
print(liked - a, liked - b)
print("L1:", np.abs(liked - a).sum(), np.abs(liked - b).sum())
print("L2:", np.linalg.norm(liked - a).round(2), np.linalg.norm(liked - b).round(2))
[-3 3 0] [-2 -2 4]
L1: 6 8
L2: 4.24 4.9
No more fake zeros: by both measures, flat A is closer to what Neha liked. Distances like these power k-nearest neighbours (episode 27), k-means clustering (episode 43) and recommendation systems.
Units warning: rent is in thousands and size in hundreds of sq ft. If rent were in plain rupees (25,000), it would swamp every other feature. Putting features on comparable scales is feature scaling (episode 16).
The dot product
Real-life example: a shopping bill. Quantities q = [2, 3, 1] (chai, samosa, cake), prices p = [15, 20, 60].
Dot product q · p = q₁p₁ + q₂p₂ + q₃p₃ = 2×15 + 3×20 + 1×60 = 150. Multiply matching components, then add: two vectors in, one scalar out.
import numpy as np
qty = np.array([2, 3, 1]) # chai, samosa, cake
price = np.array([15, 20, 60]) # Rs per item
print(qty * price) # item by item
print(np.dot(qty, price), qty @ price) # the bill
[30 60 60]
150 150
*multiplies element by element (no adding).np.dot(a, b)anda @ bcompute the dot product.- Every linear model computes a dot product: features · weights, added up (episode 18). Neural networks are built from millions of them.
Different dimensions fail:
import numpy as np
qty = np.array([2, 3, 1])
price = np.array([15, 20])
print(np.dot(qty, price))
ValueError: shapes (3,) and (2,) not aligned: 3 (dim 0) != 2 (dim 0)
Angle, cosine similarity and unit vectors
The next problem. Asha orders [1, 1, 0] (chai, samosa, cake) per visit; Ravi orders [6, 6, 0] for his office (same taste, bigger appetite); Meena orders [0, 1, 2]. Who is most like Asha?
import numpy as np
asha = np.array([1, 1, 0]) # chai, samosa, cake per visit
ravi = np.array([6, 6, 0])
meena = np.array([0, 1, 2])
def dist(p, q):
return np.linalg.norm(p - q)
def cosine(p, q):
return p @ q / (np.linalg.norm(p) * np.linalg.norm(q))
print("distance ravi:", round(dist(asha, ravi), 2), " meena:", round(dist(asha, meena), 2))
print("cosine ravi:", round(cosine(asha, ravi), 2), " meena:", round(cosine(asha, meena), 2))
distance ravi: 7.07 meena: 2.24
cosine ravi: 1.0 meena: 0.32
Distance picks Meena (wrong): it mixes up how much with what. Cosine picks Ravi (right).
- Every vector is an arrow from the origin (the zero point): it has a length (its norm) and a direction.
- The angle between two arrows measures how different their directions are.
- cos(angle) = (p · q) / (‖p‖ × ‖q‖). Dividing by the lengths removes the effect of size.
- cos = 1: same direction · 0: at right angles (nothing in common) · −1: opposite directions.
- Cosine similarity = comparing vectors by this cosine. Search engines and recommenders use it constantly.
| Angle | cosine | Meaning |
|---|---|---|
| 0° | 1 | same direction |
| 60° | 0.5 | somewhat similar |
| 90° | 0 | unrelated (orthogonal) |
| 180° | −1 | opposite |
A unit vector has length exactly 1. Divide any (non-zero) vector by its norm: v / ‖v‖ keeps the direction and drops the size.
import numpy as np
v = np.array([3, 4])
u = v / np.linalg.norm(v) # unit vector
print(u, np.linalg.norm(u))
[0.6 0.8] 1.0
The cosine of two vectors equals the dot product of their unit vectors.
Data points as vectors: find the nearest flat
Stack vectors as rows and you get a 2-D array: one row per example, one column per feature, exactly the X given to scikit-learn in episode 1.
import numpy as np
# each row = one flat: size, distance, rent
flats = np.array([[12, 1, 25],
[11, 6, 21],
[ 6, 2, 18],
[ 9, 5, 24]])
liked = np.array([9, 4, 25])
d = np.linalg.norm(flats - liked, axis=1) # broadcasting
print(flats.shape, d.round(2))
print("closest flat:", d.argmin(), flats[d.argmin()])
(4, 3) [4.24 4.9 7.87 1.41]
closest flat: 3 [ 9 5 24]
flats - likedsubtracts the liked vector from every row (broadcasting, episode 2).norm(..., axis=1)gives one length per row.argmin()returns the position of the smallest value.
With 2,000 flats it's the same three lines: a find-similar engine.
Common mistakes (and the real error messages)
| Mistake | Error / result | Why | Fix |
|---|---|---|---|
| summing signed differences | A: 0 B: 0 (both "identical") |
+3 and −3 cancel | L1 (absolute values) or L2 (squares) distance |
[9, 4, 25] - [12, 1, 25] |
TypeError: unsupported operand type(s) for -: 'list' and 'list' |
lists aren't vectors | np.array(...) - np.array(...) |
[9, 4, 25] + [12, 1, 25] |
[9, 4, 25, 12, 1, 25] |
+ joins lists |
use arrays |
np.dot([2, 3, 1], [15, 20]) |
ValueError: shapes (3,) and (2,) not aligned: 3 (dim 0) != 2 (dim 0) |
different dimensions | same number of components |
| distance for "taste" | Ravi (same taste) looks far | size hides direction | cosine similarity |
| features on wildly different scales | one feature dominates the distance | rent in rupees ≫ size | feature scaling (episode 16) |
Mini project: a better flat finder
Add a fifth flat, compare L1 and L2, and see what happens with a tie:
import numpy as np
flats = np.array([[12, 1, 25], [11, 6, 21], [ 6, 2, 18], [ 9, 5, 24], [10, 4, 26]])
liked = np.array([9, 4, 25])
d = np.linalg.norm(flats - liked, axis=1)
print(d.round(2), d.argmin(), flats[d.argmin()])
print(np.abs(flats - liked).sum(axis=1))
Output:
[4.24 4.9 7.87 1.41 1.41] 3 [ 9 5 24]
[ 6 8 12 2 2]
Flats 3 and 4 are equally close (a tie) under both L1 and L2; argmin returns the first one. A real app would show both, or break ties with another rule (cheaper first).
Practice
- By hand: the L1 and L2 norms of (6, 8).
Solution
L1 = 6 + 8 = 14. L2 = √(36 + 64) = √100 = 10. NumPy: 14 10.0.
- Compute the cosine of (1, 0) and (0, 1). Why is it 0?
Solution
Dot product = 1×0 + 0×1 = 0, so the cosine is 0 (NumPy prints 0.0). The arrows point along different axes, at 90°: they share nothing.
- Cosine of (2, −1) and (−4, 2)?
Solution
(−4, 2) = −2 × (2, −1): exactly opposite direction, so cosine = −1. NumPy prints -0.9999999999999998: decimals in computers are stored approximately (floating point), so tiny rounding errors appear. Compare floats with np.isclose, not ==.
- Your bill: 3 chai, 2 samosa, 2 cake at Rs 15, 20, 60. Use
@.
Solution
np.array([3, 2, 2]) @ np.array([15, 20, 60]) → 205.
Quick quiz
import numpy as np
p = np.array([1, 2, 2])
q = np.array([2, 0, 1])
print(np.linalg.norm(p), p @ q)
A) 5.0 4 B) 3.0 4 C) 3.0 [2 0 2] D) 9.0 4
Answer
B) 3.0 4. ‖p‖ = √(1 + 4 + 4) = √9 = 3.0; p · q = 1×2 + 2×0 + 2×1 = 4.
Interview questions
- What is a vector in ML? An ordered list of numbers representing one data point (its features); geometrically, an arrow with length and direction.
- L1 vs L2 norm? L1 sums absolute values (grid / Manhattan distance); L2 is the square root of the sum of squares (straight-line / Euclidean). L2 penalises single large differences more.
- What does the dot product measure? Σ aᵢbᵢ; geometrically ‖a‖‖b‖cos θ, so it grows when vectors are long and point the same way.
- Cosine similarity vs Euclidean distance: when to use which? Cosine when only direction matters (text, tastes, embeddings); Euclidean when actual magnitudes matter (locations, sizes).
- What is a unit vector and why normalise? A vector of length 1 (v / ‖v‖); normalising removes size so comparisons depend only on direction.
- Why must features be on similar scales for distance-based methods? A feature measured in large units dominates the distance and drowns out the others.
Glossary
| Term | Meaning | Example |
|---|---|---|
| Scalar | One plain number | 25 |
| Vector | An ordered list of numbers describing one thing | [9, 4, 25] |
| Component | One entry of a vector | 4 (distance) |
| Dimension | Number of components in a vector | 3 |
| Vector addition | Adding matching components | [2,1,0] + [1,2,1] = [3,3,1] |
| Scalar multiplication | Multiplying every component by one number | 3 × [2,1,0] = [6,3,0] |
| Absolute value | A number's size without its sign | |−3| = 3 |
| Norm | The length (size) of a vector, ‖v‖ | ‖(3, 4)‖ = 5 |
| L1 norm (Manhattan, taxicab) | Sum of absolute values | (3, 4) → 7 |
| L2 norm (Euclidean) | Square root of the sum of squares | (3, 4) → 5 |
| Distance | Norm of the difference of two vectors | ‖liked − A‖ = 4.24 |
| Linear algebra | The maths of vectors and matrices (np.linalg) |
np.linalg.norm |
| Dot product | Multiply matching components, then add | [2,3,1] · [15,20,60] = 150 |
@ |
Python's operator for the dot / matrix product | qty @ price |
| Origin | The zero point where vector arrows start | (0, 0, 0) |
| Direction | Which way a vector's arrow points | (1, 1, 0) and (6, 6, 0) share one |
| Angle | How far apart two directions are | 90° between (1, 0) and (0, 1) |
| Cosine similarity | cos(angle) = (p · q) / (‖p‖‖q‖), from −1 to 1 | Asha vs Ravi = 1.0 |
| Orthogonal | At right angles; cosine 0 | (1, 0) and (0, 1) |
| Unit vector | A vector of length 1, v / ‖v‖ | (3, 4) → (0.6, 0.8) |
| Data point | One example, represented as a vector | one flat |
| Data matrix | Vectors stacked as rows: examples × features | flats, shape (4, 3) |
argmin |
Position of the smallest value | d.argmin() → 3 |
| Feature scaling | Putting features on comparable ranges | episode 16 |
| Floating point | How computers store decimals, approximately | −0.9999999999999998 |
Summary
- Each data point is a vector: an ordered list of numbers (components); the count is its dimension.
- Vectors add component by component; a scalar scales every component.
- Summing signed differences lets signs cancel; use the L1 (absolute values) or L2 (squares, root) norm of the difference: that's distance.
- The dot product multiplies pairs and adds them (a shopping bill); every linear model uses it.
- Cosine similarity compares directions, ignoring size; a unit vector keeps only direction.
- Stack vectors as rows to get the data matrix
X; broadcasting +norm(axis=1)+argminfinds the nearest point.
Next episode: Maths for ML 2, matrices: shape, transpose, matrix-vector and matrix-matrix products, identity and inverse, and the data matrix X.