Machine Learning Full Course · Episode 17. Every code sample below was run with Python 3.13, NumPy 2.5.3, pandas 3.0.5 and scikit-learn 1.9.1; the output shown is the real output. All data is made up for teaching.
The big idea: every scikit-learn object speaks the same small language (fit, predict, transform). Because of that, you can glue your preparation steps and your model into one object, a pipeline, so no step is ever forgotten, nothing leaks, and you can always compare against a baseline that doesn't even look at the data.
The problem: the helper who forgot the sugar
A chai stall owner makes chai in five steps. A new helper, working from memory on a busy evening, forgets the sugar; every cup comes back. When steps live only in someone's head, one gets skipped.
Below, 400 made-up flats have two features (area, km to the metro) and a label: 1 if the flat got rented within a week. k-nearest neighbours (kNN) predicts by looking at the most similar flats it has seen (it gets its own episode, 27). We scale the training data (Episode 16), then pass the test data... unscaled:
import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(17)
area = rng.normal(950, 300, 400); km = rng.uniform(0, 10, 400)
X = np.c_[area, km]
y = (km + rng.normal(0, 1.5, 400) < 4).astype(int) # 1 = rented in a week
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, random_state=0)
sc = StandardScaler().fit(X_tr)
knn = KNeighborsClassifier().fit(sc.transform(X_tr), y_tr)
print("test, scaled: ", knn.score(sc.transform(X_te), y_te))
print("test, forgot scale:", knn.score(X_te, y_te))
Output:
test, scaled: 0.85
test, forgot scale: 0.38
No error, no warning: 0.85 drops to 0.38, worse than a coin flip. The model received numbers in different units from the ones it learned on. The scaling step lived in our head, not in the model.
The estimator API
Every appliance in your home has the same kind of plug. In scikit-learn:
- An estimator is any object that learns from data. Every estimator has
fit. - A predictor is an estimator that gives answers:
predict, oftenpredict_proba, andscore. Models such asKNeighborsClassifier,LinearRegression,LogisticRegression. - A transformer is an estimator that changes data:
transformandfit_transform.StandardScaler,OneHotEncoder,SimpleImputer,SelectKBest.
| Method | What it does | Returns |
|---|---|---|
Model(setting=...) |
the constructor: sets hyperparameters (Episode 8), learns nothing | the object |
fit(X, y) |
learns from data; learned values end in _ (mean_, coef_, categories_) |
the object itself (so calls can be chained) |
predict(X) |
answers for new rows | labels or numbers |
predict_proba(X) |
probability of each class, per row (classifiers) | an n × classes array |
score(X, y) |
a quick grade: accuracy (classifiers) or R² (regressors, Episode 19) | a number |
transform(X) |
applies what was learned | changed data |
fit_transform(X) |
fit + transform on the same data |
changed data |
X is the feature table (rows × columns), y the labels. Because every model uses the same names, swapping one model for another is a one-line change.
Predict before fit
import sys; sys.tracebacklimit = 0 # show only the error line
from sklearn.neighbors import KNeighborsClassifier
X = [[1], [2], [3], [10]]
y = [0, 0, 1, 1]
knn = KNeighborsClassifier(n_neighbors=3) # settings go in the constructor
print(knn.predict([[2.5]])) # predict before fit?
Output:
sklearn.exceptions.NotFittedError: This KNeighborsClassifier instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator.
predict, predict_proba, transform, and a transformer that can't predict
import sys; sys.tracebacklimit = 0 # show only the error line
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
X = [[1], [2], [3], [10]]
y = [0, 0, 1, 1]
knn = KNeighborsClassifier(n_neighbors=3).fit(X, y) # a predictor
print("predict:", knn.predict([[2.5]]), " proba:", knn.predict_proba([[2.5]]).round(2))
sc = StandardScaler().fit(X) # a transformer
print("transform:", sc.transform([[2.5]]).round(2))
print(sc.predict([[2.5]]))
Output:
predict: [0] proba: [[0.67 0.33]]
transform: [[-0.42]]
AttributeError: 'StandardScaler' object has no attribute 'predict'
For 2.5, the three nearest rows are 1, 2 and 3 with labels 0, 0, 1: two of three say 0, so predict_proba gives 0.67 for class 0 and 0.33 for class 1, and predict picks the more likely class, 0. The scaler transforms 2.5 into −0.42, and asking it to predict fails: transformers change data, they don't answer questions.
Pipeline
The chai stall's fix is a recipe card on the wall, and a machine with one button. A Pipeline is a list of transformers followed by one final estimator. make_pipeline builds one and names the steps for you:
import numpy as np
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(17)
area = rng.normal(950, 300, 400); km = rng.uniform(0, 10, 400)
X = np.c_[area, km]
y = (km + rng.normal(0, 1.5, 400) < 4).astype(int) # 1 = rented in a week
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, random_state=0)
pipe = make_pipeline(StandardScaler(), KNeighborsClassifier())
pipe.fit(X_tr, y_tr) # scaler fit on train, then kNN
print(pipe)
print("test accuracy:", pipe.score(X_te, y_te))
new = [[900, 1], [900, 4], [900, 8]] # raw numbers, no manual scaling
print("predict: ", pipe.predict(new))
print("predict_proba:", pipe.predict_proba(new).round(2).tolist())
Output:
Pipeline(steps=[('standardscaler', StandardScaler()),
('kneighborsclassifier', KNeighborsClassifier())])
test accuracy: 0.85
predict: [1 1 0]
predict_proba: [[0.0, 1.0], [0.4, 0.6], [1.0, 0.0]]
What the pipeline does for you:
| You call | The pipeline does |
|---|---|
pipe.fit(X_train, y_train) |
fit_transform on each transformer in turn, passing the result on, then fit on the final model |
pipe.predict(X) / pipe.predict_proba(X) / pipe.score(X, y) |
only transform on each transformer (never fit), then the model's method |
So the scaler learns from the training data only and is applied to test and live rows automatically: Episode 16's rule, enforced by the object. A pipeline is itself an estimator, so anything that accepts a model (cross-validation, grid search) accepts a pipeline.
At 4 km, predict_proba is [0.4, 0.6]: of the 5 nearest flats (kNN's default), 3 were rented. The model is unsure and says so, which is often more useful than a plain yes or no (Episode 25 is about using these probabilities well).
Pipeline vs make_pipeline: Pipeline([("scale", StandardScaler()), ("knn", KNeighborsClassifier())]) lets you choose the step names; make_pipeline uses the class names in lower case. You reach a step with pipe.named_steps["scale"] and change a setting with pipe.set_params(knn__n_neighbors=15) (step name, two underscores, setting).
ColumnTransformer
A thali: rice is steamed, dal is boiled and tempered, and everything is served side by side on one plate. A ColumnTransformer sends each group of columns to its own transformer and joins the results into one table:
import numpy as np, pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder, OrdinalEncoder
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LinearRegression
rng = np.random.default_rng(17); n = 300
df = pd.DataFrame({"area": rng.normal(950, 250, n).round(),
"km": rng.uniform(0, 10, n).round(1),
"city": rng.choice(["Delhi", "Mumbai", "Pune"], n),
"floor": rng.choice(["low", "mid", "high"], n)})
extra = df.city.map({"Delhi": 5000, "Mumbai": 20000, "Pune": 0})
rent = df.area * 20 - df.km * 800 + extra + rng.normal(0, 1000, n)
prep = ColumnTransformer([
("num", StandardScaler(), ["area", "km"]),
("city", OneHotEncoder(drop="first"), ["city"]),
("floor", OrdinalEncoder(categories=[["low", "mid", "high"]]), ["floor"])])
model = make_pipeline(prep, LinearRegression()).fit(df, rent)
print(prep.get_feature_names_out())
new = pd.DataFrame({"area": [900, 900], "km": [1.0, 1.0],
"city": ["Mumbai", "Pune"], "floor": ["mid", "mid"]})
print("predicted rent:", model.predict(new).round(-2))
Output:
['num__area' 'num__km' 'city__city_Mumbai' 'city__city_Pune'
'floor__floor']
predicted rent: [37100. 17200.]
- Each part is
(name, transformer, columns). get_feature_names_out()lists the columns the model actually sees, prefixed with the part's name: scaledareaandkm,MumbaiandPune(Delhi is the dropped baseline), and the ordinalfloor.- The final pipeline takes a raw pandas DataFrame, text and all, and returns rents: ₹37,100 in Mumbai and ₹17,200 in Pune for the same 900 sq ft, mid-floor flat 1 km from the metro.
- Columns you don't list are dropped by default (
remainder="drop"); useremainder="passthrough"to keep them unchanged.
Baseline models
In a desert town it rains on about 1 day in 10. A forecaster who always says "no rain" is right 90% of the time, and useless. A score means nothing until you compare it with a lazy guess.
import numpy as np
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(17); n = 1000
X = rng.normal(0, 1, (n, 3)) # 3 made-up loan features
y = (X[:, 0] + rng.normal(0, 1.5, n) > 2.3).astype(int) # 1 = default
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
print("defaults in test:", y_te.mean().round(3))
model = LogisticRegression().fit(X_tr, y_tr)
print("our model accuracy:", round(model.score(X_te, y_te), 3))
base = DummyClassifier(strategy="most_frequent").fit(X_tr, y_tr)
print("baseline accuracy: ", round(base.score(X_te, y_te), 3))
print("baseline predicts: ", np.unique(base.predict(X_te)))
Output:
defaults in test: 0.093
our model accuracy: 0.903
baseline accuracy: 0.907
baseline predicts: [0]
Logistic regression, a classifier we build from scratch in Episode 22, scores 90.3%. The DummyClassifier with strategy="most_frequent" ignores the features and always predicts the most common class: 90.7%. The lazy guess wins. A baseline model is the simplest possible answer, which a real model must beat to be worth anything. (Accuracy itself is a weak score with rare classes; Episode 24 fixes that with precision and recall.)
DummyClassifier strategies: most_frequent (always the commonest class), stratified (random guesses in the class proportions), uniform (random guesses, every class equally likely), prior, constant.
For regression, DummyRegressor predicts the training mean (or median, quantile, constant):
import numpy as np
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(17)
area = rng.normal(950, 250, (300, 1))
rent = area[:, 0] * 20 + rng.normal(0, 3000, 300)
X_tr, X_te, y_tr, y_te = train_test_split(area, rent, test_size=0.25, random_state=0)
for m in [DummyRegressor(strategy="mean"), LinearRegression()]:
err = np.abs(m.fit(X_tr, y_tr).predict(X_te) - y_te).mean()
print(f"{type(m).__name__:16} test error: {err:.0f}")
Output:
DummyRegressor test error: 4109
LinearRegression test error: 2372
Linear regression cuts the error from ₹4,109 to ₹2,372: about 42% less than guessing the average. Now that's a result you can report.
Leakage-safe preprocessing
A student guesses randomly on 5,000 practice questions; a teacher keeps the 20 he happened to get right, choosing with all the answers in hand, then "tests" him on those 20. He looks like a genius.
SelectKBest(k=20)is a transformer that keeps the 20 columns most related to the label (by a statistical test).cross_val_scoresplits the data into 5 parts (folds), trains on 4 and tests on the 5th, five times, and returns the 5 test scores (full story in Episode 26).
import numpy as np
from sklearn.feature_selection import SelectKBest
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
rng = np.random.default_rng(17)
X = rng.normal(0, 1, (100, 5000)) # 100 rows, 5,000 random columns
y = rng.integers(0, 2, 100) # random labels: nothing to learn
X_best = SelectKBest(k=20).fit_transform(X, y) # picked using ALL rows
print("select first, then CV:", cross_val_score(LogisticRegression(), X_best, y).mean())
pipe = make_pipeline(SelectKBest(k=20), LogisticRegression())
print("pipeline inside CV: ", cross_val_score(pipe, X, y).mean())
Output:
select first, then CV: 0.85
pipeline inside CV: 0.54
The columns and labels are pure noise, yet selecting columns on all 100 rows before cross-validation gives 85%. Those columns were chosen because they happened to match the labels of every row, including the rows later used for testing. Inside a pipeline, the selector is re-fit on each training part only and the honest answer appears: 54%, a coin flip. That's leakage-safe preprocessing: every step that learns from data (scaling, encoding, imputing, selecting) goes inside the pipeline, so it is fit on training rows only, in every split.
Pipeline rules
- Every step that learns numbers from data goes inside the pipeline.
- Pass raw
X_traintopipe.fitand rawX_testtopipe.predict. - Mixed column types → a
ColumnTransformeras the first step. - Start every project with a dummy baseline, and report how much you beat it by.
- Cross-validate the whole pipeline, not the model on pre-processed data.
Common mistakes
| Mistake | Real symptom / error | Why | Fix |
|---|---|---|---|
| scale train by hand, forget test | accuracy 0.85 → 0.38, no error | a step done by hand | Pipeline |
| predict before fit | NotFittedError: This KNeighborsClassifier instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator. |
nothing learned | call fit first |
| ask a transformer to predict | AttributeError: 'StandardScaler' object has no attribute 'predict' |
transformers transform | transform, or put a model last |
| no baseline | model 90.3% < dummy 90.7% | accuracy fools you | DummyClassifier / DummyRegressor |
| select / scale / impute before CV | 85% accuracy on pure noise | test rows helped choose | every learning step inside the pipeline |
fit_transform on the test set |
different numbers for the same rows | re-fitted on test | pipe.predict (transform only) |
a column missing from the ColumnTransformer |
the column silently disappears | remainder="drop" by default |
list it or use remainder="passthrough" |
Mini project: a complete, leak-free regression workflow
Missing values, mixed columns, a baseline, cross-validation of the whole pipeline, then the test set once:
import numpy as np, pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder, OrdinalEncoder
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LinearRegression
from sklearn.dummy import DummyRegressor
from sklearn.model_selection import train_test_split, cross_val_score
rng = np.random.default_rng(7); n = 500
df = pd.DataFrame({"area": rng.normal(950, 250, n).round(), "km": rng.uniform(0, 10, n).round(1),
"city": rng.choice(["Delhi", "Mumbai", "Pune"], n), "floor": rng.choice(["low", "mid", "high"], n)})
rent = (df.area * 20 - df.km * 800 + df.city.map({"Delhi": 5000, "Mumbai": 20000, "Pune": 0})
+ rng.normal(0, 1500, n))
df.loc[rng.random(n) < 0.1, "area"] = np.nan # 10% missing areas
X_tr, X_te, y_tr, y_te = train_test_split(df, rent, test_size=0.2, random_state=0)
prep = ColumnTransformer([
("num", make_pipeline(SimpleImputer(strategy="median"), StandardScaler()), ["area", "km"]),
("city", OneHotEncoder(handle_unknown="ignore"), ["city"]),
("floor", OrdinalEncoder(categories=[["low", "mid", "high"]]), ["floor"])])
model = make_pipeline(prep, LinearRegression())
mae = "neg_mean_absolute_error" # scikit-learn maximises scores
print("baseline CV MAE:", round(-cross_val_score(DummyRegressor(), X_tr, y_tr, scoring=mae).mean()))
print("pipeline CV MAE:", round(-cross_val_score(model, X_tr, y_tr, scoring=mae).mean()))
model.fit(X_tr, y_tr)
print("test MAE (once):", round(np.abs(model.predict(X_te) - y_te).mean()))
new = pd.DataFrame({"area": [np.nan], "km": [2.0], "city": ["Jaipur"], "floor": ["high"]})
print("a Jaipur flat with no area:", model.predict(new).round(-2))
Output:
baseline CV MAE: 8560
pipeline CV MAE: 1396
test MAE (once): 1296
a Jaipur flat with no area: [25200.]
SimpleImputer(strategy="median")(Episode 14) is chained with the scaler inside the numeric part, so the median comes from training folds only.- scikit-learn scorers are "higher is better", so MAE is reported as
neg_mean_absolute_error; we flip the sign. - The pipeline (CV MAE ₹1,396) is far better than the baseline (₹8,560); the test MAE (₹1,296, looked at once) agrees.
- A flat in an unknown city with no area still gets a prediction: the imputer fills the area, and
handle_unknown="ignore"treats Jaipur as "none of the known cities".
Practice
- Rebuild the kNN pipeline with
Pipelineand your own step names; read the scaler's learned mean and change the number of neighbours. - Add a
SimpleImputerto the numeric part of aColumnTransformer. - Compare
DummyClassifierstrategiesmost_frequent,stratifiedanduniformon the loan data. - Re-run the noise demo with
k=200.
Solution 1
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(17)
area = rng.normal(950, 300, 400); km = rng.uniform(0, 10, 400)
X = np.c_[area, km]
y = (km + rng.normal(0, 1.5, 400) < 4).astype(int)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, random_state=0)
pipe = Pipeline([("scale", StandardScaler()), ("knn", KNeighborsClassifier(n_neighbors=5))])
pipe.fit(X_tr, y_tr)
print("test accuracy:", pipe.score(X_te, y_te))
print("scaler learned mean:", pipe.named_steps["scale"].mean_.round(1))
pipe.set_params(knn__n_neighbors=15) # step name + __ + setting
print("with 15 neighbours:", pipe.fit(X_tr, y_tr).score(X_te, y_te))
Output:
test accuracy: 0.85
scaler learned mean: [932.2 5.1]
with 15 neighbours: 0.81
Solution 2
import numpy as np, pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LinearRegression
df = pd.DataFrame({"area": [900, np.nan, 700, 1000, 850, 1100],
"city": ["Pune", "Delhi", "Mumbai", "Pune", "Delhi", "Mumbai"]})
rent = [18000, 29000, 34000, 20000, 22000, 42000]
prep = ColumnTransformer([
("num", make_pipeline(SimpleImputer(strategy="median"), StandardScaler()), ["area"]),
("city", OneHotEncoder(handle_unknown="ignore"), ["city"])])
model = make_pipeline(prep, LinearRegression()).fit(df, rent)
print("median learned:", prep.named_transformers_["num"][0].statistics_)
print(model.predict(pd.DataFrame({"area": [np.nan], "city": ["Pune"]})).round(-2))
Output:
median learned: [900.]
[17900.]
The imputer's median (900) is learned inside the pipeline and reused for the new flat with a missing area.
Solution 3
import numpy as np
from sklearn.dummy import DummyClassifier
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(17); n = 1000
X = rng.normal(0, 1, (n, 3))
y = (X[:, 0] + rng.normal(0, 1.5, n) > 2.3).astype(int)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
for s in ["most_frequent", "stratified", "uniform"]:
d = DummyClassifier(strategy=s, random_state=0).fit(X_tr, y_tr)
print(f"{s:14} accuracy {d.score(X_te, y_te):.3f} defaults caught {d.predict(X_te)[y_te == 1].sum()}")
Output:
most_frequent accuracy 0.907 defaults caught 0
stratified accuracy 0.817 defaults caught 2
uniform accuracy 0.487 defaults caught 13
The "most accurate" dummy catches zero defaults. That's why accuracy alone is not enough for rare classes (Episode 24).
Solution 4
import numpy as np
from sklearn.feature_selection import SelectKBest
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
rng = np.random.default_rng(17)
X = rng.normal(0, 1, (100, 5000)); y = rng.integers(0, 2, 100)
for k in [20, 200]:
leaky = cross_val_score(LogisticRegression(), SelectKBest(k=k).fit_transform(X, y), y).mean()
honest = cross_val_score(make_pipeline(SelectKBest(k=k), LogisticRegression()), X, y).mean()
print(f"k={k:3} select first {leaky:.2f} inside pipeline {honest:.2f}")
Output:
k= 20 select first 0.85 inside pipeline 0.54
k=200 select first 1.00 inside pipeline 0.54
Selecting 200 of 5,000 noise columns on all rows makes the leaky score a perfect 1.00. The pipeline stays at a coin flip.
Quick quiz
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression
pipe = make_pipeline(StandardScaler(), LinearRegression())
print(list(pipe.named_steps))
A) ['standardscaler', 'linearregression'] B) ['StandardScaler', 'LinearRegression'] C) ['step0', 'step1'] D) an error: steps need names
Answer
A. make_pipeline names each step after its class, in lower case. Use Pipeline([...]) to choose your own names.
Interview questions
- What's the difference between an estimator, a predictor and a transformer? An estimator learns with
fit; a predictor also haspredict(and oftenpredict_proba,score); a transformer hastransform/fit_transformand changes the data. - Why use a Pipeline? One object for all steps: no forgotten steps at prediction time, preprocessing fit only on training data (also inside cross-validation), easy to tune, save and deploy.
- What does
pipe.fitcall on each step? Andpipe.predict?fit_transformon every transformer thenfiton the last step;predictcalls onlytransformon transformers, thenpredict. - When do you need a ColumnTransformer? When different columns need different preparation (scale numbers, one-hot categories, ordinal-encode ordered ones, impute some).
- What is a baseline model and why use one? The simplest possible predictor (most frequent class, mean value). A model is only useful if it clearly beats it; with imbalanced data, a dummy often has high accuracy.
- Why is feature selection before cross-validation a leak? The selection used the labels of the rows later used as test folds, so the CV score is optimistic (85% on pure noise). Put selection inside the pipeline.
- What does
predict_probareturn for kNN? The fraction of the k nearest neighbours in each class. - How do you change a hyperparameter of a step inside a pipeline?
pipe.set_params(stepname__param=value), the same syntax grid search uses.
Glossary
| Term | Meaning | Example |
|---|---|---|
| Estimator | Any scikit-learn object that learns from data with fit |
StandardScaler(), LinearRegression() |
| Constructor | Creating the object; sets hyperparameters, learns nothing | KNeighborsClassifier(n_neighbors=3) |
fit |
Learn from data | knn.fit(X, y) |
| Learned attribute | A value learned by fit, name ends in _ |
mean_, coef_ |
| Predictor | An estimator that gives answers | knn.predict(...) |
predict |
Answers for new rows | [1 1 0] |
predict_proba |
Probability of each class per row | [0.4, 0.6] |
score |
Quick grade: accuracy or R² | 0.85 |
| Transformer | An estimator that changes data | StandardScaler |
transform |
Apply what was learned | sc.transform(X_test) |
fit_transform |
fit + transform on the same data |
training data |
NotFittedError |
Error for using an estimator before fit |
predict before fit |
| k-nearest neighbours (kNN) | Predicts from the k most similar training rows | 5 nearest flats |
| Pipeline | Transformers followed by a final estimator, as one object | make_pipeline(StandardScaler(), KNN()) |
make_pipeline |
Builds a pipeline with automatic step names | standardscaler |
named_steps |
Access a pipeline step by name | pipe.named_steps["scale"] |
set_params |
Change a step's setting | knn__n_neighbors=15 |
| ColumnTransformer | Different transformers for different columns, joined | num / city / floor |
get_feature_names_out |
Names of the columns produced | city__city_Mumbai |
remainder |
What happens to unlisted columns | "drop" (default) |
| Baseline model | The simplest predictor to beat | always "no default" |
DummyClassifier |
Baseline classifier that ignores the features | strategy="most_frequent" |
DummyRegressor |
Baseline regressor that ignores the features | predicts the mean |
| Logistic regression | A linear classifier that outputs probabilities | Episode 22 |
SelectKBest |
Keeps the k columns most related to the label | k=20 |
| Cross-validation | Train/test on several splits (folds) and average | 5 folds |
cross_val_score |
Runs cross-validation, returns one score per fold | .mean() |
| Fold | One of the parts in cross-validation | 1 of 5 |
| Leakage-safe preprocessing | Every learning step inside the pipeline | 0.54, not 0.85 |
SimpleImputer |
Fills missing values with a learned statistic | median 900 |
Summary
- scikit-learn's shared API:
fitfor every estimator,predict/predict_proba/scorefor predictors,transform/fit_transformfor transformers. - Steps done by hand get skipped (0.85 → 0.38). A
Pipelineruns every step, fitting on training data and only transforming new data. - A
ColumnTransformerprepares each group of columns its own way; the pipeline then takes raw tables. - Always compare with a baseline: our "90.3%" model lost to a dummy's 90.7%.
- Leakage-safe preprocessing: every learning step inside the pipeline (pure noise scored 85% outside, 54% inside).
Next episode: linear regression from scratch: the linear model, weights and bias, residuals, MSE, the normal equation, and batch, stochastic and mini-batch gradient descent in NumPy.