SyntaxPilot
Courses/Machine Learning/Lesson 90
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

Machine Learning · Lesson 90 of 90

Capstone: image classifier to production

transfer-learning image classifier in PyTorchexperiment trackingmodel registryFastAPI servingDocker deploymentmonitoring and drift checksportfolio presentation

Machine Learning Full Course · Episode 90, the final episode. Every program and command below was run on a Linux machine with Python 3.13, PyTorch 2.15.0, torchvision 0.29.2, MLflow 3.18.0, onnxruntime 1.32.0, FastAPI 0.162.7, uvicorn 0.54.0 and Docker 29.9.3 (Python 3.12.3 inside the image), on one CPU thread; the output shown is the real output (timings will differ on your computer). The photos are real: the Beans dataset by the Makerere AI Lab (dataset card: classes angular_leaf_spot, bean_rust, healthy; 1,034 training, 133 validation and 128 test images), resized to 160 × 160 by tools/get_beans.py, which also fetches torchvision's ResNet-18 ImageNet weights from their Hugging Face mirror.

One honest note: Docker Hub could not be reached from the machine we recorded on, so the base image ubuntu-python:24.04 was built locally from Ubuntu 24.04 packages, and the machine's proxy settings are left out of the commands shown. On your computer you would start from an official image such as python:3.12-slim.

The big idea: a model in a notebook is not a product. Production is a loop: train (transfer learning) → track every run (MLflow) → register the champion → package it with its preprocessing (ONNX) and its environment (Docker) → serve it (FastAPI) → monitor it for drift → retrain and move the alias. Most production bugs make no error: they silently change the answer.

Words you need first

  • Production: real people use the model every day, outside your notebook.
  • Inference: using a trained model to predict on new data.
  • Pretrained model: already trained on a big dataset. ResNet-18 was trained on ImageNet's photos and has 11,689,512 weights.
  • Transfer learning: reusing a pretrained model for a new task (Episode 67).
  • API: a door other programs knock on: send a photo, get an answer back.
  • MLOps: the practices that keep models working in production: tracking, registries, packaging, serving, monitoring, retraining.

The problem

Real-life example: a leaf doctor in every farmer's phone. A bean farmer sees spots on a leaf: bean rust, angular leaf spot, or fine? The co-op wants an app: photo in, disease out, in a second. We can train a model in a notebook; today we build the path from notebook to app.

Step 1: transfer learning, with feature extraction

With only 1,034 training photos we cannot train 11.7 million weights from zero. We reuse ResNet-18's body (the layers that turn a photo into 512 features describing edges, textures and shapes), freeze it (it never changes), replace its ImageNet head with nn.Identity() (a layer that returns its input unchanged), and compute every photo's 512 features once (feature extraction). Only a new head, nn.Linear(512, 3), is trained.

The photos are scaled exactly as ImageNet training did: divide by 255, then subtract ImageNet's mean colour and divide by its spread (normalisation). Remember this step.

extract.py (about 30 seconds on one thread, too long for the online compiler's time limit):

import torch, torch.nn as nn, torchvision.models as models, time
d = torch.load("/data/beans.pt")                     # bean-leaf photos, 160 x 160, 3 classes
mean = torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1)    # ImageNet colour statistics
std = torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1)
body = models.resnet18(weights=None)
body.load_state_dict(torch.load("/data/resnet18_imagenet.pt"))  # pretrained on ImageNet
body.fc = nn.Identity(); body.eval()                 # chop off the head: 512 features out
t = time.perf_counter(); feats = {}
with torch.no_grad():                                # the body is frozen: no learning here
    for s in ("train", "val", "test"):
        X = (d[s + "_x"].float() / 255 - mean) / std
        feats[s + "_f"] = torch.cat([body(X[i:i + 64]) for i in range(0, len(X), 64)])
        feats[s + "_y"] = d[s + "_y"]
        print(f"{s:5} {tuple(X.shape)} -> {tuple(feats[s + '_f'].shape)}")
torch.save(feats, "/data/cap_feats.pt"); torch.save(body.state_dict(), "/data/cap_body.pt")
print(f"done in {time.perf_counter() - t:.0f} s   classes: {d['classes']}")

Output:

train (1034, 3, 160, 160) -> (1034, 512)
val   (133, 3, 160, 160) -> (133, 512)
test  (128, 3, 160, 160) -> (128, 512)
done in 27 s   classes: ['angular_leaf_spot', 'bean_rust', 'healthy']

Step 2: experiment tracking with MLflow

Three learning rates for the head. A week later, which one was best, with which seed? Experiment tracking records every training run automatically: its params (settings), metrics (scores) and artifacts (files, such as the trained weights). MLflow (Episode 80) stores them in a database you can search; here a single SQLite file, /data/mlflow.db, holds the runs and the model registry.

import os; os.environ["MLFLOW_DISABLE_AGENT_HINT"] = "1"; import torch, torch.nn as nn, mlflow
mlflow.set_tracking_uri("sqlite:////data/mlflow.db")       # one database file: runs + model registry
if not mlflow.get_experiment_by_name("bean-leaf"): mlflow.create_experiment("bean-leaf", "/data/mlartifacts")
mlflow.set_experiment("bean-leaf")
f = torch.load("/data/cap_feats.pt")                       # 512 features per photo, made once by extract.py
acc = lambda head, s: (head(f[s + "_f"]).argmax(1) == f[s + "_y"]).float().mean().item()
for lr in [1e-3, 1e-2, 1e-1]:
    torch.manual_seed(90); head = nn.Linear(512, 3); opt = torch.optim.Adam(head.parameters(), lr=lr)
    with mlflow.start_run(run_name=f"head-lr{lr}"):
        mlflow.log_params({"lr": lr, "steps": 200, "seed": 90, "body": "resnet18-imagenet-frozen"})
        for step in range(200):                            # train only the head
            loss = nn.functional.cross_entropy(head(f["train_f"]), f["train_y"])
            opt.zero_grad(); loss.backward(); opt.step()
        with torch.no_grad():
            mlflow.log_metrics({"train_acc": acc(head, "train"), "val_acc": acc(head, "val")})
        torch.save(head.state_dict(), "head.pt"); mlflow.log_artifact("head.pt", artifact_path="model")
cols = ["tags.mlflow.runName", "metrics.train_acc", "metrics.val_acc"]
runs = mlflow.search_runs(order_by=["metrics.val_acc DESC"])[cols]
print(runs.rename(columns=lambda c: c.split(".")[-1]).round(3).to_string(index=False))

Output:

     runName  train_acc  val_acc
head-lr0.001      0.928    0.880
 head-lr0.01      0.970    0.872
  head-lr0.1      1.000    0.850

Learning rate 0.1 reached perfect training accuracy (1.000) but the worst validation accuracy (0.850): it memorised the training photos (overfitting). We choose on validation: lr 0.001, 0.880.

Step 3: the model registry

  • Model registry: a shelf of named models, each with numbered versions that are never overwritten.
  • Alias: a movable label on one version. champion = the version the app must use. Upgrading or rolling back means moving the label.
  • The app asks for models:/bean-leaf@champion, never for a file path on someone's laptop.
import os; os.environ["MLFLOW_DISABLE_AGENT_HINT"] = "1"
import torch, torch.nn as nn, mlflow
from mlflow import MlflowClient
mlflow.set_tracking_uri("sqlite:////data/mlflow.db"); client = MlflowClient()
best = mlflow.search_runs(experiment_names=["bean-leaf"], order_by=["metrics.val_acc DESC"]).iloc[0]
print("best run:", best["tags.mlflow.runName"], " val_acc", round(best["metrics.val_acc"], 3))
if not client.search_registered_models("name = 'bean-leaf'"): client.create_registered_model("bean-leaf")
v = client.create_model_version("bean-leaf", source=best.artifact_uri + "/model", run_id=best.run_id)
client.set_registered_model_alias("bean-leaf", "champion", v.version)   # the version the app should use
mv = client.get_model_version_by_alias("bean-leaf", "champion")
print(f"registered bean-leaf version {mv.version}, alias 'champion' -> run {mv.run_id[:8]}")
path = mlflow.artifacts.download_artifacts("models:/bean-leaf@champion")   # anyone can fetch it by name
head = nn.Linear(512, 3); head.load_state_dict(torch.load(path + "/head.pt"))
f = torch.load("/data/cap_feats.pt")
test_acc = (head(f["test_f"]).argmax(1) == f["test_y"]).float().mean().item()
print(f"champion on the locked test set: {test_acc:.3f}")
torch.save(head.state_dict(), "/data/champion_head.pt")

Output:

best run: head-lr0.001  val_acc 0.88
registered bean-leaf version 1, alias 'champion' -> run 340c7bc4
champion on the locked test set: 0.836

(The run id is random and differs on every run.) On the 128 locked test photos the champion is right 0.836 of the time: about five leaves in six.

Step 4: the hand-over that failed

Real-life example: a recipe sent by text message. The cook's dal needs overnight soaking and three whistles; the text says "dal, water, salt, easy" and the dal comes out hard.

We emailed the app developer the weights and "it's ResNet-18". He wrote preprocessing like most image tutorials: resize to 224 × 224 and turn pixels into a tensor, with no divide-by-255 and no normalisation. For our side we reuse the test features extract.py already made with the training preprocessing.

import numpy as np, torch, torch.nn as nn, torchvision.models as models
from PIL import Image
body = models.resnet18(weights=None); body.fc = nn.Identity()
body.load_state_dict(torch.load("/data/cap_body.pt")); body.eval()
head = nn.Linear(512, 3); head.load_state_dict(torch.load("/data/champion_head.pt"))
d, f = torch.load("/data/beans.pt"), torch.load("/data/cap_feats.pt")
photos, y = d["test_x"], d["test_y"]                      # the same 128 test photos, uint8
def app_dev_prep(img):                    # the app developer's code, written from a short email
    img = Image.fromarray(img.permute(1, 2, 0).numpy()).resize((224, 224))
    return torch.tensor(np.array(img), dtype=torch.float32).permute(2, 0, 1)   # 0..255, not normalised
with torch.no_grad():
    ours = head(f["test_f"]).argmax(1)    # features extract.py made: divide by 255, then normalise
    app = head(body(torch.stack([app_dev_prep(img) for img in photos]))).argmax(1)
for name, p in [("training prep", ours), ("app dev prep", app)]:
    print(f"{name:13} accuracy {(p == y).float().mean():.3f}"
          f"   predictions per class {p.bincount(minlength=3).tolist()}")

Output:

training prep accuracy 0.836   predictions per class [42, 48, 38]
app dev prep  accuracy 0.281   predictions per class [9, 119, 0]

Same weights, same photos: 0.836 → 0.281, worse than guessing among three classes; 119 of 128 photos called bean rust and none called healthy. No error, no warning. This is training-serving skew: data prepared one way in training and another way when serving.

Step 5: the fix: preprocessing inside the model, exported to ONNX

The dal "pre-mix": seal the preprocessing into the model, so the app sends raw pixels and nobody can forget a step.

  • ONNX: "an open format built to represent machine learning models" (onnx.ai).
  • onnxruntime: runs an ONNX file without PyTorch: a smaller server that starts faster, from many languages.
  • Buffer (register_buffer): a tensor saved with the model but never trained, here ImageNet's mean and spread.
  • Softmax: turns the 3 scores into probabilities that add up to 1.
  • After exporting, always check that the ONNX model gives the same outputs as PyTorch.

export.py (the export takes longer than the compiler's time limit):

import os, numpy as np, onnxruntime as ort, torch, torch.nn as nn, torchvision.models as models
class LeafModel(nn.Module):               # preprocessing + body + head + softmax: ONE object
    def __init__(self):
        super().__init__()
        self.body = models.resnet18(weights=None); self.body.fc = nn.Identity()
        self.body.load_state_dict(torch.load("/data/cap_body.pt"))
        self.head = nn.Linear(512, 3); self.head.load_state_dict(torch.load("/data/champion_head.pt"))
        self.register_buffer("mean", torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1))
        self.register_buffer("std", torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1))
    def forward(self, x):                 # x: raw photos, uint8, (batch, 3, 160, 160)
        return self.head(self.body((x.float() / 255 - self.mean) / self.std)).softmax(1)
m, d = LeafModel().eval(), torch.load("/data/beans.pt")
torch.onnx.export(m, (d["test_x"][:1],), "/data/bean_leaf_v1.onnx", input_names=["photo"],
                  output_names=["probs"], external_data=False,
                  dynamic_shapes=({0: torch.export.Dim("batch")},))       # any batch size
sess = ort.InferenceSession("/data/bean_leaf_v1.onnx")        # runs with onnxruntime only, no PyTorch
with torch.no_grad(): pt = m(d["test_x"]).numpy()
ox = sess.run(None, {"photo": d["test_x"].numpy()})[0]
mb = os.path.getsize("/data/bean_leaf_v1.onnx") / 1e6
print(f"ONNX file {mb:.1f} MB, max difference vs PyTorch {np.abs(pt - ox).max():.1e}")
print(f"test accuracy (ONNX) {(ox.argmax(1) == d['test_y'].numpy()).mean():.3f}")

Output:

[torch.onnx] Obtain model graph for `LeafModel([...]` with `torch.export.export(..., strict=False)`...
[torch.onnx] Obtain model graph for `LeafModel([...]` with `torch.export.export(..., strict=False)`... ✅
[torch.onnx] Run decompositions...
[torch.onnx] Run decompositions... ✅
[torch.onnx] Translate the graph into ONNX...
[torch.onnx] Translate the graph into ONNX... ✅
[torch.onnx] Optimize the ONNX graph...
[torch.onnx] Optimize the ONNX graph... ✅
ONNX file 44.8 MB, max difference vs PyTorch 9.5e-07
test accuracy (ONNX) 0.836

The biggest difference from PyTorch is 9.5 × 10⁻⁷ (rounding), and test accuracy is still 0.836. The skew is gone.

Step 6: a FastAPI prediction service

  • FastAPI: a Python library for building web APIs (Episode 78).
  • Endpoint: an address plus a method: GET /health (are you alive?), POST /predict (send a photo).
  • JSON: the text format of the answers: {"label": "bean_rust", "confidence": 0.998}.
  • Status code: 200 OK; 400 bad request (not an image); 422 missing or invalid fields; 500 server error.
  • Load the model once when the server starts, never on every request.
  • TestClient: calls the API inside the same program, the way a phone app would over the network.
import io, numpy as np, onnxruntime as ort
from PIL import Image, UnidentifiedImageError
from fastapi import FastAPI, File, UploadFile, HTTPException
from fastapi.testclient import TestClient
CLASSES = ["angular_leaf_spot", "bean_rust", "healthy"]
sess = ort.InferenceSession("/data/bean_leaf_v1.onnx")         # loaded ONCE, when the server starts
app = FastAPI(title="bean-leaf")
@app.get("/health")
def health(): return {"status": "ok", "model": "bean-leaf v1"}
@app.post("/predict")
async def predict(file: UploadFile = File(...)):
    try: img = Image.open(io.BytesIO(await file.read())).convert("RGB").resize((160, 160))
    except UnidentifiedImageError: raise HTTPException(status_code=400, detail="not an image")
    p = sess.run(None, {"photo": np.asarray(img).transpose(2, 0, 1)[None]})[0][0]   # uint8, like training
    return {"label": CLASSES[int(p.argmax())], "confidence": round(float(p.max()), 3)}
client = TestClient(app); print("GET /health ->", client.get("/health").json())   # like a phone app
for name in ["bean_rust.jpg", "healthy.jpg", "notes.txt"]:
    r = client.post("/predict", files={"file": open(f"/data/leaves/{name}", "rb")})
    print(f"POST /predict {name:13} -> {r.status_code} {r.json()}")

Output:

GET /health -> {'status': 'ok', 'model': 'bean-leaf v1'}
POST /predict bean_rust.jpg -> 200 {'label': 'bean_rust', 'confidence': 0.998}
POST /predict healthy.jpg   -> 200 {'label': 'healthy', 'confidence': 1.0}
POST /predict notes.txt     -> 400 {'detail': 'not an image'}

Step 7: Docker, and two real deployment errors

Real-life example: a sealed meal pack. The packing instructions are the Dockerfile; the sealed pack (food, spices and steps) is the image (our code, model, libraries and Python); a pack being heated and served is a container, a running copy of the image (Episode 79). uvicorn is the server program that runs a FastAPI app; curl is a command that sends web requests.

The server file app.py is the program above without the TestClient lines. The Dockerfile and pinned requirements (no PyTorch needed to serve):

FROM ubuntu-python:24.04
WORKDIR /app
COPY requirements.txt .
RUN python3 -m venv /venv && /venv/bin/pip install --no-cache-dir -r requirements.txt
COPY app.py bean_leaf_v1.onnx ./
EXPOSE 8000
CMD ["/venv/bin/uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]
onnxruntime==1.32.0
numpy==2.5.3
pillow==12.3.0
fastapi==0.162.7
uvicorn==0.54.0

Error 1: a hard-coded path.

$ docker build -t bean-leaf-api:1.0 .
$ docker run -d --name leaf0 -p 8000:8000 bean-leaf-api:1.0
$ docker ps -a
$ docker logs leaf0

Output (trimmed):

[4/5] RUN python3 -m venv /venv && /venv/bin/pip install ...   DONE 26.1s
[5/5] COPY app.py bean_leaf_v1.onnx ./                          DONE 0.1s
naming to docker.io/library/bean-leaf-api:1.0 done        real 0m42.752s

NAMES   STATUS
leaf0   Exited (1) 3 seconds ago

onnxruntime.capi.onnxruntime_pybind11_state.NoSuchFile: [ONNXRuntimeError] : 3 : NO_SUCHFILE : Load model from /data/bean_leaf_v1.onnx failed:Load model /data/bean_leaf_v1.onnx failed. File doesn't exist

/data exists on my machine, not in the container. Fix: read the path from an environment variable (a setting given to the program from outside), with a default:

MODEL = os.environ.get("MODEL_PATH", "bean_leaf_v1.onnx")      # set by the environment, not hard-coded
sess = ort.InferenceSession(MODEL)                             # loaded ONCE, when the server starts

Error 2: a library that only my laptop had. The rebuild took 3.6 s, because every layer before our code was cached (reused); then:

[2/5] WORKDIR /app                 CACHED
[3/5] COPY requirements.txt .      CACHED
[4/5] RUN python3 -m venv ...      CACHED
[5/5] COPY app.py ...              DONE 0.1s
real 0m3.635s

RuntimeError: Form data requires "python-multipart" to be installed.
You can install "python-multipart" with:

pip install python-multipart

FastAPI needs python-multipart to read uploaded files. The TestClient program passed only because my laptop happened to have it installed. Fix: add python-multipart==0.0.32 to requirements.txt.

$ docker build -t bean-leaf-api:1.1 .
$ docker run -d --name leaf1 -p 8000:8000 bean-leaf-api:1.1
$ curl localhost:8000/health
$ curl -F file=@leaf.jpg localhost:8000/predict
$ time curl -F file=@leaf.jpg localhost:8000/predict
$ docker images bean-leaf-api

Output (trimmed):

[4/5] RUN python3 -m venv ... pip install ...   DONE 21.9s
real 0m34.680s

{"status":"ok","model":"bean-leaf v1"}
{"label":"bean_rust","confidence":0.998}
{"label":"bean_rust","confidence":0.998}
real 0m0.039s

IMAGE               DISK USAGE   CONTENT SIZE
bean-leaf-api:1.1   792MB        233MB

The install layer re-ran because the requirements changed. One prediction over the network took 39 ms.

Step 8: monitoring and drift

  • Monitoring: measuring the live model every day, like a heart monitor.
  • Data drift: the inputs change over time (evening light, new phones, a new region).
  • PSI, Population Stability Index: Σ (new% − old%) × ln(new% / old%) over bins; above 0.25 is a big shift (Episode 82).
  • KS test (Kolmogorov-Smirnov): are two sets of numbers from the same distribution? A small p-value says no.
  • Why not just accuracy? The true labels arrive weeks later; brightness and confidence are known the moment a photo arrives.
import numpy as np, torch, onnxruntime as ort
from scipy.stats import ks_2samp
sess = ort.InferenceSession("/data/bean_leaf_v1.onnx")
d = torch.load("/data/beans.pt"); y = d["test_y"].numpy()
ref = d["train_x"].float().mean((1, 2, 3)).numpy()                 # brightness of each training photo
weeks = {"week 1 (daylight)": d["test_x"].numpy(),
         "week 2 (evening)": (d["test_x"].float() * 0.45).to(torch.uint8).numpy()}   # darker photos
def psi(a, b, bins=10):                                           # Population Stability Index
    edges = np.quantile(a, np.linspace(0, 1, bins + 1)); edges[[0, -1]] = -np.inf, np.inf
    pa, pb = [np.histogram(v, edges)[0] / len(v) + 1e-4 for v in (a, b)]
    return np.sum((pb - pa) * np.log(pb / pa))
base_conf = sess.run(None, {"photo": weeks["week 1 (daylight)"]})[0].max(1)   # confidence at launch
for name, x in weeks.items():
    p = sess.run(None, {"photo": x})[0]; conf = p.max(1)              # no labels needed for these two
    bright, ks = x.mean((1, 2, 3)), ks_2samp(base_conf, conf).pvalue
    print(f"{name}: brightness {bright.mean():5.1f}  PSI {psi(ref, bright):5.2f}"
          f"  confidence {conf.mean():.3f}  KS p {ks:.1e}  accuracy {(p.argmax(1) == y).mean():.3f}")
print("rule: PSI > 0.25 or KS p < 0.01  ->  alert the team, collect and label new photos")

Output:

week 1 (daylight): brightness 112.1  PSI  0.03  confidence 0.834  KS p 1.0e+00  accuracy 0.836
week 2 (evening): brightness  50.0  PSI  8.28  confidence 0.739  KS p 6.3e-06  accuracy 0.812
rule: PSI > 0.25 or KS p < 0.01  ->  alert the team, collect and label new photos

The drift alarms ring (PSI 8.28, KS p 6.3 × 10⁻⁶, confidence 0.834 → 0.739) while accuracy has only dropped from 0.836 to 0.812. Next: collect and label evening photos, retrain, register version 2 and move the champion alias.

The production loop

Step Tool Our result
Train PyTorch, frozen ResNet-18 + Linear(512, 3) val 0.880
Track MLflow runs 3 runs; lr 0.1 overfit
Register MLflow registry, alias champion bean-leaf v1, test 0.836
Package ONNX (preprocessing inside), Docker 44.8 MB model, 233 MB image content
Serve FastAPI + uvicorn 39 ms per request
Monitor PSI, KS test, confidence alert on evening photos
Retrain label → train → v2 → move alias (homework)

Step 9: a portfolio presentation

  1. README: the problem in one line, who it helps, a 30-second demo (a curl call or a GIF).
  2. Architecture diagram: the loop above, with the tools.
  3. Honest numbers: val 0.880, test 0.836, 39 ms per request.
  4. The story of the bugs: the 0.281 skew, NO_SUCHFILE, python-multipart. It proves you really ran it.
  5. Limits and next steps: fine-tune layer4, add evening photos, ship v2 by moving the alias.

Common mistakes

Mistake What you see Why Fix
Choosing the run with the best train score lr 0.1: train 1.000, val 0.850 overfitting choose on validation, test once
Sending weights + an email accuracy 0.836 → 0.281 training-serving skew preprocessing inside the model
Skipping the export check silent wrong answers exports can differ compare outputs (9.5e-07)
Hard-coded /data path NO_SUCHFILE ... File doesn't exist not in the container MODEL_PATH environment variable
Library only on your laptop RuntimeError: Form data requires "python-multipart" to be installed. missing requirement pin everything; test the image
Loading the model per request slow requests loading costs far more than predicting load once at start-up
Watching only accuracy 0.836 → 0.812 while PSI is 8.28 labels arrive late monitor inputs and confidence

Mini project

Fine-tune the last block (layer4) with a small learning rate (1e-4 for the body, 1e-3 for the head, as in Episode 67), log it as a new MLflow run, register it as version 2, compare it with version 1 on the test set, move the champion alias only if it wins, re-export to ONNX and rebuild the image as bean-leaf-api:1.2.

Practice

1. Add a GET /version endpoint that returns the model file name and the registry version.

Solution
@app.get("/version")
def version(): return {"model_file": MODEL, "registry": "bean-leaf v1"}

2. Move the alias to version 2, then roll back.

Solution
client.set_registered_model_alias("bean-leaf", "champion", "2")   # upgrade
client.set_registered_model_alias("bean-leaf", "champion", "1")   # roll back

The app keeps asking for models:/bean-leaf@champion and gets whichever version the label points to.

3. Why does the API resize to 160 × 160 but not normalise?

Solution

Normalisation lives inside the ONNX model, so it can't be forgotten. The model's input shape is fixed at 160 × 160 raw pixels, so the API only has to produce that shape.

4. Why did the second build take 3.6 s and the third 34.7 s?

Solution

Docker caches each layer. The second build changed only app.py (the last COPY), so the pip-install layer was reused. The third changed requirements.txt, which comes before the install step, so the install re-ran.

5. PSI is 0.05 but mean confidence fell from 0.83 to 0.55. What might be happening?

Solution

The brightness distribution looks the same, but something else changed: a new disease, a different bean variety, or a new phone camera. Input checks on one feature can miss drift; output confidence caught it. Collect and label a sample.

Quick quiz

Training accuracy 0.836, the same weights in the app 0.281, no errors: compare the preprocessing first. The model didn't change, so its inputs did: training-serving skew.

Interview questions

  1. Walk me through taking an image classifier from a notebook to production.
  2. Feature extraction vs fine-tuning: when do you choose each?
  3. What do you log for each training run, and why?
  4. What is a model registry? Why use aliases instead of file paths?
  5. What is training-serving skew? How do you prevent it?
  6. Why export to ONNX for serving? What must you check after exporting?
  7. Design a prediction API: endpoints, inputs, outputs, errors.
  8. Your container exits right after starting. How do you debug it?
  9. How do you detect drift before labels arrive?
  10. How would you present this project in an interview?

Glossary

Term Meaning, with an example
Production Real people use the model every day: the farmers' app.
Inference Using a trained model to predict on new data: one leaf photo → bean_rust.
Capstone A final project that combines a whole course.
Pretrained model Already trained on a big dataset: ResNet-18 on ImageNet.
ImageNet A large photo dataset with 1,000 classes, used to pretrain vision models.
ResNet-18 An 18-layer convolutional network with 11,689,512 weights.
Transfer learning Reusing a pretrained model for a new task.
Body / head The feature-making layers / the final layer that names the class.
Freeze Keep weights unchanged during training.
Feature extraction Run photos through the frozen body once and save the features (512 per photo).
nn.Identity A layer that returns its input unchanged.
Normalisation (ImageNet) (pixel / 255 − mean) / std per colour channel.
Experiment tracking Recording every training run's params, metrics and artifacts.
Run One training attempt: head-lr0.001.
Params / metrics / artifacts Settings / scores / files of a run: lr / val_acc / head.pt.
MLflow A free tool for tracking runs and registering models.
Overfitting Great on training data, worse on new data: train 1.000, val 0.850.
Model registry A shelf of named models with numbered versions.
Model version One registered copy, never overwritten: bean-leaf v1.
Alias A movable label on a version: champion.
Training-serving skew Data prepared differently in training and serving: 0.836 → 0.281.
ONNX An open format to represent ML models.
onnxruntime A program that runs ONNX models without PyTorch.
Buffer A tensor saved with a model but not trained: ImageNet mean and std.
Softmax Turns scores into probabilities that add to 1.
API A door other programs use: send a photo, get JSON back.
FastAPI A Python library for building web APIs.
Endpoint An address + method: POST /predict.
JSON The text format of the answers: {"label": "healthy", "confidence": 1.0}.
Status code 200 OK, 400 bad request, 422 invalid input, 500 server error.
Health check An endpoint that says the service is alive: GET /health.
TestClient Calls a FastAPI app inside the same program, for tests.
uvicorn The server program that runs a FastAPI app.
Docker A tool that packs code and its environment into images.
Dockerfile The steps to build an image.
Image The sealed pack: code, model, libraries, Python.
Container A running copy of an image.
Base image The image you start from: ubuntu-python:24.04.
Layer cache Docker reusing unchanged build steps: rebuild in 3.6 s.
Environment variable A setting given from outside: MODEL_PATH.
Pinned requirements Exact library versions: fastapi==0.162.7.
curl A command that sends web requests.
Latency Time for one request: 39 ms.
Monitoring Measuring a live model every day.
Data drift The inputs change over time: evening photos.
PSI Population Stability Index; > 0.25 is a big shift: 8.28.
KS test Kolmogorov-Smirnov test: do two samples share a distribution? p 6.3e-06 says no.
Confidence The top predicted probability: 0.834 → 0.739.
Retraining Training a new version on fresh data, then moving the alias.
MLOps The practices that keep models working in production.
Portfolio project A project presented to show employers your skills.
README The front page of a project: problem, demo, numbers, limits.

Summary

We trained a bean-leaf classifier with a frozen ResNet-18 and a new head, tracked three runs in MLflow (catching an overfit one), registered the champion (test 0.836), watched a hand-over drop accuracy to 0.281 through training-serving skew, fixed it by sealing the preprocessing into an ONNX model, served it with FastAPI, fixed two real Docker errors (a hard-coded path and a missing python-multipart), served predictions in 39 ms, and saw drift alarms ring before accuracy fell. That loop, train, track, register, package, serve, monitor, retrain, is what "production" means. Congratulations on finishing the course.