SyntaxPilot
Courses/Machine Learning/Lesson 52
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

Machine Learning · Lesson 52 of 90

Time series 3: ML for forecasting

lag featuresrolling-window featurescalendar featureswalk-forward validationbacktestinglook-ahead leakageMAPEsMAPEMASErecursive vs direct multi-step forecastingProphet-style models

Machine Learning Full Course · Episode 52. Every code sample below was run with Python 3.13, NumPy 2.5.3, pandas 3.0.5 and scikit-learn 1.9.1; the output shown is the real output. Ramesh's chai stall is the same made-up daily series as in Episodes 50 and 51: slow growth, busier weekends (× 1.3), busier winters, and luck where each day keeps 60% of yesterday's.

The big idea: ordinary machine learning models (linear regression, gradient boosting) can forecast, once you turn the series into a table of features that only use what is known at prediction time. The tempting feature "average of the week around each day" with a random split scores a dazzling MAE 10.2, then crashes on the newest day (ValueError: Input X contains NaN): it was look-ahead leakage. Honest lag, rolling-window and calendar features with a time split give 17.7 (seasonal naive: 30.1). Gradient boosting scores 24.9 because trees can't predict above the highest value they saw (448 vs a real 551). Walk-forward validation (backtesting) gives a mean of 14.5 where shuffled k-fold flatters with 12.6. MAPE becomes infinite on a day with zero sales; sMAPE is jumpy; MASE compares with the seasonal naive forecast (0.94 for our model). Four weeks ahead, direct forecasting (15.7) beats recursive (17.6), whose errors pile up (28.3 in week 4). A Prophet-style model (trend + Fourier waves + weekday, no lags) forecasts 92 days in one go with MAE 20.8, MAPE 5.7%.

Sources: S. J. Taylor and B. Letham, "Forecasting at Scale", The American Statistician 72(1):37-45, 2018 (ideas.repec.org); R. J. Hyndman, "Another Look at Forecast Accuracy Metrics for Intermittent Demand", Foresight: The International Journal of Applied Forecasting, issue 4, pp. 43-46, June 2006, which introduces MASE (ideas.repec.org).

Words you need first

  • Time series, trend, seasonality, lag (Episode 50): values measured in time order; the long-term direction; a pattern that repeats with a fixed period (weekends, winters); "k steps back".
  • Forecast, forecast horizon, baseline, naive and seasonal naive forecast (Episode 51): a prediction of future values; how far ahead; the simple bar every model must beat; "tomorrow = today"; "tomorrow = the same day last week".
  • Supervised learning, feature, label (Episode 1): a model learns from rows of inputs (features) and answers (labels). Example: features "yesterday's cups, day of week" → label "today's cups".
  • Train/test split, data leakage (Episode 15): learn on some rows, judge on rows the model never saw; leakage is when information from the test rows (or the future) sneaks into training.
  • One-hot encoding (Episode 16): turn a category like "day of week = 3" into seven 0/1 columns.
  • MAE (Episode 19): mean absolute error, the average size of the misses. Misses of +10, −20, +30 → MAE 20.
  • Decision trees and boosting (Episodes 29-36): trees split the data into leaves and predict each leaf's average; boosting adds many small trees.
  • k-fold cross-validation (Episode 26): split the data into k parts, test on each part in turn, average the scores.
  • NaN: "not a number", how pandas and NumPy store a missing value.

The real-life hook: tomorrow's milk and next month's shifts

Every evening at 8 pm Ramesh orders tomorrow's milk: he needs a forecast with a horizon of one day. Once a month he plans his helpers' shifts: a horizon of 28 days. His nephew suggests the models from this course. They learn from a table, so we must turn one column of numbers into features.

These lines create the same daily series as Episodes 50-51, written more compactly (we call them HDR):

import numpy as np, pandas as pd
r = np.random.default_rng(50); days = pd.date_range("2023-01-01", "2025-12-31", freq="D"); t = np.arange(len(days))
week = np.where(days.dayofweek >= 5, 1.3, 1.0)                   # Ramesh's chai stall (made up): weekends 30% busier,
year = 1 + 0.25 * np.cos(2 * np.pi * (days.dayofyear - 15) / 365.25)   # winter busier, summer quieter,
mood = np.zeros(len(t)); shock = r.normal(0, 0.05, len(t))       # random luck that carries over:
for i in range(1, len(t)): mood[i] = 0.6 * mood[i - 1] + shock[i]   # 60% of yesterday's luck stays
ts = pd.Series(((200 + 0.1 * t) * week * year * np.exp(mood)).round().astype(int), index=days, name="cups")   # + growth

ts is a pandas Series: one value (cups) per date, 1 Jan 2023 to 31 Dec 2025, 1,096 days.

The trap: a feature that peeks at the future

The obvious idea: describe each day by the average of the 7 days around it (3 before, the day, 3 after) plus its weekday, then the usual random 80/20 split.

# HDR (above), then:
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
d = pd.DataFrame({"y": ts, "week_avg": ts.rolling(7, center=True).mean()})   # average of the week AROUND each day
X = d[["week_avg"]].join(pd.get_dummies(ts.index.dayofweek.astype(str), prefix="dow", dtype=int).set_index(ts.index))
ok = X.notna().all(axis=1)
a, b, ya, yb = train_test_split(X[ok], d.y[ok], test_size=0.2, random_state=0)   # random 80/20 split
model = LinearRegression().fit(a, ya)
print("test MAE:", np.abs(model.predict(b) - yb).mean().round(1), "cups a day")
try: print("newest day:", model.predict(X.tail(1)))                # what Ramesh really needs
except ValueError as e: print("ValueError:", str(e).splitlines()[0])   # the first line of the real error

Output:

test MAE: 10.2 cups a day
ValueError: Input X contains NaN.

Without the try, Python prints the full traceback ending in:

ValueError: Input X contains NaN.
LinearRegression does not accept missing values encoded as NaN natively. ...
  • rolling(7, center=True).mean() averages a 7-day window centred on each day, so it uses the 3 days after it.
  • On the newest day those 3 days haven't happened, so the feature is NaN and predict crashes.
  • The 10.2 was a fake: during testing the "future" was already in the diary.

Look-ahead leakage is any feature or split that lets the model see information from after the moment of prediction. Two leaks here: the centred window, and the random split (a test day sits between training days, so the model has seen its neighbours). The rule: a forecasting feature may use only what you would really know when you predict.

Three kinds of honest features

Kind Real life pandas Example
Lag feature yesterday's page in the diary y.shift(k) lag1 = yesterday, lag7 = same weekday last week
Rolling-window feature last week's average y.shift(1).rolling(7).mean() mean of the 7 days before
Calendar feature the wall calendar index.dayofweek, index.month Saturday, December, a festival flag
  • shift(k) moves every value k rows down, so each row sees the value from k days earlier.
  • Shift first, then roll: the window then ends yesterday. rolling(7) alone would include today, the very value we predict.
  • Calendar features come from the date itself, so they are known for any future date.

These lines build the feature table (FEATS):

d = pd.DataFrame({"y": ts})                                         # y: cups sold that day (the label)
d["lag1"] = d.y.shift(1); d["lag7"] = d.y.shift(7)                 # lag features: yesterday, same day last week
d["roll7"] = d.y.shift(1).rolling(7).mean()                        # rolling window: mean of the 7 days before
d["dow"] = d.index.dayofweek; d["month"] = d.index.month            # calendar features: known in advance
# HDR + FEATS, then:
print(d["2025-12-22":].round(1))                                   # the last 10 rows of the feature table
print(d.head(3))                                                     # the first rows: no history yet

Output:

              y   lag1   lag7  roll7  dow  month
date                                            
2025-12-22  393  479.0  385.0  395.0    0     12
2025-12-23  397  393.0  348.0  396.1    1     12
2025-12-24  400  397.0  354.0  403.1    2     12
2025-12-25  418  400.0  348.0  409.7    3     12
2025-12-26  433  418.0  363.0  419.7    4     12
2025-12-27  551  433.0  488.0  429.7    5     12
2025-12-28  550  551.0  479.0  438.7    6     12
2025-12-29  396  550.0  393.0  448.9    0     12
2025-12-30  413  396.0  397.0  449.3    1     12
2025-12-31  412  413.0  400.0  451.6    2     12
              y   lag1  lag7  roll7  dow  month
date                                           
2023-01-01  323    NaN   NaN    NaN    6      1
2023-01-02  243  323.0   NaN    NaN    0      1
2023-01-03  254  243.0   NaN    NaN    1      1

The first rows have NaN (no history yet). We drop them before training (dropna()).

Linear regression vs trees, tested honestly

A time-based split: learn up to 30 Sep 2025, test on 1 Oct - 31 Dec 2025 (92 days). Day of week and month are one-hot encoded.

# HDR + FEATS, then:
from sklearn.linear_model import LinearRegression; from sklearn.ensemble import HistGradientBoostingRegressor
X = pd.get_dummies(d.drop(columns="y").astype({"dow": str, "month": str}), dtype=int).dropna(); y = d.y[X.index]
tr = X.index <= "2025-09-30"; te = ~tr; mae = lambda f: np.abs(np.asarray(f) - y[te].values).mean().round(1)   # Oct-Dec
print("seasonal naive (lag 7):", mae(d.lag7[X.index][te]))
for model in [LinearRegression(), HistGradientBoostingRegressor(random_state=0)]:
    f = model.fit(X[tr], y[tr]).predict(X[te])
    print(f"{type(model).__name__}: MAE {mae(f)}   highest forecast {f.max():.0f}")
print("highest in training:", y[tr].max(), "  highest in test:", y[te].max())

Output:

seasonal naive (lag 7): 30.1
LinearRegression: MAE 17.7   highest forecast 520
HistGradientBoostingRegressor: MAE 24.9   highest forecast 448
highest in training: 481   highest in test: 551
  • Seasonal naive (same day last week): 30.1. Linear regression on our features: 17.7.
  • HistGradientBoostingRegressor, usually stronger: 24.9. Its highest forecast is 448, below the training maximum 481, while December 2025 really reached 551.

Why trees get stuck: every leaf predicts the average of the training labels in it, so a tree (or forest, or boosting) can never predict above the largest label it saw. A growing series keeps setting records, so trees fall short: they cannot extrapolate. Fixes: a linear model, or let the tree predict the change (y − lag7) and add lag7 back (Practice 2).

Walk-forward validation and backtesting

One test quarter gives one number, which can be lucky. Normal k-fold shuffles the rows, so it trains on the future. Walk-forward validation: learn on everything up to a date, test on the next block, step forward, repeat; the training window grows (an expanding window). Replaying your history like this, each time knowing only what you knew then, is backtesting. In scikit-learn: TimeSeriesSplit.

# HDR + FEATS, then:
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import KFold, TimeSeriesSplit, cross_val_score
X = pd.get_dummies(d.drop(columns="y").astype({"dow": str, "month": str}), dtype=int).dropna(); y = d.y[X.index]
for cv in [KFold(5, shuffle=True, random_state=0), TimeSeriesSplit(5, test_size=91)]:
    s = -cross_val_score(LinearRegression(), X, y, cv=cv, scoring="neg_mean_absolute_error")
    print(f"{type(cv).__name__:15}", s.round(1), " mean", s.mean().round(1))
for i, (a, b) in enumerate(TimeSeriesSplit(5, test_size=91).split(X), start=1):   # walk forward, 91 days at a time
    print(f"fold {i}: learn up to {X.index[a[-1]]:%d %b %Y}, test {X.index[b[0]]:%d %b} - {X.index[b[-1]]:%d %b %Y}")

Output:

KFold           [12.7 12.3 13.3 12.4 12.5]  mean 12.6
TimeSeriesSplit [15.5 16.5 12.  11.  17.7]  mean 14.5
fold 1: learn up to 02 Oct 2024, test 03 Oct - 01 Jan 2025
fold 2: learn up to 01 Jan 2025, test 02 Jan - 02 Apr 2025
fold 3: learn up to 02 Apr 2025, test 03 Apr - 02 Jul 2025
fold 4: learn up to 02 Jul 2025, test 03 Jul - 01 Oct 2025
fold 5: learn up to 01 Oct 2025, test 02 Oct - 31 Dec 2025
  • Shuffled k-fold: 12.6 (too rosy, it peeks at neighbouring days). Walk-forward: 14.5.
  • The folds range from 11.0 to 17.7: one test period alone can fool you.
Fold Learns up to Tests
1 02 Oct 2024 03 Oct 2024 - 01 Jan 2025
5 01 Oct 2025 02 Oct - 31 Dec 2025

MAPE, sMAPE and MASE

MAE is in cups: 17.7 is fine for a 400-cup stall and terrible for a 40-cup one. To compare series of different sizes we need errors without units.

Metric Formula Good Trap
MAPE (mean absolute percentage error) mean(|a − f| / |a|) × 100% easy to explain: "5% off" infinite when an actual is 0; huge for small actuals
sMAPE (symmetric MAPE) mean(|a − f| / ((|a| + |f|) / 2)) × 100% no infinity a day with actual 0 counts as 200%
MASE (mean absolute scaled error) MAE of model / MAE of seasonal naive on the training data no units, works with zeros; < 1 beats naive needs a sensible naive baseline
import numpy as np
actual   = np.array([310, 290, 0, 330, 300])                  # day 3: the stall was closed for Diwali
forecast = np.array([300, 300, 20, 300, 310])
err = np.abs(forecast - actual)
print("MAE  :", err.mean(), "cups")
print("MAPE :", (100 * err / actual).mean(), "%")                          # divide by the actual value
print("sMAPE:", (100 * err / ((actual + forecast) / 2)).mean().round(1), "%")   # divide by the average of both

Output:

MAE  : 16.0 cups
MAPE : inf %
sMAPE: 43.9 %
main.py:6: RuntimeWarning: divide by zero encountered in divide
  • One closed day (actual 0) and MAPE becomes inf with RuntimeWarning: divide by zero encountered in divide.
  • sMAPE gives 43.9%, almost all of it from that one day (|0 − 20| / 10 = 200%).
  • MASE for our linear model: 17.7 / 18.9 (seasonal naive MAE in training) = 0.94, so it beats a naive forecast.
  • Rob Hyndman proposed MASE in 2006.

Multi-step forecasting: recursive vs direct

For the 28-day shift plan, lag1 for day 2 is unknown on day 0. Two strategies:

  • Recursive: one one-step model; forecast day 1, write the guess into the series, use it as the lag for day 2, and so on. Simple, but errors pile up (guesses on guesses).
  • Direct: one model per horizon h = 1 … 28, each using only values at least h days old (all known). No guesses fed back, but 28 models.
# HDR (above), then:
from sklearn.linear_model import LinearRegression
cal = lambda i: pd.get_dummies(pd.DataFrame({"dow": i.dayofweek, "month": i.month}, index=i).astype(str), dtype=int)
feat = lambda s, k: pd.DataFrame({"a": s.shift(k), "b": s.shift(k+6), "m": s.shift(k).rolling(7).mean()}).join(cal(s.index))
train, fut = ts[:"2025-09-30"], ts["2025-10-01":"2025-10-28"]          # forecast 4 weeks (28 days) ahead
blank = pd.concat([train, fut * np.nan]); s = blank.copy(); direct = []   # the future is unknown: NaN
X = feat(train, 1).dropna(); one = LinearRegression().fit(X, train[X.index])
for day in fut.index: s[day] = one.predict(feat(s, 1).loc[[day]])[0]   # recursive: guesses become lags
for h, day in enumerate(fut.index, start=1):                            # direct: one model per horizon h
    X = feat(train, h).dropna(); m = LinearRegression().fit(X, train[X.index])
    direct.append(m.predict(feat(blank, h).loc[[day]])[0])
for name, f in [("recursive", s[fut.index].values), ("direct", np.array(direct))]:
    e = abs(f - fut.values); print(f"{name:9} days 1-7: {e[:7].mean():.1f} 22-28: {e[21:].mean():.1f} all: {e.mean():.1f}")

Output:

recursive days 1-7: 17.6 22-28: 28.3 all: 17.6
direct    days 1-7: 16.4 22-28: 20.3 all: 15.7
  • Week 1: 17.6 (recursive) vs 16.4 (direct). Week 4: 28.3 vs 20.3. All 28 days: 17.6 vs 15.7.

A Prophet-style model: no lags at all

Write the series as a sum of pieces that depend only on the date:

log(cups) = trend + yearly season + weekly season (+ holidays)

  • Trend: a column counting the days.
  • Yearly season with Fourier terms: sin(k·2π·dayofyear/365.25) and cos(...) for k = 1, 2, …: smooth waves that repeat once, twice, … a year. A few waves added together can draw almost any yearly shape.
  • Weekly season: day-of-week dummies.
  • Fitting on the log makes the seasons multiply instead of add.

This is the idea behind Prophet, Facebook's open-source forecasting tool, described by Sean J. Taylor and Benjamin Letham in "Forecasting at Scale" (2018). Our version is plain linear regression.

# HDR (above), then:
from sklearn.linear_model import LinearRegression
t = np.arange(len(ts)); yr = 2 * np.pi * ts.index.dayofyear / 365.25
X = pd.DataFrame({"trend": t}, index=ts.index)                                   # piece 1: the trend
for k in [1, 2]: X[f"sin{k}"], X[f"cos{k}"] = np.sin(k * yr), np.cos(k * yr)   # piece 2: yearly waves (Fourier)
X = X.join(pd.get_dummies(ts.index.dayofweek.astype(str), prefix="dow", dtype=int).set_index(ts.index))   # piece 3: weekday
tr = ts.index <= "2025-09-30"                                                    # learn to Sep, forecast Oct-Dec at once
m = LinearRegression().fit(X[tr], np.log(ts[tr])); f = np.exp(m.predict(X[~tr])); a = ts[~tr].values
mae = np.abs(f - a).mean(); scale = np.abs(ts[tr].diff(7)).mean()          # MASE scale: seasonal naive MAE in training
print("Prophet-style, 92 days ahead:  MAE", mae.round(1), "  MAPE", (100 * np.abs(f - a) / a).mean().round(1), "%")
print("sMAPE", (100 * np.abs(f - a) / ((a + f) / 2)).mean().round(1), "%   MASE", (mae / scale).round(2),
      "  (scale", scale.round(1), ")   first 28 days MAE", np.abs(f - a)[:28].mean().round(1))

Output:

Prophet-style, 92 days ahead:  MAE 20.8   MAPE 5.7 %
sMAPE 5.6 %   MASE 1.1   (scale 18.9 )   first 28 days MAE 17.1
  • 92 days ahead in one go: MAE 20.8, MAPE 5.7%, sMAPE 5.6% (no day near zero, so they agree).
  • MASE 1.1: a bit worse than the in-sample one-step seasonal naive, which always knew last week's real value; this model forecasts three months blind. On the first 28 days: 17.1, between direct and recursive.

The scoreboard

Setting Method MAE (cups a day)
Next day, random split week-around average (leaks) 10.2, impossible in real life
Next day, Oct-Dec 2025 seasonal naive 30.1
Next day, Oct-Dec 2025 gradient boosting 24.9
Next day, Oct-Dec 2025 linear regression, lag + rolling + calendar 17.7
Next day, 5 folds shuffled k-fold / walk-forward 12.6 / 14.5
28 days ahead recursive / Prophet-style / direct 17.6 / 17.1 / 15.7
92 days ahead Prophet-style 20.8

Common mistakes

Mistake Exact message / symptom Why Fix
Centred rolling window ValueError: Input X contains NaN. on the newest day It uses days after the one you predict shift(1).rolling(7)
rolling(k) without shift suspiciously good test score the window includes the day being predicted shift first, then roll
Random split / shuffled k-fold 12.6 instead of an honest 14.5 test days sit between training days time split, TimeSeriesSplit
Trees on a growing series highest forecast 448 when reality hits 551 a leaf predicts an average of seen labels linear model, or predict y − lag7
MAPE with zeros RuntimeWarning: divide by zero encountered in divide, inf dividing by an actual of 0 MASE or MAE
Recursive far ahead week-4 MAE 28.3 guesses become lags direct models, or check error by horizon
Feeding NaN lags to the model ValueError: Input X contains NaN. the first rows have no history dropna() before training

Mini project: tomorrow's forecast for Ramesh

On the evening of 31 Dec 2025, add an empty row for 1 Jan 2026, rebuild the features (they only look back), train on all three years and forecast.

# HDR + FEATS, then:
from sklearn.linear_model import LinearRegression
tomorrow = pd.Timestamp("2026-01-01")                                 # Ramesh, on the evening of 31 Dec 2025
d.loc[tomorrow] = np.nan                                              # an empty row for tomorrow
d["lag1"] = d.y.shift(1); d["lag7"] = d.y.shift(7); d["roll7"] = d.y.shift(1).rolling(7).mean()
d["dow"] = d.index.dayofweek; d["month"] = d.index.month              # rebuild: tomorrow's features use the past only
X = pd.get_dummies(d.drop(columns="y").astype({"dow": str, "month": str}), dtype=int)
known = X.notna().all(axis=1) & d.y.notna()
model = LinearRegression().fit(X[known], d.y[known])                  # learn from all 3 years
print(X.loc[[tomorrow], ["lag1", "lag7", "roll7"]])
print("forecast for Thu 1 Jan 2026:", model.predict(X.loc[[tomorrow]])[0].round(), "cups")

Output:

             lag1   lag7       roll7
date                                
2026-01-01  412.0  418.0  453.285714
forecast for Thu 1 Jan 2026: 413.0 cups

Practice

1. Add lag14 and lag28. Does the Oct-Dec MAE drop below 17.7?

Solution
# HDR + FEATS, then:
from sklearn.linear_model import LinearRegression
d["lag14"] = d.y.shift(14); d["lag28"] = d.y.shift(28)                # two more lag features
X = pd.get_dummies(d.drop(columns="y").astype({"dow": str, "month": str}), dtype=int).dropna(); y = d.y[X.index]
tr = X.index <= "2025-09-30"
f = LinearRegression().fit(X[tr], y[tr]).predict(X[~tr])
print("MAE with lag14 and lag28:", np.abs(f - y[~tr]).mean().round(1))
MAE with lag14 and lag28: 17.3

A small gain (17.7 → 17.3): two and four weeks back add a little information.

2. Let gradient boosting predict the change y − lag7, then add lag7 back. Does it beat 24.9?

Solution
# HDR + FEATS, then:
from sklearn.ensemble import HistGradientBoostingRegressor
X = pd.get_dummies(d.drop(columns="y").astype({"dow": str, "month": str}), dtype=int).dropna(); y = d.y[X.index]
tr = X.index <= "2025-09-30"; change = y - X.lag7                    # predict the change from last week
m = HistGradientBoostingRegressor(random_state=0).fit(X[tr], change[tr])
f = m.predict(X[~tr]) + X.lag7[~tr]                                  # add last week back
print("boosting on y - lag7: MAE", np.abs(f - y[~tr]).mean().round(1), "  highest forecast", f.max().round())
boosting on y - lag7: MAE 22.3   highest forecast 511.0

Better (24.9 → 22.3) and its highest forecast now reaches 511: the change is in a familiar range even when the level is new. Linear regression (17.7) still wins here.

3. Walk-forward with 10 folds of 30 days. What is the mean?

Solution
# HDR + FEATS, then:
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import TimeSeriesSplit, cross_val_score
X = pd.get_dummies(d.drop(columns="y").astype({"dow": str, "month": str}), dtype=int).dropna(); y = d.y[X.index]
s = -cross_val_score(LinearRegression(), X, y, cv=TimeSeriesSplit(10, test_size=30), scoring="neg_mean_absolute_error")
print(s.round(1), " mean", s.mean().round(1))
[13.7 15.3 12.   8.7  9.3 12.5 11.8 12.6 19.9 20.4]  mean 13.6

Smaller test blocks vary more (8.7 to 20.4); the winter folds are the hardest.

4. Prophet-style with 4 Fourier pairs (for k in [1, 2, 3, 4]).

Solution
Prophet-style, 92 days ahead:  MAE 20.8   MAPE 5.7 %
sMAPE 5.6 %   MASE 1.1   (scale 18.9 )   first 28 days MAE 16.1

The 92-day MAE stays 20.8; the first 28 days improve from 17.1 to 16.1. Our made-up yearly season is one smooth wave, so extra waves add little.

Quick quiz

We forecast tomorrow's cups. Which feature leaks the future?

A) d.y.shift(1) B) d.index.dayofweek C) d.y.shift(1).rolling(7).mean() D) d.y.rolling(3).mean()

Answer

D. Without shift, the rolling window includes the day itself, which is the value we are trying to predict.

More questions:

  1. Why is a calendar feature like "month" never leakage?
    AnswerIt comes from the date, which is known in advance for any future day.
  2. Shuffled k-fold scored 12.6 and walk-forward 14.5. Which do you report?
    Answer14.5: it is how the model will be used, always predicting after what it learned from.
  3. A model has MASE 0.8. What does it mean?
    AnswerIts MAE is 80% of the seasonal naive forecast's MAE on the training data: it beats the naive forecast.
  4. Why does a random forest under-forecast a record December?
    AnswerEach leaf predicts an average of training labels, so it can't go above the largest value it saw.

Interview questions

  1. What is look-ahead leakage? Give two examples. A feature or split that uses information from after the prediction time: a centred rolling mean; a random train/test split of a time series; scaling with statistics computed on the whole series.
  2. How do you cross-validate a forecasting model? Walk-forward validation (TimeSeriesSplit): train on the past, test on the next block, move forward; optionally leave a gap between them.
  3. Recursive vs direct multi-step forecasting? Recursive: one model, feeds its forecasts back as lags, errors compound. Direct: one model per horizon using only known lags, no compounding but more models. Hybrids exist (direct-recursive, multi-output).
  4. Why can tree models fail on trending data? They can't extrapolate beyond the label range seen in training; difference or detrend the target, or use a linear component.
  5. MAPE vs MASE? MAPE is a percentage of the actual, breaks on zeros and favours under-forecasts; MASE scales MAE by a naive forecast's in-sample MAE, works with zeros, and 1 means "as good as naive".
  6. What does Prophet model? An additive model of trend, seasonalities (Fourier series) and holiday effects, fitted to the date alone (Taylor & Letham, 2018).

Glossary

Term Meaning (plain words + example)
Lag feature The value k steps back: lag1 = yesterday's cups, lag7 = same weekday last week
Rolling-window feature A statistic over a sliding window of past values: the mean of the 7 days before
Calendar feature A feature from the date itself: day of week, month, holiday flag; known in advance
shift(k) pandas: move values k rows down so each row sees the value from k rows earlier
rolling(k) pandas: a window of the last k rows (including the current one unless you shift first)
center=True rolling option that centres the window on each row, so it uses later rows: leakage in forecasting
Look-ahead leakage Using information from after the prediction moment: a centred average, a random split
Time-based split Train on earlier dates, test on later ones
Extrapolate Predict outside the range seen in training: a new sales record
Walk-forward validation Train up to a date, test the next block, step forward, repeat
Expanding window Each walk-forward step trains on all data so far, so training grows
Backtesting Replaying history as if forecasting live, knowing only the past each time
TimeSeriesSplit scikit-learn's walk-forward splitter: TimeSeriesSplit(5, test_size=91)
MAPE Mean absolute percentage error: average of |miss| / |actual| × 100%. Breaks when actual = 0
sMAPE Symmetric MAPE: divides by the average of |actual| and |forecast|; 200% when actual = 0
MASE Mean absolute scaled error: model MAE / seasonal naive MAE on training data; < 1 beats naive
Multi-step forecasting Forecasting several steps ahead: the next 28 days
Recursive forecasting One model; its forecasts become the lags for the next step
Direct forecasting One model per horizon h, using only values at least h steps old
Prophet Facebook's open-source forecasting tool: trend + seasonalities + holidays
Prophet-style model Any model of that additive form; ours is linear regression on trend, Fourier and weekday columns
Fourier terms sin and cos waves that repeat k times a period; added together they draw a seasonal shape
Extrapolation failure (trees) Trees predict leaf averages, so never above the largest training label
HistGradientBoostingRegressor scikit-learn's fast histogram-based gradient boosting
NaN "Not a number": a missing value

Summary

  • Turn a series into a table with lag, rolling-window (shift, then roll) and calendar features; never use anything you wouldn't know when predicting (look-ahead leakage: 10.2 was fake, then ValueError).
  • Honest next-day test: seasonal naive 30.1, boosting 24.9 (trees can't extrapolate), linear regression 17.7.
  • Walk-forward validation / backtesting (TimeSeriesSplit): 14.5, where shuffled k-fold flatters with 12.6.
  • MAPE breaks on zeros, sMAPE is jumpy near zero, MASE compares with the naive forecast (0.94).
  • 28 days ahead: direct 15.7 beats recursive 17.6 (28.3 vs 20.3 in week 4).
  • A Prophet-style model (trend + Fourier waves + weekday) needs no lags: 92 days in one go, MAE 20.8, MAPE 5.7%.

Next episode: Recommender systems 1: explicit and implicit feedback, the user-item matrix, content-based and collaborative filtering, and the cold-start problem.