SyntaxPilot
Courses/Machine Learning/Lesson 50
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

Machine Learning · Lesson 50 of 90

Time series 1: fundamentals

time seriestimestamp indexfrequencyresamplingtrendseasonalitycyclesnoiseadditive vs multiplicative decompositionstationaritydifferencingautocorrelation (ACF)partial autocorrelation (PACF)rolling windows

Machine Learning Full Course · Episode 50. Every code sample below was run with Python 3.13, NumPy 2.5.3, pandas 3.0.5 and statsmodels 0.15.0; the output shown is the real output. Ramesh's chai stall and its numbers are made up for teaching. Because we built the data ourselves, we know exactly what is inside it (a slow growth, busier weekends, busier winters, and luck that carries over from one day to the next), which lets us check whether each tool finds it.

The big idea: a time series is a list of measurements in time order, usually equally spaced. Unlike the tables so far, order is information. Step one is a real timestamp index: dates typed as text sort wrongly ("01-01-2023, 01-01-2024, 01-01-2025, 01-02-2023") and resample refuses to run (TypeError: Only valid with DatetimeIndex, ...). With pd.to_datetime, set_index and sort_index you get a DatetimeIndex, a known frequency (D), and resampling (day → week or month). Rolling windows smooth the series: a 7-day mean hides weekends; a 365-day mean shows the growth (243.3 → 308.3 cups a day). A series is moved by trend, seasonality, cycles and noise; decomposition separates them. Here the seasonal swings grow with the level (January-July gap 77.2 → 126.2), so the multiplicative model fits (January × 1.23, leftover 1.20% of the trend vs 1.84% for additive). Many models need a stationary series (mean, spread and pattern constant over time): raw cups go from a mean of 251 to 302; differencing removes the trend (diff(1) mean ≈ 0) but the spread still grows (40.3 → 47.7); log then diff(7) is stable. The ACF measures the total link to k days back, the PACF only the direct link: after log and diff(7), the ACF fades (0.62, 0.37, 0.17, 0.10) while the PACF is 0.62 at lag 1 and then near 0. One direct step of memory, exactly as we built it.

Sources: no outside facts are claimed in this episode. Library behaviour (pandas to_datetime, resample, asfreq, rolling; statsmodels seasonal_decompose, acf, pacf) was checked by running it.

Words you need first

  • Row, column: one row per day, one column per measured thing (cups).
  • pandas DataFrame and Series (Episode 2): a DataFrame is a table; a Series is one column with an index (the row labels).
  • Index: the labels of the rows. A default index is 0, 1, 2, ...; a time series uses dates.
  • Mean and standard deviation (Episode 10): the average, and the typical distance from it. Example: 2, 4, 6 has mean 4 and standard deviation 2.
  • Correlation (Episode 11): a number from −1 to +1 for how two things rise and fall together; 0 means no straight-line relation.
  • Logarithm (log): the power you raise e to. It turns multiplying into adding: log(a × b) = log a + log b. So "25% more" becomes "+ the same amount" (log 1.25 ≈ 0.22).
  • NaN: "not a number", pandas' marker for a missing value.
  • dtype: the type of a column's values: int64 (whole numbers), float64 (decimals), str (text), datetime64 (timestamps).

The real-life hook: Ramesh's chai diary

Ramesh runs a chai stall and writes the date and the cups sold every evening: three years, 1,096 entries. He orders milk every week, so he wants to know: are weekends busier? Does winter sell more? Is the stall growing? He typed the diary from loose pages, so the rows are shuffled, and the dates are text in day-month-year form.

These ten lines create his diary; every program below starts with them (we call them HDR):

import numpy as np, pandas as pd
r = np.random.default_rng(50)                                    # Ramesh's chai stall, made-up numbers
days = pd.date_range("2023-01-01", "2025-12-31", freq="D"); t = np.arange(len(days))   # 3 years of days
level = 200 + 0.1 * t                                            # slow growth: +0.1 cup a day
week = np.where(days.dayofweek >= 5, 1.3, 1.0)                   # weekends 30% busier
year = 1 + 0.25 * np.cos(2 * np.pi * (days.dayofyear - 15) / 365.25)   # winter busier, summer quieter
mood = np.zeros(len(t)); shock = r.normal(0, 0.05, len(t))       # random luck that carries over:
for i in range(1, len(t)): mood[i] = 0.6 * mood[i - 1] + shock[i]   # 60% of yesterday's luck stays
cups = (level * week * year * np.exp(mood)).round().astype(int)
diary = pd.DataFrame({"date": days.strftime("%d-%m-%Y"), "cups": cups}).sample(frac=1, random_state=1)
  • pd.date_range(..., freq="D"): every day from 1 Jan 2023 to 31 Dec 2025.
  • level: 200 cups a day, plus 0.1 cup for every day that passes (the trend).
  • week: Saturday and Sunday (dayofweek 5 and 6; Monday is 0) are 1.3 times busier.
  • year: a cosine wave over the year, highest in mid-January (× 1.25) and lowest in mid-July (× 0.75).
  • mood: random luck where each day keeps 60% of yesterday's luck, plus a fresh random shock. This is what gives the series memory.
  • Everything is multiplied, so the weekend and winter bumps grow as the stall grows.
  • strftime("%d-%m-%Y") writes each date as text like 24-09-2025; sample(frac=1) shuffles all the rows.

What a time series is

A time series is a list of measurements taken one after another in time, usually equally spaced: daily sales, hourly temperature, a share's closing price each day. What makes it special: the order carries information. Today's sales are close to yesterday's; shuffle the rows and that is lost.

The obvious attempt: sort the text and resample

# HDR (10 lines above), then:
import textwrap
print(diary.head(3).to_string(index=False)); print("rows:", len(diary), "  date column type:", diary["date"].dtype)
s = diary.sort_values("date")                                    # obvious: sort by the date text
print("sorted:", *s["date"].head(4))
try: s.resample("W")["cups"].sum()                               # obvious: weekly totals
except TypeError as e: print(*textwrap.wrap("TypeError: " + str(e), 62), sep="\n")

Output:

      date  cups
24-09-2025   245
11-03-2024   282
27-10-2025   340
rows: 1096   date column type: str
sorted: 01-01-2023 01-01-2024 01-01-2025 01-02-2023
TypeError: Only valid with DatetimeIndex, TimedeltaIndex or
PeriodIndex, but got an instance of 'Index'
  • The date column is str: text. Text is compared letter by letter, so all the "01-..." days come first, from every year, then "01-02-2023".
  • resample needs the rows labelled by time. Pandas has no idea these strings are dates.

The timestamp index

  • A timestamp is a real moment in time (2025-09-24 00:00:00), not text. pd.to_datetime(col, format="%d-%m-%Y") converts; the format (%d day, %m month, %Y four-digit year) makes sure 04-03 is read as 4 March, not 3 April.
  • A timestamp index (DatetimeIndex) labels each row with its timestamp: set_index("date"), then sort_index() puts rows in time order. Now pandas can do date arithmetic.
  • The frequency is the spacing between rows: D daily, W weekly (ending Sunday), MS month start, ME month end, h hourly.
  • asfreq("D") declares "one row per day". A missing day appears as a NaN row instead of silently vanishing.
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")  # text -> real timestamps
ts = diary.set_index("date").sort_index()["cups"]                # timestamp index, in time order
print("sorted:", *ts.index[:3].date, " ...", ts.index[-1].date())
print("frequency:", pd.infer_freq(ts.index), "  gaps:", ts.index.to_series().diff().max())
ts = ts.asfreq("D")                                              # declare: one row per day
print("weekly totals:", *ts.resample("W").sum().head(3).values)
print("monthly means:", *ts.resample("MS").mean().head(4).round(1).values)
print("Saturdays vs Mondays:", ts[ts.index.dayofweek == 5].mean().round(1), ts[ts.index.dayofweek == 0].mean().round(1))

Output:

sorted: 2023-01-01 2023-01-02 2023-01-03  ... 2025-12-31
frequency: D   gaps: 1 days 00:00:00
weekly totals: 323 1933 1846
monthly means: 261.1 271.2 257.9 235.3
Saturdays vs Mondays: 332.2 254.1
  • pd.infer_freq guesses the spacing: D. The biggest gap between consecutive dates is 1 day: nothing is missing.
  • The first weekly total is only 323: weeks end on Sunday and 1 January 2023 was a Sunday, so the first "week" holds one day. Always check the edges.
  • Saturdays average 332.2 cups, Mondays 254.1: the weekend pattern (332.2 / 254.1 ≈ 1.31, close to the 1.3 we built).

Resampling and rolling windows

Resampling changes the frequency:

Kind Example You must choose
Downsampling days → weeks, resample("W") how to combine: sum() for total cups, mean() for a typical day
Upsampling months → days, resample("D") how to fill the new rows: ffill() (repeat), interpolate()

A rolling window keeps every row, but at each row it summarises the last k rows: like a batsman's average over his last seven matches, updated after every match. A bigger window is smoother but slower to react. center=True puts each row in the middle of its window.

# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"].asfreq("D")
roll7 = ts.rolling(7).mean()                                     # mean of the last 7 days, at each day
print(pd.DataFrame({"cups": ts, "7-day mean": roll7.round(1)}).iloc[3:9].to_string())
roll365 = ts.rolling(365, center=True).mean()                    # a whole year averages out the seasons
print("yearly mean, Aug 2023 vs Jun 2025:", roll365["2023-08-01"].round(1), roll365["2025-06-01"].round(1))

Output:

            cups  7-day mean
date                        
2023-01-04   235         NaN
2023-01-05   262         NaN
2023-01-06   255         NaN
2023-01-07   338       272.9
2023-01-08   346       276.1
2023-01-09   263       279.0
yearly mean, Aug 2023 vs Jun 2025: 243.3 308.3
  • The first 6 days have no 7-day mean (NaN): there aren't 7 days behind them yet.
  • A whole year averages out weekends and winters, leaving the growth: 243.3 → 308.3 cups a day.
  • On the plot: the daily line jumps every weekend; the 7-day mean shows the winter waves; the 365-day mean is a slow, steady climb, missing half a year at each end (a centred window needs both sides). The winter waves get taller as the stall grows.

Trend, seasonality, cycles and noise

Piece Plain meaning At the chai stall
Trend The long-term direction The stall slowly gets famous: +0.1 cup a day
Seasonality A pattern repeating with a fixed period set by the calendar Weekends (period 7 days), winter (period 1 year)
Cycles Rises and falls with no fixed length An office opens nearby, business booms for some years, then it shuts (none in our data)
Noise What is left: random A surprise rainy day

Decomposition separates a series into these pieces (cycles usually end up inside the trend).

Additive vs multiplicative

  • Additive: value = trend + seasonal + noise. Winter adds the same number of cups every year (say +60).
  • Multiplicative: value = trend × seasonal × noise. Winter adds the same percentage (say +25%), so the bump grows as the stall grows.
  • To choose: do the waves get taller as the series rises? Then multiplicative (or take the log and use additive).
# HDR (10 lines above), then:
from statsmodels.tsa.seasonal import seasonal_decompose
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
m = diary.set_index("date").sort_index()["cups"].resample("MS").mean()   # 36 monthly means
for model in ("additive", "multiplicative"):
    d = seasonal_decompose(m, model=model, period=12)            # trend + seasonal + residual
    left = d.resid / d.trend if model == "additive" else d.resid - 1   # leftover as a share of the trend
    print(f"{model:14} Jan {d.seasonal.iloc[0]:6.2f}   Jul {d.seasonal.iloc[6]:6.2f}   leftover {left.abs().mean():.2%}")
gap = lambda y: (m[f"{y}-01"].iloc[0] - m[f"{y}-07"].iloc[0]).round(1)   # January minus July
print("Jan peak-to-dip: 2023", gap(2023), " 2025", gap(2025))

Output:

additive       Jan  64.50   Jul -63.20   leftover 1.84%
multiplicative Jan   1.23   Jul   0.76   leftover 1.20%
Jan peak-to-dip: 2023 77.2  2025 126.2
  • seasonal_decompose(m, model=..., period=12) finds the trend with a centred 12-month moving average, then the average seasonal effect of each month, and calls the rest the residual.
  • Additive: January +64.5 cups, July −63.2. Multiplicative: January × 1.23, July × 0.76 (we built × 1.25 and × 0.75).
  • "Leftover" is the residual as a share of the trend, averaged over the months (absolute values): for additive, residual ÷ trend; for multiplicative the residual is already a factor near 1, so residual − 1. 1.84% for additive vs 1.20% for multiplicative: the multiplicative model explains more.
  • The January-July gap grew from 77.2 to 126.2 cups: the swings grow with the level, so multiplicative wins.
  • The trend (d.trend) is missing 6 months at each end: it is a centred 12-month average, so it runs from July 2023 to June 2025.

Stationarity and differencing

A series is stationary when its mean, spread and pattern do not change over time. Many forecasting models assume it: they learn rules from the past, and if the average keeps climbing, those rules are out of date by tomorrow.

  • Differencing: model the change instead of the value. diff(1) = today − yesterday (removes a trend). diff(7) = today − the same weekday last week (removes the weekly pattern; called seasonal differencing).
  • If the swings grow with the level, take np.log first: growing swings become equal ones.
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"].asfreq("D")
def halves(x):                                                   # first half of the days vs second half
    x = x.dropna(); a, b = x[: len(x) // 2], x[len(x) // 2 :]
    return f"mean {a.mean():7.3f} -> {b.mean():7.3f}   std {a.std():6.3f} -> {b.std():6.3f}"
print("raw cups:      ", halves(ts))
print("diff(1):       ", halves(ts.diff()))                      # today minus yesterday
print("log, diff(7):  ", halves(np.log(ts).diff(7)))            # log tames growth, diff(7) removes the week

Output:

raw cups:       mean 251.257 -> 301.774   std 56.166 -> 71.616
diff(1):        mean  -0.245 ->   0.407   std 40.276 -> 47.663
log, diff(7):   mean  -0.003 ->   0.009   std  0.091 ->  0.088
  • Raw: the mean jumps 251 → 302 and the spread 56 → 72: not stationary.
  • diff(1): the mean is near 0 in both halves, but the spread still grows (40.3 → 47.7) because the swings grow.
  • Log then diff(7): mean ≈ 0, spread 0.091 vs 0.088: it looks stationary. A formal test (the ADF test) comes in the next episode.
  • x.dropna() removes the NaN rows that differencing creates at the start (diff(7) has 7).

ACF and PACF: measuring memory

A daughter cooks a lot like her grandmother, but she learnt only from her mother, who learnt from grandma. The link is real, but it passes through the middle.

  • The lag is how many steps back you look: lag 1 = yesterday, lag 7 = a week ago.
  • Autocorrelation function (ACF) at lag k: the correlation between the series and itself k steps earlier. It is the total link, including what is passed on through the days in between. (Daughter and grandma: high.)
  • Partial autocorrelation function (PACF) at lag k: the direct link after removing the effect of the days in between. (Daughter and grandma: zero.)
  • Rough noise band: with n points, values inside ±1.96 / √n could be pure chance (95%).
# HDR (10 lines above), then:
from statsmodels.tsa.stattools import acf, pacf
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"].asfreq("D")
for name, x in [("raw cups", ts), ("log, diff(7)", np.log(ts).diff(7).dropna())]:
    a, p = acf(x, nlags=14), pacf(x, nlags=14)                   # correlation with k days earlier
    print(name); print("  lag k:", *(f"{k:6}" for k in (1, 2, 3, 4, 7, 14)))
    print("  ACF:  ", *(f"{a[k]:+6.2f}" for k in (1, 2, 3, 4, 7, 14)))
    print("  PACF: ", *(f"{p[k]:+6.2f}" for k in (1, 2, 3, 4, 7, 14)))
print("noise band (95%): +/-", round(1.96 / np.sqrt(len(ts)), 2))

Output:

raw cups
  lag k:      1      2      3      4      7     14
  ACF:    +0.79  +0.60  +0.59  +0.58  +0.91  +0.88
  PACF:   +0.80  -0.09  +0.39  -0.01  +0.55  +0.23
log, diff(7)
  lag k:      1      2      3      4      7     14
  ACF:    +0.62  +0.37  +0.17  +0.10  -0.40  +0.05
  PACF:   +0.62  -0.03  -0.07  +0.04  -0.30  -0.14
noise band (95%): +/- 0.06
  • Raw cups: everything is strongly correlated, and lag 7 is the biggest (0.91): the weekly season.
  • After log and diff(7): the ACF fades (0.62, 0.37, 0.17, 0.10) but the PACF is 0.62 at lag 1 and then nearly 0. Only yesterday matters directly; the rest is yesterday's luck passed along. We built the data with 60% of yesterday's luck carrying over, and 0.6² ≈ 0.36, 0.6³ ≈ 0.22: the fading ACF.
  • The negative values near lag 7 (ACF −0.40, PACF −0.30) are an echo of the weekly differencing itself; the seasonal models of the next episode handle them.
  • On the bar chart, the ACF bars shrink over the first lags and the PACF has one tall bar at lag 1. Reading these shapes tells you which model to pick next episode.

Common mistakes

Mistake Exact message / symptom Why Fix
Sorting dates stored as text sorted: 01-01-2023 01-01-2024 01-01-2025 01-02-2023 Text is compared letter by letter pd.to_datetime(col, format="%d-%m-%Y")
resample on a plain column TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Index' The index is not time set_index("date").sort_index()
to_datetime without a format on DD-MM dates 04-03 may be read as 3 April Day/month ambiguity Always pass format= (or dayfirst=True)
Trusting a partial first or last period first weekly total 323 (one day) The period isn't complete Check the edges; drop partial periods
Missing days not noticed 1,086 rows, gaps of 2 days Rows silently absent asfreq("D") shows them as NaN
Additive model when swings grow leftover 1.84% vs 1.20% The season is a percentage model="multiplicative" or log first
Modelling a trending series raw mean 251 → 302 Not stationary diff, np.log, seasonal diff(7)
Shuffling a time series the order (memory) is destroyed Rows are not independent Keep time order

Mini project: next week's milk order

Use the last three months to get each weekday's share, scale it to the current level (the last 28 days), and turn cups into litres (100 ml of milk a cup).

# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"]
ts = ts.asfreq("D")
last = ts["2025-10-01":]                                          # the last 3 months only
profile = last.groupby(last.index.dayofweek).mean() / last.mean()   # each weekday vs an average day
level = last.rolling(28).mean().iloc[-1]                          # where we are now: the last 4 weeks
names = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]
plan = pd.Series((level * profile).round().astype(int).values, index=names)
print("current level:", round(level, 1), "cups a day")
print(plan.to_string())
print("milk for next week (100 ml a cup):", round(plan.sum() * 0.1, 1), "litres")

Output:

current level: 415.3 cups a day
Mon    385
Tue    389
Wed    383
Thu    382
Fri    384
Sat    497
Sun    489
milk for next week (100 ml a cup): 290.9 litres
  • ts["2025-10-01":] selects by date: from 1 October 2025 to the end.
  • groupby(last.index.dayofweek) groups by weekday (0 = Monday); dividing by the overall mean gives each weekday's factor.
  • This is a simple seasonal forecast: level × weekday factor. It ignores that December is still climbing; the next episode builds proper forecasting models and scores them.

Practice

1. Lose 10 random diary pages. What do asfreq("D") and interpolate() do?

Solution
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"]
gappy = ts.drop(ts.sample(10, random_state=3).index)            # 10 diary pages lost
print("rows:", len(gappy), "  biggest gap:", gappy.index.to_series().diff().max())
full = gappy.asfreq("D")
print("after asfreq: rows", len(full), "  missing:", full.isna().sum())
print("first missing day:", full[full.isna()].index[0].date())
print("filled by interpolation:", full.interpolate().isna().sum(), "missing left")

Output:

rows: 1086   biggest gap: 2 days 00:00:00
after asfreq: rows 1096   missing: 10
first missing day: 2023-03-27
filled by interpolation: 0 missing left

asfreq("D") brings the 10 missing days back as NaN; interpolate() fills each one with a straight line between its neighbours.

2. Weekly totals with weeks ending on Monday ("W-MON"). How does the first week change?

Solution
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"]
ts = ts.asfreq("D")
print("W     (weeks end Sunday):", *ts.resample("W").sum().head(3).values)
print("W-MON (weeks end Monday):", *ts.resample("W-MON").sum().head(3).values)
print("first W-MON bin ends on", ts.resample("W-MON").sum().index[0].date())

Output:

W     (weeks end Sunday): 323 1933 1846
W-MON (weeks end Monday): 566 1953 1796
first W-MON bin ends on 2023-01-02

The first bin now covers Sunday 1 and Monday 2 January: 2 days, still partial. The label of each weekly bin is its last day.

3. Decompose the daily series with period=7. What are the weekday factors?

Solution
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"]
from statsmodels.tsa.seasonal import seasonal_decompose
ts = ts.asfreq("D")
d = seasonal_decompose(ts, model="multiplicative", period=7)
names = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]
f = d.seasonal.groupby(d.seasonal.index.dayofweek).first()
print("weekday factors:", *(f"{n} {v:.2f}" for n, v in zip(names, f)))

Output:

weekday factors: Mon 0.92 Tue 0.92 Wed 0.92 Thu 0.92 Fri 0.93 Sat 1.20 Sun 1.19

Weekend ÷ weekday ≈ 1.20 / 0.92 ≈ 1.30: the 30% we built. The factors average to 1.

4. ACF and PACF of diff(1) on the log (instead of diff(7)). What stands out?

Solution
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"]
from statsmodels.tsa.stattools import acf, pacf
ts = ts.asfreq("D"); x = np.log(ts).diff().dropna()            # today minus yesterday, on the log
a, p = acf(x, nlags=14), pacf(x, nlags=14)
print("lag k:", *(f"{k:6}" for k in (1, 2, 5, 7, 14)))
print("ACF:  ", *(f"{a[k]:+6.2f}" for k in (1, 2, 5, 7, 14)))
print("PACF: ", *(f"{p[k]:+6.2f}" for k in (1, 2, 5, 7, 14)))

Output:

lag k:      1      2      5      7     14
ACF:    -0.02  -0.45  -0.44  +0.86  +0.85
PACF:   -0.02  -0.45  -0.66  +0.50  +0.30

diff(1) removes the trend but not the weekly season: lags 7 and 14 still have ACF 0.86 and 0.85. That's why seasonal differencing (diff(7)) is needed for this series.

Quick quiz

1. pd.Series([100, 120, 90, 130]).diff().tolist() prints? Options: [nan, 20.0, -30.0, 40.0], [20.0, -30.0, 40.0], [nan, 20.0, 30.0, 40.0], [100, 120, 90, 130].

Answer

[nan, 20.0, -30.0, 40.0]: each value minus the one before; the first has nothing before it.

2. Sales jump every December by about 30% of that year's level. Additive or multiplicative?

Answer

Multiplicative: the bump is a percentage, so it grows with the level.

3. A series' ACF is 0.5, 0.25, 0.12 at lags 1-3, and its PACF is 0.5, 0, 0. How many steps of direct memory?

Answer

One: only lag 1 has a direct link; the fading ACF is that link passed along (0.5, 0.5², 0.5³).

4. What does ts.rolling(7).mean() give on the 3rd day of the series?

Answer

NaN: a 7-day window needs 7 values; there are only 3.

5. Is a series with a steadily rising mean stationary?

Answer

No: its mean changes over time. Difference it (diff(1)) to remove the trend.

Interview questions

  1. What makes time-series data different from ordinary tabular data? Rows are ordered and not independent: nearby observations are related (autocorrelation), so you can't shuffle or split randomly.
  2. How do you prepare a raw date column in pandas? pd.to_datetime with an explicit format, set_index, sort_index, check pd.infer_freq and gaps, and asfreq to make missing periods explicit.
  3. Resampling vs rolling? Resampling changes the frequency (one row per week); rolling keeps the frequency and summarises a sliding window at each row.
  4. Trend vs seasonality vs cycles? Trend: long-term direction; seasonality: fixed calendar period; cycles: rises and falls of no fixed length (business cycles).
  5. How do you choose additive or multiplicative decomposition? If seasonal swings grow with the level, multiplicative (or log + additive); if they stay constant, additive. Compare the residuals.
  6. What is stationarity and why does it matter? Constant mean, variance and autocorrelation over time. Models such as ARIMA learn fixed rules from the past; non-stationary data breaks those rules.
  7. How do you make a series stationary? Difference it (diff(1) for trend, seasonal diff(s) for seasons), log or Box-Cox transform for growing variance; check with plots and the ADF test.
  8. ACF vs PACF? ACF: total correlation with lag k, including indirect effects; PACF: direct correlation after removing intermediate lags. A PACF that cuts off after lag p suggests an AR(p) model.
  9. What is a lag? The number of time steps back: lag 1 = the previous observation.
  10. Why check the edges after resampling? The first and last periods may be partial (our first week had 1 day), which distorts totals.

Glossary

Term Meaning (plain words + example)
Time series Measurements in time order, usually equally spaced: cups sold each day
Timestamp A real moment in time, not text: 2025-09-24 00:00:00
pd.to_datetime Converts text to timestamps; format="%d-%m-%Y" reads day-month-year
Timestamp index (DatetimeIndex) Row labels that are timestamps; needed for resample
set_index, sort_index Make a column the row labels; sort rows by those labels
Frequency The spacing between rows: D day, W week (ends Sunday), MS month start, h hour
pd.infer_freq Guesses the frequency from the index: D
asfreq Declares a frequency; missing periods appear as NaN
Resampling Changing the frequency: days → weeks with resample("W").sum()
Downsampling / upsampling To a coarser frequency (combine) / to a finer one (fill)
Rolling window A sliding block of the last k rows, summarised at every row: 7-day mean
center=True Put each row in the middle of its rolling window
Trend Long-term direction: +0.1 cup a day
Seasonality A pattern repeating with a fixed period: weekends (7 days), winter (1 year)
Period The length of one seasonal repeat: 7 days, 12 months
Cycles Rises and falls of no fixed length: a few boom years after an office opens
Noise (residual) What's left after trend and season: a surprise rainy day
Decomposition Splitting a series into trend, seasonal and residual parts
Additive model value = trend + seasonal + noise: winter adds +64.5 cups
Multiplicative model value = trend × seasonal × noise: winter × 1.23
seasonal_decompose statsmodels function that decomposes a series with a given period
Stationary Mean, spread and pattern don't change over time
Differencing Value minus an earlier value: diff(1) today − yesterday
Seasonal differencing Value minus the same season last time: diff(7)
Logarithm (log) Turns multiplying into adding; makes growing swings equal
Lag Steps back in time: lag 7 = a week ago
Autocorrelation Correlation of a series with itself shifted by a lag
ACF Autocorrelation function: autocorrelation at every lag (total link)
PACF Partial autocorrelation function: the direct link at each lag, middle removed
Noise band ±1.96 / √n: autocorrelations inside it could be chance
interpolate Fills NaNs with a straight line between neighbours
TypeError Python error for the wrong type: resample on a non-time index

Summary

  • A time series is ordered measurements; order is information.
  • Dates as text sort wrongly and resample fails; to_datetime + set_index + sort_index give a timestamp index; asfreq fixes the frequency and exposes gaps.
  • Resampling changes the frequency (check partial edges: first week 323); rolling windows smooth it (yearly mean 243.3 → 308.3).
  • A series = trend + seasonality + cycles + noise; when swings grow with the level, the multiplicative decomposition fits (January × 1.23).
  • Stationary = constant mean, spread and pattern. Differencing (diff(1), seasonal diff(7)) and log make Ramesh's series stationary.
  • ACF (total link) fades, PACF (direct link) shows one step of memory at lag 1: 0.62, as we built (0.6).

Next episode: Time series 2: classical forecasting: baselines, exponential smoothing, Holt-Winters, the ADF test, AR, MA, ARIMA and SARIMA.