Machine Learning Full Course · Episode 50. Every code sample below was run with Python 3.13, NumPy 2.5.3, pandas 3.0.5 and statsmodels 0.15.0; the output shown is the real output. Ramesh's chai stall and its numbers are made up for teaching. Because we built the data ourselves, we know exactly what is inside it (a slow growth, busier weekends, busier winters, and luck that carries over from one day to the next), which lets us check whether each tool finds it.
The big idea: a time series is a list of measurements in time order, usually equally spaced. Unlike the tables so far, order is information. Step one is a real timestamp index: dates typed as text sort wrongly ("01-01-2023, 01-01-2024, 01-01-2025, 01-02-2023") and resample refuses to run (TypeError: Only valid with DatetimeIndex, ...). With pd.to_datetime, set_index and sort_index you get a DatetimeIndex, a known frequency (D), and resampling (day → week or month). Rolling windows smooth the series: a 7-day mean hides weekends; a 365-day mean shows the growth (243.3 → 308.3 cups a day). A series is moved by trend, seasonality, cycles and noise; decomposition separates them. Here the seasonal swings grow with the level (January-July gap 77.2 → 126.2), so the multiplicative model fits (January × 1.23, leftover 1.20% of the trend vs 1.84% for additive). Many models need a stationary series (mean, spread and pattern constant over time): raw cups go from a mean of 251 to 302; differencing removes the trend (diff(1) mean ≈ 0) but the spread still grows (40.3 → 47.7); log then diff(7) is stable. The ACF measures the total link to k days back, the PACF only the direct link: after log and diff(7), the ACF fades (0.62, 0.37, 0.17, 0.10) while the PACF is 0.62 at lag 1 and then near 0. One direct step of memory, exactly as we built it.
Sources: no outside facts are claimed in this episode. Library behaviour (pandas to_datetime, resample, asfreq, rolling; statsmodels seasonal_decompose, acf, pacf) was checked by running it.
Words you need first
- Row, column: one row per day, one column per measured thing (
cups). - pandas DataFrame and Series (Episode 2): a DataFrame is a table; a Series is one column with an index (the row labels).
- Index: the labels of the rows. A default index is 0, 1, 2, ...; a time series uses dates.
- Mean and standard deviation (Episode 10): the average, and the typical distance from it. Example: 2, 4, 6 has mean 4 and standard deviation 2.
- Correlation (Episode 11): a number from −1 to +1 for how two things rise and fall together; 0 means no straight-line relation.
- Logarithm (log): the power you raise e to. It turns multiplying into adding: log(a × b) = log a + log b. So "25% more" becomes "+ the same amount" (log 1.25 ≈ 0.22).
- NaN: "not a number", pandas' marker for a missing value.
dtype: the type of a column's values:int64(whole numbers),float64(decimals),str(text),datetime64(timestamps).
The real-life hook: Ramesh's chai diary
Ramesh runs a chai stall and writes the date and the cups sold every evening: three years, 1,096 entries. He orders milk every week, so he wants to know: are weekends busier? Does winter sell more? Is the stall growing? He typed the diary from loose pages, so the rows are shuffled, and the dates are text in day-month-year form.
These ten lines create his diary; every program below starts with them (we call them HDR):
import numpy as np, pandas as pd
r = np.random.default_rng(50) # Ramesh's chai stall, made-up numbers
days = pd.date_range("2023-01-01", "2025-12-31", freq="D"); t = np.arange(len(days)) # 3 years of days
level = 200 + 0.1 * t # slow growth: +0.1 cup a day
week = np.where(days.dayofweek >= 5, 1.3, 1.0) # weekends 30% busier
year = 1 + 0.25 * np.cos(2 * np.pi * (days.dayofyear - 15) / 365.25) # winter busier, summer quieter
mood = np.zeros(len(t)); shock = r.normal(0, 0.05, len(t)) # random luck that carries over:
for i in range(1, len(t)): mood[i] = 0.6 * mood[i - 1] + shock[i] # 60% of yesterday's luck stays
cups = (level * week * year * np.exp(mood)).round().astype(int)
diary = pd.DataFrame({"date": days.strftime("%d-%m-%Y"), "cups": cups}).sample(frac=1, random_state=1)
pd.date_range(..., freq="D"): every day from 1 Jan 2023 to 31 Dec 2025.level: 200 cups a day, plus 0.1 cup for every day that passes (the trend).week: Saturday and Sunday (dayofweek5 and 6; Monday is 0) are 1.3 times busier.year: a cosine wave over the year, highest in mid-January (× 1.25) and lowest in mid-July (× 0.75).mood: random luck where each day keeps 60% of yesterday's luck, plus a fresh random shock. This is what gives the series memory.- Everything is multiplied, so the weekend and winter bumps grow as the stall grows.
strftime("%d-%m-%Y")writes each date as text like24-09-2025;sample(frac=1)shuffles all the rows.
What a time series is
A time series is a list of measurements taken one after another in time, usually equally spaced: daily sales, hourly temperature, a share's closing price each day. What makes it special: the order carries information. Today's sales are close to yesterday's; shuffle the rows and that is lost.
The obvious attempt: sort the text and resample
# HDR (10 lines above), then:
import textwrap
print(diary.head(3).to_string(index=False)); print("rows:", len(diary), " date column type:", diary["date"].dtype)
s = diary.sort_values("date") # obvious: sort by the date text
print("sorted:", *s["date"].head(4))
try: s.resample("W")["cups"].sum() # obvious: weekly totals
except TypeError as e: print(*textwrap.wrap("TypeError: " + str(e), 62), sep="\n")
Output:
date cups
24-09-2025 245
11-03-2024 282
27-10-2025 340
rows: 1096 date column type: str
sorted: 01-01-2023 01-01-2024 01-01-2025 01-02-2023
TypeError: Only valid with DatetimeIndex, TimedeltaIndex or
PeriodIndex, but got an instance of 'Index'
- The date column is
str: text. Text is compared letter by letter, so all the "01-..." days come first, from every year, then "01-02-2023". resampleneeds the rows labelled by time. Pandas has no idea these strings are dates.
The timestamp index
- A timestamp is a real moment in time (
2025-09-24 00:00:00), not text.pd.to_datetime(col, format="%d-%m-%Y")converts; the format (%dday,%mmonth,%Yfour-digit year) makes sure 04-03 is read as 4 March, not 3 April. - A timestamp index (
DatetimeIndex) labels each row with its timestamp:set_index("date"), thensort_index()puts rows in time order. Now pandas can do date arithmetic. - The frequency is the spacing between rows:
Ddaily,Wweekly (ending Sunday),MSmonth start,MEmonth end,hhourly. asfreq("D")declares "one row per day". A missing day appears as aNaNrow instead of silently vanishing.
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y") # text -> real timestamps
ts = diary.set_index("date").sort_index()["cups"] # timestamp index, in time order
print("sorted:", *ts.index[:3].date, " ...", ts.index[-1].date())
print("frequency:", pd.infer_freq(ts.index), " gaps:", ts.index.to_series().diff().max())
ts = ts.asfreq("D") # declare: one row per day
print("weekly totals:", *ts.resample("W").sum().head(3).values)
print("monthly means:", *ts.resample("MS").mean().head(4).round(1).values)
print("Saturdays vs Mondays:", ts[ts.index.dayofweek == 5].mean().round(1), ts[ts.index.dayofweek == 0].mean().round(1))
Output:
sorted: 2023-01-01 2023-01-02 2023-01-03 ... 2025-12-31
frequency: D gaps: 1 days 00:00:00
weekly totals: 323 1933 1846
monthly means: 261.1 271.2 257.9 235.3
Saturdays vs Mondays: 332.2 254.1
pd.infer_freqguesses the spacing:D. The biggest gap between consecutive dates is 1 day: nothing is missing.- The first weekly total is only 323: weeks end on Sunday and 1 January 2023 was a Sunday, so the first "week" holds one day. Always check the edges.
- Saturdays average 332.2 cups, Mondays 254.1: the weekend pattern (332.2 / 254.1 ≈ 1.31, close to the 1.3 we built).
Resampling and rolling windows
Resampling changes the frequency:
| Kind | Example | You must choose |
|---|---|---|
| Downsampling | days → weeks, resample("W") |
how to combine: sum() for total cups, mean() for a typical day |
| Upsampling | months → days, resample("D") |
how to fill the new rows: ffill() (repeat), interpolate() |
A rolling window keeps every row, but at each row it summarises the last k rows: like a batsman's average over his last seven matches, updated after every match. A bigger window is smoother but slower to react. center=True puts each row in the middle of its window.
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"].asfreq("D")
roll7 = ts.rolling(7).mean() # mean of the last 7 days, at each day
print(pd.DataFrame({"cups": ts, "7-day mean": roll7.round(1)}).iloc[3:9].to_string())
roll365 = ts.rolling(365, center=True).mean() # a whole year averages out the seasons
print("yearly mean, Aug 2023 vs Jun 2025:", roll365["2023-08-01"].round(1), roll365["2025-06-01"].round(1))
Output:
cups 7-day mean
date
2023-01-04 235 NaN
2023-01-05 262 NaN
2023-01-06 255 NaN
2023-01-07 338 272.9
2023-01-08 346 276.1
2023-01-09 263 279.0
yearly mean, Aug 2023 vs Jun 2025: 243.3 308.3
- The first 6 days have no 7-day mean (
NaN): there aren't 7 days behind them yet. - A whole year averages out weekends and winters, leaving the growth: 243.3 → 308.3 cups a day.
- On the plot: the daily line jumps every weekend; the 7-day mean shows the winter waves; the 365-day mean is a slow, steady climb, missing half a year at each end (a centred window needs both sides). The winter waves get taller as the stall grows.
Trend, seasonality, cycles and noise
| Piece | Plain meaning | At the chai stall |
|---|---|---|
| Trend | The long-term direction | The stall slowly gets famous: +0.1 cup a day |
| Seasonality | A pattern repeating with a fixed period set by the calendar | Weekends (period 7 days), winter (period 1 year) |
| Cycles | Rises and falls with no fixed length | An office opens nearby, business booms for some years, then it shuts (none in our data) |
| Noise | What is left: random | A surprise rainy day |
Decomposition separates a series into these pieces (cycles usually end up inside the trend).
Additive vs multiplicative
- Additive: value = trend + seasonal + noise. Winter adds the same number of cups every year (say +60).
- Multiplicative: value = trend × seasonal × noise. Winter adds the same percentage (say +25%), so the bump grows as the stall grows.
- To choose: do the waves get taller as the series rises? Then multiplicative (or take the log and use additive).
# HDR (10 lines above), then:
from statsmodels.tsa.seasonal import seasonal_decompose
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
m = diary.set_index("date").sort_index()["cups"].resample("MS").mean() # 36 monthly means
for model in ("additive", "multiplicative"):
d = seasonal_decompose(m, model=model, period=12) # trend + seasonal + residual
left = d.resid / d.trend if model == "additive" else d.resid - 1 # leftover as a share of the trend
print(f"{model:14} Jan {d.seasonal.iloc[0]:6.2f} Jul {d.seasonal.iloc[6]:6.2f} leftover {left.abs().mean():.2%}")
gap = lambda y: (m[f"{y}-01"].iloc[0] - m[f"{y}-07"].iloc[0]).round(1) # January minus July
print("Jan peak-to-dip: 2023", gap(2023), " 2025", gap(2025))
Output:
additive Jan 64.50 Jul -63.20 leftover 1.84%
multiplicative Jan 1.23 Jul 0.76 leftover 1.20%
Jan peak-to-dip: 2023 77.2 2025 126.2
seasonal_decompose(m, model=..., period=12)finds the trend with a centred 12-month moving average, then the average seasonal effect of each month, and calls the rest the residual.- Additive: January +64.5 cups, July −63.2. Multiplicative: January × 1.23, July × 0.76 (we built × 1.25 and × 0.75).
- "Leftover" is the residual as a share of the trend, averaged over the months (absolute values): for additive, residual ÷ trend; for multiplicative the residual is already a factor near 1, so residual − 1. 1.84% for additive vs 1.20% for multiplicative: the multiplicative model explains more.
- The January-July gap grew from 77.2 to 126.2 cups: the swings grow with the level, so multiplicative wins.
- The trend (
d.trend) is missing 6 months at each end: it is a centred 12-month average, so it runs from July 2023 to June 2025.
Stationarity and differencing
A series is stationary when its mean, spread and pattern do not change over time. Many forecasting models assume it: they learn rules from the past, and if the average keeps climbing, those rules are out of date by tomorrow.
- Differencing: model the change instead of the value.
diff(1)= today − yesterday (removes a trend).diff(7)= today − the same weekday last week (removes the weekly pattern; called seasonal differencing). - If the swings grow with the level, take
np.logfirst: growing swings become equal ones.
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"].asfreq("D")
def halves(x): # first half of the days vs second half
x = x.dropna(); a, b = x[: len(x) // 2], x[len(x) // 2 :]
return f"mean {a.mean():7.3f} -> {b.mean():7.3f} std {a.std():6.3f} -> {b.std():6.3f}"
print("raw cups: ", halves(ts))
print("diff(1): ", halves(ts.diff())) # today minus yesterday
print("log, diff(7): ", halves(np.log(ts).diff(7))) # log tames growth, diff(7) removes the week
Output:
raw cups: mean 251.257 -> 301.774 std 56.166 -> 71.616
diff(1): mean -0.245 -> 0.407 std 40.276 -> 47.663
log, diff(7): mean -0.003 -> 0.009 std 0.091 -> 0.088
- Raw: the mean jumps 251 → 302 and the spread 56 → 72: not stationary.
diff(1): the mean is near 0 in both halves, but the spread still grows (40.3 → 47.7) because the swings grow.- Log then
diff(7): mean ≈ 0, spread 0.091 vs 0.088: it looks stationary. A formal test (the ADF test) comes in the next episode. x.dropna()removes the NaN rows that differencing creates at the start (diff(7) has 7).
ACF and PACF: measuring memory
A daughter cooks a lot like her grandmother, but she learnt only from her mother, who learnt from grandma. The link is real, but it passes through the middle.
- The lag is how many steps back you look: lag 1 = yesterday, lag 7 = a week ago.
- Autocorrelation function (ACF) at lag k: the correlation between the series and itself k steps earlier. It is the total link, including what is passed on through the days in between. (Daughter and grandma: high.)
- Partial autocorrelation function (PACF) at lag k: the direct link after removing the effect of the days in between. (Daughter and grandma: zero.)
- Rough noise band: with n points, values inside ±1.96 / √n could be pure chance (95%).
# HDR (10 lines above), then:
from statsmodels.tsa.stattools import acf, pacf
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"].asfreq("D")
for name, x in [("raw cups", ts), ("log, diff(7)", np.log(ts).diff(7).dropna())]:
a, p = acf(x, nlags=14), pacf(x, nlags=14) # correlation with k days earlier
print(name); print(" lag k:", *(f"{k:6}" for k in (1, 2, 3, 4, 7, 14)))
print(" ACF: ", *(f"{a[k]:+6.2f}" for k in (1, 2, 3, 4, 7, 14)))
print(" PACF: ", *(f"{p[k]:+6.2f}" for k in (1, 2, 3, 4, 7, 14)))
print("noise band (95%): +/-", round(1.96 / np.sqrt(len(ts)), 2))
Output:
raw cups
lag k: 1 2 3 4 7 14
ACF: +0.79 +0.60 +0.59 +0.58 +0.91 +0.88
PACF: +0.80 -0.09 +0.39 -0.01 +0.55 +0.23
log, diff(7)
lag k: 1 2 3 4 7 14
ACF: +0.62 +0.37 +0.17 +0.10 -0.40 +0.05
PACF: +0.62 -0.03 -0.07 +0.04 -0.30 -0.14
noise band (95%): +/- 0.06
- Raw cups: everything is strongly correlated, and lag 7 is the biggest (0.91): the weekly season.
- After log and
diff(7): the ACF fades (0.62, 0.37, 0.17, 0.10) but the PACF is 0.62 at lag 1 and then nearly 0. Only yesterday matters directly; the rest is yesterday's luck passed along. We built the data with 60% of yesterday's luck carrying over, and 0.6² ≈ 0.36, 0.6³ ≈ 0.22: the fading ACF. - The negative values near lag 7 (ACF −0.40, PACF −0.30) are an echo of the weekly differencing itself; the seasonal models of the next episode handle them.
- On the bar chart, the ACF bars shrink over the first lags and the PACF has one tall bar at lag 1. Reading these shapes tells you which model to pick next episode.
Common mistakes
| Mistake | Exact message / symptom | Why | Fix |
|---|---|---|---|
| Sorting dates stored as text | sorted: 01-01-2023 01-01-2024 01-01-2025 01-02-2023 |
Text is compared letter by letter | pd.to_datetime(col, format="%d-%m-%Y") |
resample on a plain column |
TypeError: Only valid with DatetimeIndex, TimedeltaIndex or PeriodIndex, but got an instance of 'Index' |
The index is not time | set_index("date").sort_index() |
to_datetime without a format on DD-MM dates |
04-03 may be read as 3 April | Day/month ambiguity | Always pass format= (or dayfirst=True) |
| Trusting a partial first or last period | first weekly total 323 (one day) | The period isn't complete | Check the edges; drop partial periods |
| Missing days not noticed | 1,086 rows, gaps of 2 days | Rows silently absent | asfreq("D") shows them as NaN |
| Additive model when swings grow | leftover 1.84% vs 1.20% | The season is a percentage | model="multiplicative" or log first |
| Modelling a trending series raw | mean 251 → 302 | Not stationary | diff, np.log, seasonal diff(7) |
| Shuffling a time series | the order (memory) is destroyed | Rows are not independent | Keep time order |
Mini project: next week's milk order
Use the last three months to get each weekday's share, scale it to the current level (the last 28 days), and turn cups into litres (100 ml of milk a cup).
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"]
ts = ts.asfreq("D")
last = ts["2025-10-01":] # the last 3 months only
profile = last.groupby(last.index.dayofweek).mean() / last.mean() # each weekday vs an average day
level = last.rolling(28).mean().iloc[-1] # where we are now: the last 4 weeks
names = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]
plan = pd.Series((level * profile).round().astype(int).values, index=names)
print("current level:", round(level, 1), "cups a day")
print(plan.to_string())
print("milk for next week (100 ml a cup):", round(plan.sum() * 0.1, 1), "litres")
Output:
current level: 415.3 cups a day
Mon 385
Tue 389
Wed 383
Thu 382
Fri 384
Sat 497
Sun 489
milk for next week (100 ml a cup): 290.9 litres
ts["2025-10-01":]selects by date: from 1 October 2025 to the end.groupby(last.index.dayofweek)groups by weekday (0 = Monday); dividing by the overall mean gives each weekday's factor.- This is a simple seasonal forecast: level × weekday factor. It ignores that December is still climbing; the next episode builds proper forecasting models and scores them.
Practice
1. Lose 10 random diary pages. What do asfreq("D") and interpolate() do?
Solution
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"]
gappy = ts.drop(ts.sample(10, random_state=3).index) # 10 diary pages lost
print("rows:", len(gappy), " biggest gap:", gappy.index.to_series().diff().max())
full = gappy.asfreq("D")
print("after asfreq: rows", len(full), " missing:", full.isna().sum())
print("first missing day:", full[full.isna()].index[0].date())
print("filled by interpolation:", full.interpolate().isna().sum(), "missing left")
Output:
rows: 1086 biggest gap: 2 days 00:00:00
after asfreq: rows 1096 missing: 10
first missing day: 2023-03-27
filled by interpolation: 0 missing left
asfreq("D") brings the 10 missing days back as NaN; interpolate() fills each one with a straight line between its neighbours.
2. Weekly totals with weeks ending on Monday ("W-MON"). How does the first week change?
Solution
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"]
ts = ts.asfreq("D")
print("W (weeks end Sunday):", *ts.resample("W").sum().head(3).values)
print("W-MON (weeks end Monday):", *ts.resample("W-MON").sum().head(3).values)
print("first W-MON bin ends on", ts.resample("W-MON").sum().index[0].date())
Output:
W (weeks end Sunday): 323 1933 1846
W-MON (weeks end Monday): 566 1953 1796
first W-MON bin ends on 2023-01-02
The first bin now covers Sunday 1 and Monday 2 January: 2 days, still partial. The label of each weekly bin is its last day.
3. Decompose the daily series with period=7. What are the weekday factors?
Solution
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"]
from statsmodels.tsa.seasonal import seasonal_decompose
ts = ts.asfreq("D")
d = seasonal_decompose(ts, model="multiplicative", period=7)
names = ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"]
f = d.seasonal.groupby(d.seasonal.index.dayofweek).first()
print("weekday factors:", *(f"{n} {v:.2f}" for n, v in zip(names, f)))
Output:
weekday factors: Mon 0.92 Tue 0.92 Wed 0.92 Thu 0.92 Fri 0.93 Sat 1.20 Sun 1.19
Weekend ÷ weekday ≈ 1.20 / 0.92 ≈ 1.30: the 30% we built. The factors average to 1.
4. ACF and PACF of diff(1) on the log (instead of diff(7)). What stands out?
Solution
# HDR (10 lines above), then:
diary["date"] = pd.to_datetime(diary["date"], format="%d-%m-%Y")
ts = diary.set_index("date").sort_index()["cups"]
from statsmodels.tsa.stattools import acf, pacf
ts = ts.asfreq("D"); x = np.log(ts).diff().dropna() # today minus yesterday, on the log
a, p = acf(x, nlags=14), pacf(x, nlags=14)
print("lag k:", *(f"{k:6}" for k in (1, 2, 5, 7, 14)))
print("ACF: ", *(f"{a[k]:+6.2f}" for k in (1, 2, 5, 7, 14)))
print("PACF: ", *(f"{p[k]:+6.2f}" for k in (1, 2, 5, 7, 14)))
Output:
lag k: 1 2 5 7 14
ACF: -0.02 -0.45 -0.44 +0.86 +0.85
PACF: -0.02 -0.45 -0.66 +0.50 +0.30
diff(1) removes the trend but not the weekly season: lags 7 and 14 still have ACF 0.86 and 0.85. That's why seasonal differencing (diff(7)) is needed for this series.
Quick quiz
1. pd.Series([100, 120, 90, 130]).diff().tolist() prints? Options: [nan, 20.0, -30.0, 40.0], [20.0, -30.0, 40.0], [nan, 20.0, 30.0, 40.0], [100, 120, 90, 130].
Answer
[nan, 20.0, -30.0, 40.0]: each value minus the one before; the first has nothing before it.
2. Sales jump every December by about 30% of that year's level. Additive or multiplicative?
Answer
Multiplicative: the bump is a percentage, so it grows with the level.
3. A series' ACF is 0.5, 0.25, 0.12 at lags 1-3, and its PACF is 0.5, 0, 0. How many steps of direct memory?
Answer
One: only lag 1 has a direct link; the fading ACF is that link passed along (0.5, 0.5², 0.5³).
4. What does ts.rolling(7).mean() give on the 3rd day of the series?
Answer
NaN: a 7-day window needs 7 values; there are only 3.
5. Is a series with a steadily rising mean stationary?
Answer
No: its mean changes over time. Difference it (diff(1)) to remove the trend.
Interview questions
- What makes time-series data different from ordinary tabular data? Rows are ordered and not independent: nearby observations are related (autocorrelation), so you can't shuffle or split randomly.
- How do you prepare a raw date column in pandas?
pd.to_datetimewith an explicit format,set_index,sort_index, checkpd.infer_freqand gaps, andasfreqto make missing periods explicit. - Resampling vs rolling? Resampling changes the frequency (one row per week); rolling keeps the frequency and summarises a sliding window at each row.
- Trend vs seasonality vs cycles? Trend: long-term direction; seasonality: fixed calendar period; cycles: rises and falls of no fixed length (business cycles).
- How do you choose additive or multiplicative decomposition? If seasonal swings grow with the level, multiplicative (or log + additive); if they stay constant, additive. Compare the residuals.
- What is stationarity and why does it matter? Constant mean, variance and autocorrelation over time. Models such as ARIMA learn fixed rules from the past; non-stationary data breaks those rules.
- How do you make a series stationary? Difference it (
diff(1)for trend, seasonaldiff(s)for seasons), log or Box-Cox transform for growing variance; check with plots and the ADF test. - ACF vs PACF? ACF: total correlation with lag k, including indirect effects; PACF: direct correlation after removing intermediate lags. A PACF that cuts off after lag p suggests an AR(p) model.
- What is a lag? The number of time steps back: lag 1 = the previous observation.
- Why check the edges after resampling? The first and last periods may be partial (our first week had 1 day), which distorts totals.
Glossary
| Term | Meaning (plain words + example) |
|---|---|
| Time series | Measurements in time order, usually equally spaced: cups sold each day |
| Timestamp | A real moment in time, not text: 2025-09-24 00:00:00 |
pd.to_datetime |
Converts text to timestamps; format="%d-%m-%Y" reads day-month-year |
Timestamp index (DatetimeIndex) |
Row labels that are timestamps; needed for resample |
set_index, sort_index |
Make a column the row labels; sort rows by those labels |
| Frequency | The spacing between rows: D day, W week (ends Sunday), MS month start, h hour |
pd.infer_freq |
Guesses the frequency from the index: D |
asfreq |
Declares a frequency; missing periods appear as NaN |
| Resampling | Changing the frequency: days → weeks with resample("W").sum() |
| Downsampling / upsampling | To a coarser frequency (combine) / to a finer one (fill) |
| Rolling window | A sliding block of the last k rows, summarised at every row: 7-day mean |
center=True |
Put each row in the middle of its rolling window |
| Trend | Long-term direction: +0.1 cup a day |
| Seasonality | A pattern repeating with a fixed period: weekends (7 days), winter (1 year) |
| Period | The length of one seasonal repeat: 7 days, 12 months |
| Cycles | Rises and falls of no fixed length: a few boom years after an office opens |
| Noise (residual) | What's left after trend and season: a surprise rainy day |
| Decomposition | Splitting a series into trend, seasonal and residual parts |
| Additive model | value = trend + seasonal + noise: winter adds +64.5 cups |
| Multiplicative model | value = trend × seasonal × noise: winter × 1.23 |
seasonal_decompose |
statsmodels function that decomposes a series with a given period |
| Stationary | Mean, spread and pattern don't change over time |
| Differencing | Value minus an earlier value: diff(1) today − yesterday |
| Seasonal differencing | Value minus the same season last time: diff(7) |
| Logarithm (log) | Turns multiplying into adding; makes growing swings equal |
| Lag | Steps back in time: lag 7 = a week ago |
| Autocorrelation | Correlation of a series with itself shifted by a lag |
| ACF | Autocorrelation function: autocorrelation at every lag (total link) |
| PACF | Partial autocorrelation function: the direct link at each lag, middle removed |
| Noise band | ±1.96 / √n: autocorrelations inside it could be chance |
interpolate |
Fills NaNs with a straight line between neighbours |
TypeError |
Python error for the wrong type: resample on a non-time index |
Summary
- A time series is ordered measurements; order is information.
- Dates as text sort wrongly and
resamplefails;to_datetime+set_index+sort_indexgive a timestamp index;asfreqfixes the frequency and exposes gaps. - Resampling changes the frequency (check partial edges: first week 323); rolling windows smooth it (yearly mean 243.3 → 308.3).
- A series = trend + seasonality + cycles + noise; when swings grow with the level, the multiplicative decomposition fits (January × 1.23).
- Stationary = constant mean, spread and pattern. Differencing (
diff(1), seasonaldiff(7)) and log make Ramesh's series stationary. - ACF (total link) fades, PACF (direct link) shows one step of memory at lag 1: 0.62, as we built (0.6).
Next episode: Time series 2: classical forecasting: baselines, exponential smoothing, Holt-Winters, the ADF test, AR, MA, ARIMA and SARIMA.