Machine Learning Full Course · Episode 16. Every code sample below was run with Python 3.13, NumPy 2.5.3, pandas 3.0.5 and scikit-learn 1.9.1; the output shown is the real output. All data is made up for teaching.
The big idea: a model only sees numbers. Two things go wrong before it learns anything: numbers in big units bully numbers in small units, and text columns aren't numbers at all. Feature scaling fixes the first, encoding fixes the second, and the rule fit on train, transform everything keeps both fixes from leaking.
The problem: the interview that never mattered
A college adds two scores: board marks out of 500 and an interview out of 10. Asha: 420 + 10 = 430. Ravi: 430 + 2 = 432. Ravi wins, and the interview can never change the result, because its numbers are fifty times smaller. Any model that adds or compares raw features falls into the same trap.
A feature (Episode 1) is one input column: area, distance to the metro, city. Each flat below is a vector of two features (Episode 3), and the distance between two vectors says how similar two flats are:
import numpy as np
# a flat = [area in sq ft, distance to the metro in km]
A = np.array([900, 0.5])
B = np.array([910, 9.0]) # same size, far from the metro
C = np.array([1000, 0.6]) # a bit bigger, next to the metro
print("A to B:", round(np.linalg.norm(A - B), 1))
print("A to C:", round(np.linalg.norm(A - C), 1))
Output:
A to B: 13.1
A to C: 100.0
B is the same size as A but 9 km from the metro; C is a little bigger but right next to the metro. Common sense says C is the better match, yet the maths says B: a 100 sq ft difference counts as 100, an 8.5 km difference counts as only 8.5. Area is measured in bigger numbers, so it dominates.
Feature scaling
Feature scaling means changing each feature's numbers so that all features sit on a similar range, without changing their order (a bigger flat is still bigger after scaling).
| Model family | Needs scaling? | Why |
|---|---|---|
| k-nearest neighbours, k-means, SVM | yes | they measure distances |
| linear / logistic regression trained with gradient descent | yes | big features make giant gradient steps (Episode 18) |
| regularised linear models (ridge, lasso) | yes | the penalty treats every weight equally (Episode 21) |
| PCA | yes | it looks for directions of big variance |
| decision trees, random forests, gradient boosting | no | they only ask "is area > 950?"; the order is all that matters |
Min-max normalisation
The college's fix: turn every score into a percentage. 420/500 = 84%, 10/10 = 100%. In ML this is min-max normalisation (also called min-max scaling):
x_scaled = (x − min) / (max − min) → every value lands between 0 and 1
import numpy as np
area = np.array([900, 910, 1000, 650, 1400]) # A, B, C + 2 more
km = np.array([0.5, 9.0, 0.6, 3.0, 5.0])
def minmax(x): return (x - x.min()) / (x.max() - x.min())
a, k = minmax(area), minmax(km)
print("area scaled:", a.round(2))
print("km scaled: ", k.round(2))
P = np.c_[a, k] # back to points
print("A to B:", round(np.linalg.norm(P[0] - P[1]), 2))
print("A to C:", round(np.linalg.norm(P[0] - P[2]), 2))
Output:
area scaled: [0.33 0.35 0.47 0. 1. ]
km scaled: [0. 1. 0.01 0.29 0.53]
A to B: 1.0
A to C: 0.13
The smallest flat becomes 0, the biggest 1; 9 km becomes 1. Now A to C (0.13) is much closer than A to B (1.0), which matches common sense.
Weakness: outliers. An outlier is a value far away from the rest (Episode 14). Add one 9,000 sq ft villa and the maximum becomes 9,000, so every normal flat is squashed between 0 and 0.09 (see the comparison below).
Standardisation (z-score)
In a class, the average height is 150 cm and most children are within about 10 cm of it (that typical distance is the standard deviation, Episode 10). Raj is 170 cm: 20 cm above average = two standard deviations above. His z-score is +2. A z-score says how unusual a value is, whatever its unit.
z = (x − mean) / std → the new feature has mean 0 and standard deviation 1
Turning a feature into z-scores is standardisation; scikit-learn's StandardScaler does it. It is the usual default for most models.
import numpy as np
area = np.array([650, 900, 910, 1000, 1400])
z = (area - area.mean()) / area.std() # standardisation
print("mean", area.mean(), " std", area.std().round(1))
print("z-scores:", z.round(2))
print("new mean", z.mean().round(2), " new std", z.std().round(2))
Output:
mean 972.0 std 243.6
z-scores: [-1.32 -0.3 -0.25 0.11 1.76]
new mean 0.0 new std 1.0
(np.std divides by n, the population standard deviation, which is also what StandardScaler uses.)
Robust scaling
Ten customers at a chai stall earn about ₹30,000 a month: mean and median (the middle value) both say 30,000. A billionaire walks in: the mean and the standard deviation explode, but the median barely moves. Neither does the interquartile range (IQR), the width of the middle half of the data, Q3 − Q1 (Episode 11).
x_robust = (x − median) / IQR
That's robust scaling (RobustScaler). All three side by side, with the villa:
import numpy as np
from sklearn.preprocessing import MinMaxScaler, StandardScaler, RobustScaler
area = np.array([[650], [900], [910], [1000], [1400], [9000]]) # + a villa
for s in [MinMaxScaler(), StandardScaler(), RobustScaler()]:
print(f"{type(s).__name__:15}", s.fit_transform(area).ravel().round(2))
Output:
MinMaxScaler [0. 0.03 0.03 0.04 0.09 1. ]
StandardScaler [-0.55 -0.47 -0.47 -0.44 -0.3 2.23]
RobustScaler [-0.77 -0.14 -0.11 0.11 1.12 20.24]
- Min-max: the five normal flats are crushed into 0 to 0.09.
- Standard: better, but they're bunched between −0.55 and −0.30 because the villa dragged the mean and the std.
- Robust: the normal flats spread from −0.77 to 1.12; the villa stays far away at 20.24. The outlier stays an outlier and doesn't damage everyone else.
| Scaler | Formula | Result | Use when |
|---|---|---|---|
MinMaxScaler |
(x − min) / (max − min) | 0 to 1 | a fixed range is natural (pixels 0 to 255), no big outliers |
StandardScaler |
(x − mean) / std | mean 0, std 1 | the default for most models |
RobustScaler |
(x − median) / IQR | median 0 | the column has outliers |
Scaling does not remove outliers; if the villa is a data error, fix it in cleaning (Episode 14).
Encoding categorical features
A column of labels like city is a categorical feature (Episode 13). A model is maths, so it can't use text:
import sys; sys.tracebacklimit = 0 # show only the error line
import pandas as pd
from sklearn.linear_model import LinearRegression
df = pd.DataFrame({"area": [900, 1200, 700, 1000, 850, 1100],
"city": ["Pune", "Delhi", "Mumbai", "Pune", "Delhi", "Mumbai"],
"rent": [18000, 29000, 34000, 20000, 22000, 42000]})
model = LinearRegression().fit(df[["area", "city"]], df["rent"])
Output:
ValueError: could not convert string to float: 'Pune'
Encoding means turning each category into numbers. There are two honest ways, depending on whether the categories have an order.
The obvious way fails: cities as 0, 1, 2
import numpy as np, pandas as pd
from sklearn.linear_model import LinearRegression
rng = np.random.default_rng(16)
city = rng.choice(["Delhi", "Mumbai", "Pune"], 300)
area = rng.normal(950, 250, 300).round()
extra = pd.Series(city).map({"Delhi": 5000, "Mumbai": 20000, "Pune": 0})
rent = area * 20 + extra + rng.normal(0, 1000, 300)
def mae(X):
m = LinearRegression().fit(X, rent)
return round(np.abs(m.predict(X) - rent).mean())
code = pd.Series(city).map({"Delhi": 0, "Mumbai": 1, "Pune": 2})
print("city as 0, 1, 2:", mae(np.c_[area, code]))
onehot = pd.get_dummies(pd.Series(city), dtype=int)
print(onehot.head(3))
print("one-hot cities: ", mae(np.c_[area, onehot]))
Output:
city as 0, 1, 2: 7455
Delhi Mumbai Pune
0 0 1 0
1 0 1 0
2 0 0 1
one-hot cities: 730
Same model, same data: 7,455 with codes 0/1/2 versus 730 with one-hot columns. A linear model believes the numbers: with Delhi = 0, Mumbai = 1, Pune = 2 it must add the same amount for each step Delhi → Mumbai → Pune, and it thinks Mumbai is halfway between the other two. In the data Mumbai is the most expensive. The codes invented an order that doesn't exist.
One-hot encoding
Like an OMR sheet: one bubble per option, exactly one filled. One-hot encoding makes one column per category, with a 1 in the column of the row's category and 0 everywhere else:
| city | Delhi | Mumbai | Pune |
|---|---|---|---|
| Delhi | 1 | 0 | 0 |
| Mumbai | 0 | 1 | 0 |
| Pune | 0 | 0 | 1 |
No column is "bigger" than another, so there's no fake order; the model learns a separate amount for each city. In pandas: pd.get_dummies; in scikit-learn: OneHotEncoder (which remembers the categories, so it can be used again on test and live data: always prefer it in real projects). One-hot columns are also called dummy variables.
The dummy variable trap
If a flat is not in Delhi and not in Mumbai, it must be in Pune: the third column adds no information. Worse, the city columns always add up to 1, which is exactly the bias column of ones that a linear model has. One column is a copy of a combination of others (perfect multicollinearity, Episode 19). This is the dummy variable trap.
import numpy as np, pandas as pd
city = pd.Series(["Delhi", "Mumbai", "Pune", "Delhi", "Pune", "Mumbai"])
for drop in [False, True]:
onehot = pd.get_dummies(city, dtype=float, drop_first=drop)
X = np.c_[np.ones(6), onehot] # bias column of 1s + city columns
print("columns:", ["bias"] + onehot.columns.tolist())
print(" rank:", np.linalg.matrix_rank(X), "of", X.shape[1], "columns")
print(" biggest number in inv(XᵀX):", f"{np.abs(np.linalg.inv(X.T @ X)).max():.3g}")
Output:
columns: ['bias', 'Delhi', 'Mumbai', 'Pune']
rank: 3 of 4 columns
biggest number in inv(XᵀX): 4.5e+15
columns: ['bias', 'Mumbai', 'Pune']
rank: 3 of 3 columns
biggest number in inv(XᵀX): 1
- The rank (Episode 4: the number of truly independent columns) is 3 of 4, so XᵀX has no real inverse; floating-point rounding turned it into junk like 4.5 × 10¹⁵.
- With
drop_first=True(pandas) orOneHotEncoder(drop="first"), one category (here Delhi, the first alphabetically) becomes the baseline: the Mumbai and Pune weights say how much more or less they cost than Delhi. - scikit-learn's
LinearRegressiondoes not crash on the full one-hot (it uses a least-squares solver), and regularised models and trees don't mind it. But for plain linear and logistic regression the weights are not unique and can't be interpreted. Drop one there.
Ordinal encoding
Some categories do have an order: T-shirt sizes S < M < L, floor level low < mid < high. These are ordinal features. Numbering them 0, 1, 2 is honest because the order is real: that's ordinal encoding (one column instead of many). It's also often called label encoding.
from sklearn.preprocessing import OrdinalEncoder
floor = [["low"], ["mid"], ["high"], ["mid"], ["low"]]
enc = OrdinalEncoder() # default: alphabetical order
print(enc.fit_transform(floor).ravel(), enc.categories_)
enc = OrdinalEncoder(categories=[["low", "mid", "high"]])
print(enc.fit_transform(floor).ravel())
Output:
[1. 2. 0. 2. 1.] [array(['high', 'low', 'mid'], dtype=object)]
[0. 1. 2. 1. 0.]
The default OrdinalEncoder() sorts categories alphabetically (h, l, m), so high = 0, low = 1, mid = 2: high floors look lower than low floors. Always pass categories=[[...]] in the real order.
LabelEncoder does the same alphabetical numbering, but it's meant for the target column y (e.g. class names → 0, 1, 2), not for features.
| Encoding | Columns | Use when | Example |
|---|---|---|---|
| one-hot | one per category | no order | city, payment method, colour |
| one-hot, drop first | one per category − 1 | no order + plain linear/logistic regression | city with a bias column |
| ordinal | 1 | a real order | size S/M/L, floor low/mid/high, education level |
Unknown categories
The encoder learned Delhi, Mumbai and Pune from the training data. After the model goes live, someone asks about Jaipur. A category the encoder never saw during fit is an unknown category:
import sys; sys.tracebacklimit = 0 # show only the error line
from sklearn.preprocessing import OneHotEncoder
train = [["Delhi"], ["Mumbai"], ["Pune"], ["Delhi"]]
enc = OneHotEncoder().fit(train)
print(enc.categories_)
print(enc.transform([["Pune"]]).toarray())
print(enc.transform([["Jaipur"]]).toarray()) # a city it never saw
Output:
[array(['Delhi', 'Mumbai', 'Pune'], dtype=object)]
[[0. 0. 1.]]
ValueError: Found unknown categories ['Jaipur'] in column 0 during transform
In a live app, one new city takes the whole service down. The fix:
from sklearn.preprocessing import OneHotEncoder
train = [["Delhi"], ["Mumbai"], ["Pune"], ["Delhi"]]
enc = OneHotEncoder(handle_unknown="ignore", sparse_output=False).fit(train)
print(enc.get_feature_names_out())
print(enc.transform([["Pune"], ["Jaipur"]]))
Output:
['x0_Delhi' 'x0_Mumbai' 'x0_Pune']
[[0. 0. 1.]
[0. 0. 0.]]
With handle_unknown="ignore", an unknown category becomes all zeros: "none of the cities I know". sparse_output=False returns a normal NumPy array instead of a compressed sparse matrix (a format that stores only the non-zero values, useful when there are thousands of categories). Ignoring is safe, not perfect: collect data for the new category and retrain. (With drop="first", all zeros means the dropped baseline category, so an unknown city would be treated like Delhi, and scikit-learn warns you. Keep that in mind.)
Fit vs transform
A tailor measures you once (chest, length, sleeves) and then stitches many shirts with the same measurements.
fit(X)learns numbers from data: a scaler learns the mean and std (mean_,scale_), an encoder learns the list of categories (categories_). Learned attributes end with an underscore.transform(X)applies those learned numbers to any rows.fit_transform(X)=fit(X)thentransform(X)on the same data.
The tempting mistake: fit a new scaler on each set.
from sklearn.preprocessing import StandardScaler
train = [[650], [900], [1000], [1400]] # areas in the training set
test = [[1000], [2000]] # areas in the test set
print("train:", StandardScaler().fit_transform(train).ravel().round(2))
print("test: ", StandardScaler().fit_transform(test).ravel().round(2))
one_flat = [[1000]] # a single new flat
print("one flat:", StandardScaler().fit_transform(one_flat).ravel())
Output:
train: [-1.25 -0.32 0.05 1.53]
test: [-1. 1.]
one flat: [0.]
The 1,000 sq ft flat becomes 0.05 in training but −1.0 in the test set: same flat, different numbers. And a single new flat always becomes 0, because it is its own mean. The right way:
from sklearn.preprocessing import StandardScaler
train = [[650], [900], [1000], [1400]]
test = [[1000], [2000]]
sc = StandardScaler().fit(train) # fit: LEARN mean and std
print("learned mean", sc.mean_, " std", sc.scale_.round(1))
print("train:", sc.transform(train).ravel().round(2)) # transform: APPLY
print("test: ", sc.transform(test).ravel().round(2))
print("one flat:", sc.transform([[1000]]).ravel().round(2))
Output:
learned mean [987.5] std [270.1]
train: [-1.25 -0.32 0.05 1.53]
test: [0.05 3.75]
one flat: [0.05]
Now 1,000 sq ft is 0.05 everywhere, and the 2,000 sq ft test flat is honestly reported as 3.75 standard deviations above anything typical in training. Fitting only on the training set is also Episode 15's leakage rule: if the scaler learned its mean from all the data, the test set would have leaked into training.
The rules
fit(orfit_transform) on the training set only.transformthe training set, the validation and test sets, and live data with the same fitted object.- Number columns → scale (standard by default, min-max for fixed ranges, robust for outliers).
- Categories with no order → one-hot (drop one for plain linear models).
- Categories with a real order → ordinal, with
categories=given. - Anything that faces live data →
handle_unknown="ignore".
Common mistakes
| Mistake | Real symptom / error | Why | Fix |
|---|---|---|---|
| raw area + km in a distance model | the 9 km flat looks closest (13.1 vs 100.0) | big units dominate | scale features |
| a text column in a model | ValueError: could not convert string to float: 'Pune' |
models need numbers | encode it |
| cities as 0, 1, 2 | error 7,455 vs 730 | fake order | one-hot |
| full one-hot + bias in the normal equation | rank 3 of 4, inverse full of 4.5e+15 | dummy variable trap | drop="first" |
OrdinalEncoder() with defaults |
high = 0, low = 1, mid = 2 | alphabetical order | categories=[["low", "mid", "high"]] |
| a new category at prediction time | ValueError: Found unknown categories ['Jaipur'] in column 0 during transform |
not seen in fit |
handle_unknown="ignore" |
fit_transform on test or live data |
the same flat gets a different number; one flat → 0 | re-measured on the wrong data | fit on train, transform all |
| min-max on a column with outliers | normal values squashed into 0 to 0.09 | max is the outlier | RobustScaler |
pd.get_dummies separately on train and test |
different columns in train and test | each call sees different categories | OneHotEncoder fitted on train |
Mini project: prepare a mixed table by hand
Four columns (area, km, city, floor), the rules above applied by hand: split first, fit every scaler and encoder on train, transform both sets with the same objects, then a linear regression. (Episode 17 does the same in one object with a pipeline.)
import numpy as np, pandas as pd
from sklearn.preprocessing import StandardScaler, OneHotEncoder, OrdinalEncoder
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(16); n = 400
df = pd.DataFrame({"area": rng.normal(950, 250, n).round(), "km": rng.uniform(0, 10, n).round(1),
"city": rng.choice(["Delhi", "Mumbai", "Pune"], n), "floor": rng.choice(["low", "mid", "high"], n)})
rent = (df.area * 20 - df.km * 800 + df.city.map({"Delhi": 5000, "Mumbai": 20000, "Pune": 0})
+ df.floor.map({"low": 0, "mid": 1000, "high": 2000}) + rng.normal(0, 1000, n))
tr, te, y_tr, y_te = train_test_split(df, rent, test_size=0.25, random_state=0) # 1. split first
num = StandardScaler().fit(tr[["area", "km"]]) # 2. fit on train only
city = OneHotEncoder(drop="first", handle_unknown="ignore").fit(tr[["city"]])
floor = OrdinalEncoder(categories=[["low", "mid", "high"]]).fit(tr[["floor"]])
def prepare(d): # 3. transform any rows
return np.c_[num.transform(d[["area", "km"]]), city.transform(d[["city"]]).toarray(),
floor.transform(d[["floor"]])]
model = LinearRegression().fit(prepare(tr), y_tr)
print("test MAE:", round(np.abs(model.predict(prepare(te)) - y_te).mean()))
print("coefficients:", model.coef_.round(0))
Output:
test MAE: 802
coefficients: [ 5100. -2302. 14918. -5192. 1066.]
The test MAE is ₹802. The coefficients read: +5,100 per standard deviation of area (about 250 sq ft × ₹20), −2,302 per standard deviation of distance, Mumbai +14,918 and Pune −5,192 compared with Delhi (the hidden recipe used +15,000 and −5,000), and +1,066 per floor level.
Practice
- Scale
[5, 10, 15, 100]with all three scalers. Which one keeps 5, 10 and 15 well apart? - One-hot encode a
paymentcolumn (cash / card / UPI) by hand, then check it againstpd.get_dummies. - Encode T-shirt sizes S / M / L / XL in the right order.
- Find the bug:
scaler = StandardScaler().fit(X_test).
Solution 1
import numpy as np
from sklearn.preprocessing import MinMaxScaler, StandardScaler, RobustScaler
x = np.array([[5], [10], [15], [100]]) # 100 is an outlier
for s in [MinMaxScaler(), StandardScaler(), RobustScaler()]:
print(f"{type(s).__name__:15}", s.fit_transform(x).ravel().round(2))
Output:
MinMaxScaler [0. 0.05 0.11 1. ]
StandardScaler [-0.7 -0.57 -0.45 1.72]
RobustScaler [-0.27 -0.09 0.09 3.18]
Robust scaling keeps 5, 10 and 15 apart (−0.27, −0.09, 0.09); min-max squashes them into 0 to 0.11.
Solution 2
import pandas as pd
payment = ["cash", "card", "UPI", "UPI", "cash"]
cats = sorted(set(payment)) # ['UPI', 'card', 'cash']
by_hand = [[1 if p == c else 0 for c in cats] for p in payment]
print(cats)
for p, row in zip(payment, by_hand): print(f"{p:5}", row)
print(pd.get_dummies(pd.Series(payment), dtype=int).values.tolist() == by_hand)
Output:
['UPI', 'card', 'cash']
cash [0, 0, 1]
card [0, 1, 0]
UPI [1, 0, 0]
UPI [1, 0, 0]
cash [0, 0, 1]
True
Python sorts capital letters before lower-case ones, so UPI comes first.
Solution 3
from sklearn.preprocessing import OrdinalEncoder
sizes = [["M"], ["XL"], ["S"], ["L"], ["M"]]
print(OrdinalEncoder().fit_transform(sizes).ravel()) # alphabetical: wrong
enc = OrdinalEncoder(categories=[["S", "M", "L", "XL"]])
print(enc.fit_transform(sizes).ravel()) # 0 < 1 < 2 < 3
Output:
[1. 3. 2. 0. 1.]
[1. 3. 0. 2. 1.]
The default sorts L, M, S, XL alphabetically, which breaks the order (S = 2). Passing the order gives S = 0, M = 1, L = 2, XL = 3.
Solution 4
import numpy as np
from sklearn.preprocessing import StandardScaler
X_train = np.array([[650], [900], [1000], [1400]])
X_test = np.array([[1000], [2000]])
# Bug: scaler = StandardScaler().fit(X_test) -> learns the TEST mean (1500)
scaler = StandardScaler().fit(X_train) # fix: learn from training data only
print("mean learned:", scaler.mean_)
print(scaler.transform(X_test).ravel().round(2))
Output:
mean learned: [987.5]
[0.05 3.75]
Fitting on X_test learns the test mean (1,500) instead of the training mean; the test set has leaked into the preparation, and live single rows would all become 0.
Quick quiz
from sklearn.preprocessing import StandardScaler
train = [[10], [20], [30]]
sc = StandardScaler().fit(train)
print(sc.transform([[20], [40]]).ravel().round(2))
A) [0. 2.45] B) [-1. 1.] C) [0. 1. ] D) [0.5 1. ]
Answer
A) [0. 2.45]. The mean (20) and std (8.16) come from training. 20 → 0 and 40 → (40 − 20) / 8.16 = 2.45. B is what you'd get by wrongly fitting a new scaler on the two new values.
Interview questions
- Why do we scale features? Which models need it? Features in big units dominate distances and gradient steps. kNN, k-means, SVMs, PCA, regularised linear models and anything trained by gradient descent need it; tree-based models don't.
- Standardisation vs normalisation? Standardisation gives mean 0 and std 1 (z-scores); min-max normalisation squeezes values into [0, 1]. Standardisation is the usual default; min-max is sensitive to outliers.
- When would you use
RobustScaler? When a feature has outliers; it uses the median and IQR, which outliers barely move. - One-hot vs ordinal encoding? One-hot for categories without an order (no fake order, one column each); ordinal for categories with a real order (one column, numbers follow the order).
- What is the dummy variable trap? With a bias term, the full set of one-hot columns sums to the bias column, making the columns linearly dependent; drop one category for plain linear/logistic regression.
- How do you handle a category at prediction time that wasn't in training?
OneHotEncoder(handle_unknown="ignore")(all zeros), or group rare/new categories into "other"; monitor and retrain. - Why fit the scaler only on the training data? Fitting on all data leaks test information into training (optimistic scores), and fitting on test or live data gives the same value different encodings.
- What problem does one-hot cause with very many categories (e.g. 50,000 pin codes)? Huge sparse matrices and many weights to learn; options are grouping rare categories, target encoding (with care for leakage) or embeddings.
Glossary
| Term | Meaning | Example |
|---|---|---|
| Feature | One input column the model uses | area, km, city |
| Vector | An ordered list of numbers; one row of features | [900, 0.5] |
| Distance (Euclidean) | Straight-line length between two vectors | A to C = 100.0 |
| Feature scaling | Changing feature values to a similar range, keeping their order | area and km both 0 to 1 |
| Min-max normalisation | (x − min) / (max − min), giving 0 to 1 | 420/500 → 0.84 |
| Outlier | A value far from the rest | a 9,000 sq ft villa |
| Mean | The average | 972 sq ft |
| Standard deviation (std) | The typical distance of values from the mean | 243.6 sq ft |
| z-score | (x − mean) / std: how many stds from the mean | Raj = +2 |
| Standardisation | Turning a feature into z-scores (mean 0, std 1) | StandardScaler |
| Median | The middle value when sorted | ₹30,000 at the chai stall |
| Interquartile range (IQR) | Q3 − Q1, the width of the middle half | used by RobustScaler |
| Robust scaling | (x − median) / IQR | villa → 20.24, flats −0.77 to 1.12 |
| Categorical feature | A column of labels instead of numbers | city |
| Encoding | Turning categories into numbers | Delhi → 1 0 0 |
| One-hot encoding | One 0/1 column per category, exactly one 1 per row | Mumbai → 0 1 0 |
| Dummy variables | The 0/1 columns made by one-hot encoding | pd.get_dummies |
| Bias column | A column of 1s that lets a linear model learn its intercept | np.ones(n) |
| Dummy variable trap | One-hot columns + bias are linearly dependent | rank 3 of 4 |
| Rank | The number of truly independent columns | 3 |
| Baseline category | The dropped category; others are measured against it | Delhi |
| Ordinal feature | Categories with a real order | low < mid < high |
| Ordinal (label) encoding | Numbering ordered categories in order | S = 0, M = 1, L = 2 |
LabelEncoder |
Alphabetical numbering meant for the target y |
class names → 0, 1, 2 |
| Unknown category | A category not seen during fit |
Jaipur |
handle_unknown="ignore" |
Encode unknown categories as all zeros | Jaipur → 0 0 0 |
| Sparse matrix | Storage that keeps only the non-zero values | OneHotEncoder default output |
fit |
Learn numbers (mean, std, categories) from data | sc.fit(X_train) |
transform |
Apply the learned numbers to rows | sc.transform(X_test) |
fit_transform |
fit then transform on the same data |
training data only |
| Fit on train only | Learn every statistic from the training set | prevents leakage |
Summary
- Features in big units bully small ones in distances (13.1 vs 100.0): scale them.
- Min-max → 0 to 1 (outliers squash it); standardisation → mean 0, std 1 (the default); robust → median and IQR (for outliers).
- Models need numbers: encode categories. Codes 0/1/2 invent an order (7,455 vs 730); one-hot doesn't. Drop one column for plain linear models (the dummy variable trap).
- Ordinal encoding for real orders, with the order given (the default is alphabetical).
- Unknown categories crash
OneHotEncoderunlesshandle_unknown="ignore". fitlearns,transformapplies: fit on the training set only, transform everything with the same object.
Next episode: the scikit-learn API and pipelines: estimators, fit, predict, predict_proba, transformers, Pipeline, ColumnTransformer, baseline models and leakage-safe preprocessing.