SyntaxPilot
Courses/Machine Learning/Lesson 17

Machine Learning · Lesson 17 of 90 · 13 min

The scikit-learn API and pipelines

estimatorfitpredictpredict_probatransformfit_transformtransformerPipelineColumnTransformermake_pipelinebaseline modelDummyClassifier and DummyRegressorleakage-safe preprocessing

Machine Learning Full Course · Episode 17. Every code sample below was run with Python 3.13, NumPy 2.5.3, pandas 3.0.5 and scikit-learn 1.9.1; the output shown is the real output. All data is made up for teaching.

The big idea: every scikit-learn object speaks the same small language (fit, predict, transform). Because of that, you can glue your preparation steps and your model into one object, a pipeline, so no step is ever forgotten, nothing leaks, and you can always compare against a baseline that doesn't even look at the data.

The problem: the helper who forgot the sugar

A chai stall owner makes chai in five steps. A new helper, working from memory on a busy evening, forgets the sugar; every cup comes back. When steps live only in someone's head, one gets skipped.

Below, 400 made-up flats have two features (area, km to the metro) and a label: 1 if the flat got rented within a week. k-nearest neighbours (kNN) predicts by looking at the most similar flats it has seen (it gets its own episode, 27). We scale the training data (Episode 16), then pass the test data... unscaled:

import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(17)
area = rng.normal(950, 300, 400); km = rng.uniform(0, 10, 400)
X = np.c_[area, km]
y = (km + rng.normal(0, 1.5, 400) < 4).astype(int)   # 1 = rented in a week
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, random_state=0)
sc = StandardScaler().fit(X_tr)
knn = KNeighborsClassifier().fit(sc.transform(X_tr), y_tr)
print("test, scaled:      ", knn.score(sc.transform(X_te), y_te))
print("test, forgot scale:", knn.score(X_te, y_te))

Output:

test, scaled:       0.85
test, forgot scale: 0.38

No error, no warning: 0.85 drops to 0.38, worse than a coin flip. The model received numbers in different units from the ones it learned on. The scaling step lived in our head, not in the model.

The estimator API

Every appliance in your home has the same kind of plug. In scikit-learn:

  • An estimator is any object that learns from data. Every estimator has fit.
  • A predictor is an estimator that gives answers: predict, often predict_proba, and score. Models such as KNeighborsClassifier, LinearRegression, LogisticRegression.
  • A transformer is an estimator that changes data: transform and fit_transform. StandardScaler, OneHotEncoder, SimpleImputer, SelectKBest.
Method What it does Returns
Model(setting=...) the constructor: sets hyperparameters (Episode 8), learns nothing the object
fit(X, y) learns from data; learned values end in _ (mean_, coef_, categories_) the object itself (so calls can be chained)
predict(X) answers for new rows labels or numbers
predict_proba(X) probability of each class, per row (classifiers) an n × classes array
score(X, y) a quick grade: accuracy (classifiers) or R² (regressors, Episode 19) a number
transform(X) applies what was learned changed data
fit_transform(X) fit + transform on the same data changed data

X is the feature table (rows × columns), y the labels. Because every model uses the same names, swapping one model for another is a one-line change.

Predict before fit

import sys; sys.tracebacklimit = 0   # show only the error line
from sklearn.neighbors import KNeighborsClassifier
X = [[1], [2], [3], [10]]
y = [0, 0, 1, 1]
knn = KNeighborsClassifier(n_neighbors=3)    # settings go in the constructor
print(knn.predict([[2.5]]))                  # predict before fit?

Output:

sklearn.exceptions.NotFittedError: This KNeighborsClassifier instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator.

predict, predict_proba, transform, and a transformer that can't predict

import sys; sys.tracebacklimit = 0   # show only the error line
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
X = [[1], [2], [3], [10]]
y = [0, 0, 1, 1]
knn = KNeighborsClassifier(n_neighbors=3).fit(X, y)   # a predictor
print("predict:", knn.predict([[2.5]]), " proba:", knn.predict_proba([[2.5]]).round(2))
sc = StandardScaler().fit(X)                          # a transformer
print("transform:", sc.transform([[2.5]]).round(2))
print(sc.predict([[2.5]]))

Output:

predict: [0]  proba: [[0.67 0.33]]
transform: [[-0.42]]
AttributeError: 'StandardScaler' object has no attribute 'predict'

For 2.5, the three nearest rows are 1, 2 and 3 with labels 0, 0, 1: two of three say 0, so predict_proba gives 0.67 for class 0 and 0.33 for class 1, and predict picks the more likely class, 0. The scaler transforms 2.5 into −0.42, and asking it to predict fails: transformers change data, they don't answer questions.

Pipeline

The chai stall's fix is a recipe card on the wall, and a machine with one button. A Pipeline is a list of transformers followed by one final estimator. make_pipeline builds one and names the steps for you:

import numpy as np
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(17)
area = rng.normal(950, 300, 400); km = rng.uniform(0, 10, 400)
X = np.c_[area, km]
y = (km + rng.normal(0, 1.5, 400) < 4).astype(int)   # 1 = rented in a week
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, random_state=0)
pipe = make_pipeline(StandardScaler(), KNeighborsClassifier())
pipe.fit(X_tr, y_tr)                       # scaler fit on train, then kNN
print(pipe)
print("test accuracy:", pipe.score(X_te, y_te))
new = [[900, 1], [900, 4], [900, 8]]       # raw numbers, no manual scaling
print("predict:      ", pipe.predict(new))
print("predict_proba:", pipe.predict_proba(new).round(2).tolist())

Output:

Pipeline(steps=[('standardscaler', StandardScaler()),
                ('kneighborsclassifier', KNeighborsClassifier())])
test accuracy: 0.85
predict:       [1 1 0]
predict_proba: [[0.0, 1.0], [0.4, 0.6], [1.0, 0.0]]

What the pipeline does for you:

You call The pipeline does
pipe.fit(X_train, y_train) fit_transform on each transformer in turn, passing the result on, then fit on the final model
pipe.predict(X) / pipe.predict_proba(X) / pipe.score(X, y) only transform on each transformer (never fit), then the model's method

So the scaler learns from the training data only and is applied to test and live rows automatically: Episode 16's rule, enforced by the object. A pipeline is itself an estimator, so anything that accepts a model (cross-validation, grid search) accepts a pipeline.

At 4 km, predict_proba is [0.4, 0.6]: of the 5 nearest flats (kNN's default), 3 were rented. The model is unsure and says so, which is often more useful than a plain yes or no (Episode 25 is about using these probabilities well).

Pipeline vs make_pipeline: Pipeline([("scale", StandardScaler()), ("knn", KNeighborsClassifier())]) lets you choose the step names; make_pipeline uses the class names in lower case. You reach a step with pipe.named_steps["scale"] and change a setting with pipe.set_params(knn__n_neighbors=15) (step name, two underscores, setting).

ColumnTransformer

A thali: rice is steamed, dal is boiled and tempered, and everything is served side by side on one plate. A ColumnTransformer sends each group of columns to its own transformer and joins the results into one table:

import numpy as np, pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder, OrdinalEncoder
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LinearRegression
rng = np.random.default_rng(17); n = 300
df = pd.DataFrame({"area": rng.normal(950, 250, n).round(),
                   "km": rng.uniform(0, 10, n).round(1),
                   "city": rng.choice(["Delhi", "Mumbai", "Pune"], n),
                   "floor": rng.choice(["low", "mid", "high"], n)})
extra = df.city.map({"Delhi": 5000, "Mumbai": 20000, "Pune": 0})
rent = df.area * 20 - df.km * 800 + extra + rng.normal(0, 1000, n)
prep = ColumnTransformer([
    ("num", StandardScaler(), ["area", "km"]),
    ("city", OneHotEncoder(drop="first"), ["city"]),
    ("floor", OrdinalEncoder(categories=[["low", "mid", "high"]]), ["floor"])])
model = make_pipeline(prep, LinearRegression()).fit(df, rent)
print(prep.get_feature_names_out())
new = pd.DataFrame({"area": [900, 900], "km": [1.0, 1.0],
                    "city": ["Mumbai", "Pune"], "floor": ["mid", "mid"]})
print("predicted rent:", model.predict(new).round(-2))

Output:

['num__area' 'num__km' 'city__city_Mumbai' 'city__city_Pune'
 'floor__floor']
predicted rent: [37100. 17200.]
  • Each part is (name, transformer, columns).
  • get_feature_names_out() lists the columns the model actually sees, prefixed with the part's name: scaled area and km, Mumbai and Pune (Delhi is the dropped baseline), and the ordinal floor.
  • The final pipeline takes a raw pandas DataFrame, text and all, and returns rents: ₹37,100 in Mumbai and ₹17,200 in Pune for the same 900 sq ft, mid-floor flat 1 km from the metro.
  • Columns you don't list are dropped by default (remainder="drop"); use remainder="passthrough" to keep them unchanged.

Baseline models

In a desert town it rains on about 1 day in 10. A forecaster who always says "no rain" is right 90% of the time, and useless. A score means nothing until you compare it with a lazy guess.

import numpy as np
from sklearn.dummy import DummyClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(17); n = 1000
X = rng.normal(0, 1, (n, 3))                        # 3 made-up loan features
y = (X[:, 0] + rng.normal(0, 1.5, n) > 2.3).astype(int)    # 1 = default
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
print("defaults in test:", y_te.mean().round(3))
model = LogisticRegression().fit(X_tr, y_tr)
print("our model accuracy:", round(model.score(X_te, y_te), 3))
base = DummyClassifier(strategy="most_frequent").fit(X_tr, y_tr)
print("baseline accuracy: ", round(base.score(X_te, y_te), 3))
print("baseline predicts: ", np.unique(base.predict(X_te)))

Output:

defaults in test: 0.093
our model accuracy: 0.903
baseline accuracy:  0.907
baseline predicts:  [0]

Logistic regression, a classifier we build from scratch in Episode 22, scores 90.3%. The DummyClassifier with strategy="most_frequent" ignores the features and always predicts the most common class: 90.7%. The lazy guess wins. A baseline model is the simplest possible answer, which a real model must beat to be worth anything. (Accuracy itself is a weak score with rare classes; Episode 24 fixes that with precision and recall.)

DummyClassifier strategies: most_frequent (always the commonest class), stratified (random guesses in the class proportions), uniform (random guesses, every class equally likely), prior, constant.

For regression, DummyRegressor predicts the training mean (or median, quantile, constant):

import numpy as np
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(17)
area = rng.normal(950, 250, (300, 1))
rent = area[:, 0] * 20 + rng.normal(0, 3000, 300)
X_tr, X_te, y_tr, y_te = train_test_split(area, rent, test_size=0.25, random_state=0)
for m in [DummyRegressor(strategy="mean"), LinearRegression()]:
    err = np.abs(m.fit(X_tr, y_tr).predict(X_te) - y_te).mean()
    print(f"{type(m).__name__:16} test error: {err:.0f}")

Output:

DummyRegressor   test error: 4109
LinearRegression test error: 2372

Linear regression cuts the error from ₹4,109 to ₹2,372: about 42% less than guessing the average. Now that's a result you can report.

Leakage-safe preprocessing

A student guesses randomly on 5,000 practice questions; a teacher keeps the 20 he happened to get right, choosing with all the answers in hand, then "tests" him on those 20. He looks like a genius.

  • SelectKBest(k=20) is a transformer that keeps the 20 columns most related to the label (by a statistical test).
  • cross_val_score splits the data into 5 parts (folds), trains on 4 and tests on the 5th, five times, and returns the 5 test scores (full story in Episode 26).
import numpy as np
from sklearn.feature_selection import SelectKBest
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
rng = np.random.default_rng(17)
X = rng.normal(0, 1, (100, 5000))     # 100 rows, 5,000 random columns
y = rng.integers(0, 2, 100)           # random labels: nothing to learn
X_best = SelectKBest(k=20).fit_transform(X, y)      # picked using ALL rows
print("select first, then CV:", cross_val_score(LogisticRegression(), X_best, y).mean())
pipe = make_pipeline(SelectKBest(k=20), LogisticRegression())
print("pipeline inside CV:   ", cross_val_score(pipe, X, y).mean())

Output:

select first, then CV: 0.85
pipeline inside CV:    0.54

The columns and labels are pure noise, yet selecting columns on all 100 rows before cross-validation gives 85%. Those columns were chosen because they happened to match the labels of every row, including the rows later used for testing. Inside a pipeline, the selector is re-fit on each training part only and the honest answer appears: 54%, a coin flip. That's leakage-safe preprocessing: every step that learns from data (scaling, encoding, imputing, selecting) goes inside the pipeline, so it is fit on training rows only, in every split.

Pipeline rules

  1. Every step that learns numbers from data goes inside the pipeline.
  2. Pass raw X_train to pipe.fit and raw X_test to pipe.predict.
  3. Mixed column types → a ColumnTransformer as the first step.
  4. Start every project with a dummy baseline, and report how much you beat it by.
  5. Cross-validate the whole pipeline, not the model on pre-processed data.

Common mistakes

Mistake Real symptom / error Why Fix
scale train by hand, forget test accuracy 0.85 → 0.38, no error a step done by hand Pipeline
predict before fit NotFittedError: This KNeighborsClassifier instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator. nothing learned call fit first
ask a transformer to predict AttributeError: 'StandardScaler' object has no attribute 'predict' transformers transform transform, or put a model last
no baseline model 90.3% < dummy 90.7% accuracy fools you DummyClassifier / DummyRegressor
select / scale / impute before CV 85% accuracy on pure noise test rows helped choose every learning step inside the pipeline
fit_transform on the test set different numbers for the same rows re-fitted on test pipe.predict (transform only)
a column missing from the ColumnTransformer the column silently disappears remainder="drop" by default list it or use remainder="passthrough"

Mini project: a complete, leak-free regression workflow

Missing values, mixed columns, a baseline, cross-validation of the whole pipeline, then the test set once:

import numpy as np, pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder, OrdinalEncoder
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LinearRegression
from sklearn.dummy import DummyRegressor
from sklearn.model_selection import train_test_split, cross_val_score
rng = np.random.default_rng(7); n = 500
df = pd.DataFrame({"area": rng.normal(950, 250, n).round(), "km": rng.uniform(0, 10, n).round(1),
                   "city": rng.choice(["Delhi", "Mumbai", "Pune"], n), "floor": rng.choice(["low", "mid", "high"], n)})
rent = (df.area * 20 - df.km * 800 + df.city.map({"Delhi": 5000, "Mumbai": 20000, "Pune": 0})
        + rng.normal(0, 1500, n))
df.loc[rng.random(n) < 0.1, "area"] = np.nan                     # 10% missing areas
X_tr, X_te, y_tr, y_te = train_test_split(df, rent, test_size=0.2, random_state=0)
prep = ColumnTransformer([
    ("num", make_pipeline(SimpleImputer(strategy="median"), StandardScaler()), ["area", "km"]),
    ("city", OneHotEncoder(handle_unknown="ignore"), ["city"]),
    ("floor", OrdinalEncoder(categories=[["low", "mid", "high"]]), ["floor"])])
model = make_pipeline(prep, LinearRegression())
mae = "neg_mean_absolute_error"                                  # scikit-learn maximises scores
print("baseline CV MAE:", round(-cross_val_score(DummyRegressor(), X_tr, y_tr, scoring=mae).mean()))
print("pipeline CV MAE:", round(-cross_val_score(model, X_tr, y_tr, scoring=mae).mean()))
model.fit(X_tr, y_tr)
print("test MAE (once):", round(np.abs(model.predict(X_te) - y_te).mean()))
new = pd.DataFrame({"area": [np.nan], "km": [2.0], "city": ["Jaipur"], "floor": ["high"]})
print("a Jaipur flat with no area:", model.predict(new).round(-2))

Output:

baseline CV MAE: 8560
pipeline CV MAE: 1396
test MAE (once): 1296
a Jaipur flat with no area: [25200.]
  • SimpleImputer(strategy="median") (Episode 14) is chained with the scaler inside the numeric part, so the median comes from training folds only.
  • scikit-learn scorers are "higher is better", so MAE is reported as neg_mean_absolute_error; we flip the sign.
  • The pipeline (CV MAE ₹1,396) is far better than the baseline (₹8,560); the test MAE (₹1,296, looked at once) agrees.
  • A flat in an unknown city with no area still gets a prediction: the imputer fills the area, and handle_unknown="ignore" treats Jaipur as "none of the known cities".

Practice

  1. Rebuild the kNN pipeline with Pipeline and your own step names; read the scaler's learned mean and change the number of neighbours.
  2. Add a SimpleImputer to the numeric part of a ColumnTransformer.
  3. Compare DummyClassifier strategies most_frequent, stratified and uniform on the loan data.
  4. Re-run the noise demo with k=200.
Solution 1
import numpy as np
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.neighbors import KNeighborsClassifier
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(17)
area = rng.normal(950, 300, 400); km = rng.uniform(0, 10, 400)
X = np.c_[area, km]
y = (km + rng.normal(0, 1.5, 400) < 4).astype(int)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.25, random_state=0)
pipe = Pipeline([("scale", StandardScaler()), ("knn", KNeighborsClassifier(n_neighbors=5))])
pipe.fit(X_tr, y_tr)
print("test accuracy:", pipe.score(X_te, y_te))
print("scaler learned mean:", pipe.named_steps["scale"].mean_.round(1))
pipe.set_params(knn__n_neighbors=15)          # step name + __ + setting
print("with 15 neighbours:", pipe.fit(X_tr, y_tr).score(X_te, y_te))

Output:

test accuracy: 0.85
scaler learned mean: [932.2   5.1]
with 15 neighbours: 0.81
Solution 2
import numpy as np, pandas as pd
from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LinearRegression
df = pd.DataFrame({"area": [900, np.nan, 700, 1000, 850, 1100],
                   "city": ["Pune", "Delhi", "Mumbai", "Pune", "Delhi", "Mumbai"]})
rent = [18000, 29000, 34000, 20000, 22000, 42000]
prep = ColumnTransformer([
    ("num", make_pipeline(SimpleImputer(strategy="median"), StandardScaler()), ["area"]),
    ("city", OneHotEncoder(handle_unknown="ignore"), ["city"])])
model = make_pipeline(prep, LinearRegression()).fit(df, rent)
print("median learned:", prep.named_transformers_["num"][0].statistics_)
print(model.predict(pd.DataFrame({"area": [np.nan], "city": ["Pune"]})).round(-2))

Output:

median learned: [900.]
[17900.]

The imputer's median (900) is learned inside the pipeline and reused for the new flat with a missing area.

Solution 3
import numpy as np
from sklearn.dummy import DummyClassifier
from sklearn.model_selection import train_test_split
rng = np.random.default_rng(17); n = 1000
X = rng.normal(0, 1, (n, 3))
y = (X[:, 0] + rng.normal(0, 1.5, n) > 2.3).astype(int)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, random_state=0, stratify=y)
for s in ["most_frequent", "stratified", "uniform"]:
    d = DummyClassifier(strategy=s, random_state=0).fit(X_tr, y_tr)
    print(f"{s:14} accuracy {d.score(X_te, y_te):.3f}  defaults caught {d.predict(X_te)[y_te == 1].sum()}")

Output:

most_frequent  accuracy 0.907  defaults caught 0
stratified     accuracy 0.817  defaults caught 2
uniform        accuracy 0.487  defaults caught 13

The "most accurate" dummy catches zero defaults. That's why accuracy alone is not enough for rare classes (Episode 24).

Solution 4
import numpy as np
from sklearn.feature_selection import SelectKBest
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
rng = np.random.default_rng(17)
X = rng.normal(0, 1, (100, 5000)); y = rng.integers(0, 2, 100)
for k in [20, 200]:
    leaky = cross_val_score(LogisticRegression(), SelectKBest(k=k).fit_transform(X, y), y).mean()
    honest = cross_val_score(make_pipeline(SelectKBest(k=k), LogisticRegression()), X, y).mean()
    print(f"k={k:3}  select first {leaky:.2f}   inside pipeline {honest:.2f}")

Output:

k= 20  select first 0.85   inside pipeline 0.54
k=200  select first 1.00   inside pipeline 0.54

Selecting 200 of 5,000 noise columns on all rows makes the leaky score a perfect 1.00. The pipeline stays at a coin flip.

Quick quiz

from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression

pipe = make_pipeline(StandardScaler(), LinearRegression())
print(list(pipe.named_steps))

A) ['standardscaler', 'linearregression'] B) ['StandardScaler', 'LinearRegression'] C) ['step0', 'step1'] D) an error: steps need names

Answer

A. make_pipeline names each step after its class, in lower case. Use Pipeline([...]) to choose your own names.

Interview questions

  1. What's the difference between an estimator, a predictor and a transformer? An estimator learns with fit; a predictor also has predict (and often predict_proba, score); a transformer has transform/fit_transform and changes the data.
  2. Why use a Pipeline? One object for all steps: no forgotten steps at prediction time, preprocessing fit only on training data (also inside cross-validation), easy to tune, save and deploy.
  3. What does pipe.fit call on each step? And pipe.predict? fit_transform on every transformer then fit on the last step; predict calls only transform on transformers, then predict.
  4. When do you need a ColumnTransformer? When different columns need different preparation (scale numbers, one-hot categories, ordinal-encode ordered ones, impute some).
  5. What is a baseline model and why use one? The simplest possible predictor (most frequent class, mean value). A model is only useful if it clearly beats it; with imbalanced data, a dummy often has high accuracy.
  6. Why is feature selection before cross-validation a leak? The selection used the labels of the rows later used as test folds, so the CV score is optimistic (85% on pure noise). Put selection inside the pipeline.
  7. What does predict_proba return for kNN? The fraction of the k nearest neighbours in each class.
  8. How do you change a hyperparameter of a step inside a pipeline? pipe.set_params(stepname__param=value), the same syntax grid search uses.

Glossary

Term Meaning Example
Estimator Any scikit-learn object that learns from data with fit StandardScaler(), LinearRegression()
Constructor Creating the object; sets hyperparameters, learns nothing KNeighborsClassifier(n_neighbors=3)
fit Learn from data knn.fit(X, y)
Learned attribute A value learned by fit, name ends in _ mean_, coef_
Predictor An estimator that gives answers knn.predict(...)
predict Answers for new rows [1 1 0]
predict_proba Probability of each class per row [0.4, 0.6]
score Quick grade: accuracy or R² 0.85
Transformer An estimator that changes data StandardScaler
transform Apply what was learned sc.transform(X_test)
fit_transform fit + transform on the same data training data
NotFittedError Error for using an estimator before fit predict before fit
k-nearest neighbours (kNN) Predicts from the k most similar training rows 5 nearest flats
Pipeline Transformers followed by a final estimator, as one object make_pipeline(StandardScaler(), KNN())
make_pipeline Builds a pipeline with automatic step names standardscaler
named_steps Access a pipeline step by name pipe.named_steps["scale"]
set_params Change a step's setting knn__n_neighbors=15
ColumnTransformer Different transformers for different columns, joined num / city / floor
get_feature_names_out Names of the columns produced city__city_Mumbai
remainder What happens to unlisted columns "drop" (default)
Baseline model The simplest predictor to beat always "no default"
DummyClassifier Baseline classifier that ignores the features strategy="most_frequent"
DummyRegressor Baseline regressor that ignores the features predicts the mean
Logistic regression A linear classifier that outputs probabilities Episode 22
SelectKBest Keeps the k columns most related to the label k=20
Cross-validation Train/test on several splits (folds) and average 5 folds
cross_val_score Runs cross-validation, returns one score per fold .mean()
Fold One of the parts in cross-validation 1 of 5
Leakage-safe preprocessing Every learning step inside the pipeline 0.54, not 0.85
SimpleImputer Fills missing values with a learned statistic median 900

Summary

  • scikit-learn's shared API: fit for every estimator, predict / predict_proba / score for predictors, transform / fit_transform for transformers.
  • Steps done by hand get skipped (0.85 → 0.38). A Pipeline runs every step, fitting on training data and only transforming new data.
  • A ColumnTransformer prepares each group of columns its own way; the pipeline then takes raw tables.
  • Always compare with a baseline: our "90.3%" model lost to a dummy's 90.7%.
  • Leakage-safe preprocessing: every learning step inside the pipeline (pure noise scored 85% outside, 54% inside).

Next episode: linear regression from scratch: the linear model, weights and bias, residuals, MSE, the normal equation, and batch, stochastic and mini-batch gradient descent in NumPy.