SyntaxPilot
Courses/Machine Learning/Lesson 1
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

Machine Learning · Lesson 1 of 90

What machine learning is and the ML workflow

machine learning vs rule-based programmingmodeltraininginference (prediction)datasetexample (sample, row)featurelabel (target)supervised learningunsupervised learningsemi-supervised learningself-supervised learningreinforcement learningregression vs classification

Machine Learning Full Course · Episode 1. You need only basic Python (variables, lists, loops, functions, dictionaries). Every code sample below was run with Python 3.13 and scikit-learn 1.9.1, and the output shown is the real output.

The problem: spam messages on your phone

A message arrives: "Congratulations, you won Rs 50 lakh, click this link." That is spam: an unwanted message, usually a trick to steal money or data. People in the industry jokingly call a real, wanted message ham, the opposite of spam.

Our job: write a program that reads a message and says spam or ham, without blocking real messages. Spammers send huge numbers of these and keep changing their tricks, so whatever we build has to keep up.

The obvious fix: hand-written rules (and why they fail)

Any programmer's first idea is a list of suspicious words:

def is_spam(msg):
    words = ["lottery", "won", "free", "prize"]
    for w in words:
        if w in msg.lower():
            return True
    return False

tests = ["You won a FREE prize! Click now",
         "Y0u w0n Rs 50 lakh, claim fast",
         "Is the lottery ticket for the school fair Rs 20?",
         "Meeting at 5, bring the notes"]
for t in tests:
    print(is_spam(t), "|", t)

Output:

True | You won a FREE prize! Click now
False | Y0u w0n Rs 50 lakh, claim fast
True | Is the lottery ticket for the school fair Rs 20?
False | Meeting at 5, bring the notes

Two failures:

  • A miss: "Y0u w0n" uses the digit zero instead of the letter o, so the rule "won" in msg doesn't match. Spam got through.
  • A false alarm: a parent asking about a school-fair lottery ticket is blocked, just because it contains the word "lottery".

Why rules break:

  1. Spammers change spelling every day (w0n, fr33, l0ttery). You'd add a new rule every morning.
  2. One word alone isn't proof. Whether "lottery" is spammy depends on the other words around it.
  3. Every fix is one more rule that a human must think of and type, forever.

The new idea: don't write the rules. Show the computer many messages that people have already marked spam or ham, and let it find the patterns itself.

Machine learning vs rule-based programming

Rule-based programming Machine learning
You provide rules + data data + correct answers
You get answers rules (a model)
When the world changes a human rewrites the rules retrain on new examples

Machine learning (ML) is a way of building programs that get better at a task by learning from examples, instead of being told every rule.

Tom Mitchell's textbook definition (Machine Learning, 1997): a computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E. For our spam filter: T = label messages as spam/ham, P = the fraction labelled correctly, E = the labelled example messages.

History. Arthur Samuel, a researcher at IBM, coined the term machine learning in 1959, in his paper "Some studies in machine learning using the game of checkers" (IBM Journal of Research and Development). His checkers program ran on the IBM 701, IBM's first commercial computer, and improved partly by playing thousands of games against itself.

The vocabulary: dataset, example, feature, label

Real-life example: a student preparing for an exam with a pile of past papers and the answer key.

Exam preparation Machine learning In our spam filter
The pile of past papers Dataset: a collection of examples 6 labelled messages
One solved question Example (also sample, row): one item of the dataset ("w0n rs 50 lakh claim fast", "spam")
The clues inside a question Features: the inputs the model looks at the words of the message
The answer key Label (also target): the correct answer to predict spam or ham

Datasets are usually tables: one example per row, features in columns, and the label in its own column.

 message (features)                | label
-----------------------------------+------
 you won free prize claim now      | spam
 w0n rs 50 lakh claim fast         | spam
 meeting at 5 bring notes          | ham
 school fair lottery ticket rs 20  | ham

Training and inference: a spam filter that learns, in plain Python

Training = the program looks at labelled examples and builds its knowledge. Here, training is just counting how often each word appears in spam and in ham messages.

Inference (also called prediction) = using what was learned to answer for a new example that has no label.

train = [("you won free prize claim now", "spam"), ("w0n rs 50 lakh claim fast", "spam"),
         ("free recharge claim offer now", "spam"), ("meeting at 5 bring notes", "ham"),
         ("school fair lottery ticket rs 20", "ham"), ("mom says call me now", "ham")]

# training: count each word in spam and in ham
counts = {"spam": {}, "ham": {}}
for text, label in train:
    for w in text.split():
        counts[label][w] = counts[label].get(w, 0) + 1

# inference: every word of a new message votes
def predict(text):
    score = 0
    for w in text.split():
        score += counts["spam"].get(w, 0) - counts["ham"].get(w, 0)
    return "spam" if score > 0 else "ham"

print(predict("y0u w0n a prize claim fast"))
print(predict("is the school fair lottery on sunday"))
print(predict("bring the notes to the meeting"))

Output:

spam
ham
ham

No rule mentions the digit zero, yet "y0u w0n a prize claim fast" is caught: the words w0n, prize, claim and fast were seen in spam. The school-fair message is ham because school, fair and ticket were seen in a real message. The words around "lottery" changed the answer, which the hand-written rule could never do.

Model = the learned pattern, stored so it can be used later. Here, the model is the counts dictionary. In real ML, a model is usually a set of numbers (called parameters) learned by a mathematical procedure.

How the vote works for "y0u w0n a prize claim fast":

word spam count ham count vote
y0u 0 0 0
w0n 1 0 +1
a 0 0 0
prize 1 0 +1
claim 3 0 +3
fast 1 0 +1
total +6 → spam

Evaluation: is the model any good?

Three lucky answers prove nothing. Evaluation means measuring a model on examples it never saw during training, whose correct labels we already know. Accuracy = correct predictions ÷ total predictions.

# (after the training and predict code above)
test = [("claim your free prize", "spam"), ("bring notes to school", "ham"), ("rs 50 lakh won", "spam"),
        ("urgent kyc update needed today", "spam"), ("call me after the meeting", "ham")]
correct = 0
for text, label in test:
    print(predict(text), "| real:", label, "|", text)
    correct += (predict(text) == label)
print("accuracy:", correct, "/", len(test))

Output (with the training code above):

spam | real: spam | claim your free prize
ham | real: ham | bring notes to school
spam | real: spam | rs 50 lakh won
ham | real: spam | urgent kyc update needed today
ham | real: ham | call me after the meeting
accuracy: 4 / 5

The miss is a classic bank-KYC scam. None of its words were in the six training messages, so every vote was 0 and the model fell back to "ham". A model only knows what its examples taught it. The fix is not a new rule: it's more, and more varied, data.

Never judge a model on its own training examples: a perfect score there may just be memory, not learning.

Classification vs regression

Task Predicts Examples
Classification a category from a fixed list spam/ham, cat/dog, loan repaid yes/no
Regression a number cups of chai tomorrow, house price, delivery time in minutes

Both are supervised (see below): every training example has a label.

Your first scikit-learn model

scikit-learn (imported as sklearn) is the standard Python library for classic machine learning. Raju's chai stall: predict cups sold (label) from the day's temperature (feature).

from sklearn.linear_model import LinearRegression

X = [[12], [16], [20], [24], [28], [32]]   # temperature
y = [182, 165, 141, 120, 104, 79]          # cups sold

model = LinearRegression()
model.fit(X, y)                            # training

for t in [18, 30]:
    cups = model.predict([[t]])[0]           # inference
    print(t, "C ->", round(cups), "cups")

Output:

18 C -> 152 cups
30 C -> 91 cups
  • X (capital) holds the features: a list of lists, one inner list per example (row), one number per feature (column).
  • y (small) holds the labels, one per example.
  • fit = training. predict = inference. It answers for temperatures it never saw (18 and 30).
  • LinearRegression fits a straight line through the data; episode 18 builds it from scratch.

Kinds of learning

Kind What the data gives you Real-life example
Supervised learning every example has a label spam filter trained on tagged messages; chai sales from temperature
Unsupervised learning no labels; find groups or structure a supermarket grouping customers by what they buy (clustering)
Semi-supervised learning a few labels + lots of unlabelled data you name your mother in 3 photos and the gallery app finds her in 3,000
Self-supervised learning labels created from the data itself keyboard next-word suggestion: hide a word in a real sentence, guess it
Reinforcement learning rewards for actions, by trial and error training a puppy with treats; a program learning a game from wins and losses

"Supervised" means the labels act like a teacher checking answers. Classification and regression are both supervised.

The ML workflow

1. Frame the problem -> 2. Data -> 3. Model -> 4. Evaluate -> 5. Deploy
        ^                                          |
        +-------- not good enough? go back --------+
  1. Problem framing: decide exactly what to predict, whether it's classification or regression, and how success is measured ("block spam, but never block a real bank message").
  2. Data: collect examples, clean out mistakes, make sure the labels are correct. In real projects this takes most of the time.
  3. Model: choose a type of model and train it (fit).
  4. Evaluation: test on unseen examples. Not good enough? Go back to step 2 or 3.
  5. Deployment: put the model where real users get its predictions (inside an app or website), then keep monitoring it, because the world changes (spammers invent new tricks) and the model must be retrained.

Common mistakes (and the real error messages)

Mistake What happens Why Fix
if "won" in msg: spam "Y0u w0n Rs 50 lakh" → False (missed) hand-written rules can't keep up learn the pattern from labelled examples
Testing on the training examples looks perfect that's memory, not learning evaluate on unseen examples
model.predict(...) before fit sklearn.exceptions.NotFittedError: This LinearRegression instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator. the model hasn't learned anything call model.fit(X, y) first
X = [12, 16, 20, 24, 28, 32] ValueError: Expected 2D array, got 1D array instead: … Reshape your data either using array.reshape(-1, 1) if your data has a single feature or array.reshape(1, -1) if it contains a single sample. X must be a table: rows × features X = [[12], [16], [20], ...]
Too little data a new scam (urgent kyc update needed today) → ham the model never saw those words add more, varied examples

Mini project: chai sales with two features

Add a second feature, whether it's a weekend (1) or a weekday (0):

from sklearn.linear_model import LinearRegression

# features: [temperature (C), is_weekend (1 = yes, 0 = no)]   label: cups sold
X = [[12, 0], [12, 1], [18, 0], [18, 1], [24, 0], [24, 1], [30, 0], [30, 1]]
y = [180, 230, 150, 205, 121, 172, 90, 140]

model = LinearRegression()
model.fit(X, y)                      # training

tests = [[20, 0], [20, 1]]
for t, cups in zip(tests, model.predict(tests)):   # inference
    print(t, "->", round(cups), "cups")

Output:

[20, 0] -> 140 cups
[20, 1] -> 192 cups

Same temperature, about 52 more cups on a weekend. Each row of X now has two numbers because each example has two features.

Practice

  1. For each, say supervised or unsupervised, and classification or regression where it applies: (a) predict a flat's rent from its size and area, (b) group songs into playlists with no tags, (c) decide whether a photo contains a dog.
Solution

(a) supervised regression (rent is a number). (b) unsupervised (clustering, no labels). (c) supervised classification (dog / no dog).

  1. In the spam dataset, what are the features and what is the label? How many examples are there?
Solution

Features: the words of each message. Label: spam or ham. Examples: 6 (one per row).

  1. Add two training messages so that the KYC scam gets caught, without breaking real messages.
Solution
train = [("you won free prize claim now", "spam"),
         ("w0n rs 50 lakh claim fast", "spam"),
         ("free recharge claim offer now", "spam"),
         ("urgent kyc update your account", "spam"),
         ("meeting at 5 bring notes", "ham"),
         ("school fair lottery ticket rs 20", "ham"),
         ("mom says call me now", "ham"),
         ("your account statement is ready", "ham")]
# ... same counting (training) and predict code as above ...
print(predict("urgent kyc update needed today"))
print(predict("your account statement for october"))

Output:

spam
ham

urgent, kyc and update now vote spam. your and account appear once in each class, so they cancel, and statement votes ham.

  1. Why does X = [12, 16, 20] fail in scikit-learn but X = [[12], [16], [20]] works?
Solution

scikit-learn expects a 2-D table: one row per example and one column per feature. [12, 16, 20] is 1-D, so it can't tell whether it is 3 examples with 1 feature or 1 example with 3 features.

Quick quiz

A bank predicts each applicant's credit limit in rupees from past customers whose limits are known. Which task is it?

A) Supervised classification B) Supervised regression C) Unsupervised clustering D) Reinforcement learning

Answer

B) Supervised regression. Past customers come with the correct answer (a label), so it's supervised; the answer is a number of rupees, so it's regression.

Interview questions

  1. What is machine learning? Building programs that improve at a task by learning patterns from data (examples), instead of following hand-written rules. Mitchell: performance at task T, measured by P, improves with experience E.
  2. ML vs traditional programming? Traditional: rules + data → answers. ML: data + answers → rules (a model).
  3. What is a model? Training? Inference? A model is the learned pattern (usually a set of numbers). Training learns it from labelled data; inference uses it to predict for new data.
  4. Feature vs label? Features are the inputs the model reads; the label (target) is the answer it must predict.
  5. Classification vs regression? Classification predicts a category; regression predicts a number.
  6. Supervised vs unsupervised vs reinforcement? Supervised learns from labelled examples; unsupervised finds structure without labels; reinforcement learns actions from rewards by trial and error.
  7. Why evaluate on unseen data? A model can memorise its training examples; only new examples show whether it generalises.
  8. Name the steps of an ML project. Problem framing, data, model, evaluation, deployment (then monitoring and retraining).

Glossary

Term Meaning Example
Spam / ham Unwanted, usually tricky message / a real, wanted message "You WON Rs 50 lakh" / "Meeting at 5"
Machine learning (ML) Programs that improve at a task by learning from examples a spam filter trained on tagged messages
Rule-based programming A human writes every rule (if statements) by hand if "won" in msg: return True
Model The learned pattern, stored and used to predict the counts dictionary; a fitted LinearRegression
Training Learning a model from labelled examples counting words; model.fit(X, y)
Inference (prediction) Using a trained model on new data predict("y0u w0n a prize") → spam
Dataset A collection of examples 6 labelled messages
Example (sample, row) One item of a dataset ("meeting at 5 bring notes", "ham")
Feature An input the model looks at the words; the temperature
Label (target) The correct answer the model must learn to predict spam; cups sold
X / y Conventional names: feature table / labels X = [[12], [16]], y = [182, 165]
Evaluation Measuring a model on examples it never saw 5 test messages
Accuracy Correct predictions ÷ total predictions 4 / 5 = 80%
Classification Predicting a category from a fixed list spam or ham
Regression Predicting a number 152 cups
Supervised learning Learning from labelled examples spam filter, chai sales
Unsupervised learning Finding patterns in data without labels customer groups
Clustering Grouping similar examples together (unsupervised) shoppers with similar baskets
Semi-supervised learning A few labels plus lots of unlabelled data naming a face in 3 photos, found in 3,000
Self-supervised learning Labels created automatically from the data itself guessing a hidden next word
Reinforcement learning Learning actions from rewards by trial and error a game-playing program
ML workflow Frame the problem → data → model → evaluate → deploy building and shipping the spam filter
Problem framing Deciding exactly what to predict and how success is measured "block spam, never a bank message"
Deployment Putting a model where real users get its predictions the filter inside a messaging app
scikit-learn (sklearn) The standard Python library for classic ML LinearRegression
fit / predict scikit-learn's train / infer commands model.fit(X, y), model.predict([[18]])
NotFittedError Error raised when you predict with an untrained model predict before fit

Summary

  • Hand-written rules miss new tricks and block innocent messages; machine learning learns the rules from labelled examples.
  • A dataset is made of examples (rows); each has features (inputs) and a label (the answer).
  • Training builds the model; inference (prediction) uses it on new data; evaluation checks it on unseen data (accuracy).
  • Classification predicts categories, regression predicts numbers.
  • Five kinds of learning: supervised, unsupervised, semi-supervised, self-supervised, reinforcement.
  • Every project follows the workflow: frame the problem, data, model, evaluate, deploy (and keep monitoring).

Next episode: the Python toolkit for ML: Jupyter notebooks, NumPy arrays, pandas DataFrames, matplotlib and scikit-learn.

Sources for the facts above: Arthur Samuel, 1959, IBM 701 checkers program (Wikipedia, "Arthur Samuel (computer scientist)"); Tom Mitchell's definition, Machine Learning, McGraw Hill, 1997 (KDnuggets, "A Concise Explanation of Learning Algorithms with the Mitchell Paradigm").