These are the notes for AI episode 3. In episode 2 Raju always knew the right answer for every row of his notebook. Here that assumption breaks three times, and each time you invent a new way for a machine to learn: supervised, unsupervised (clustering), reinforcement (rewards, explore vs exploit) and self-supervised learning, which is how large language models pretrain. Every Python example below runs as-is in the SyntaxPilot compiler, and the outputs shown are real.
Supervised learning: learning with an answer key
Think of a student with a practice worksheet:
| In a classroom | In machine learning |
|---|---|
| Practice questions (the worksheet) | Features: the inputs |
| The answer key at the back of the book | Labels: the right outputs |
| The student answers, checks the key, corrects themself | Supervised learning: learning from labelled examples |
Raju's notebook in episode 2 was exactly this: weather and time (features) plus what each customer actually ordered (label).
Supervised = learning with the answer key.
Problem 1: no labels
Business is good, and Raju wants to send different offers to different kinds of customers: a loyalty card for some, a party discount for others. His new notebook has two numbers per customer: cups per week and average bill in rupees. But nobody ever wrote down which kind of customer each person is. There is no answer column: no labels.
The obvious move is to reuse the supervised trainer from episode 2.
Demo 1: supervised code crashes
# Raju's new notebook: (cups per week, average bill in Rs)
# No label column: nobody wrote down "which group"
customers = [(14, 20), (12, 18), (15, 22), (2, 160),
(3, 150), (13, 21), (1, 170), (2, 140)]
# Try the supervised trainer from episode 2
for cups, bill, label in customers:
print(cups, bill, label)
Output:
Traceback (most recent call last):
File "main.py", line 7, in <module>
for cups, bill, label in customers:
^^^^^^^^^^^^^^^^^
ValueError: not enough values to unpack (expected 3, got 2)
Python looked for the label and found nothing. Supervised learning needs the answer key, and this data has none. We need a way to learn without any answers.
The fix: find groups (unsupervised learning)
Think of sorting a pile of laundry:
- No shirt has a tag saying which pile it belongs in (data with no labels).
- You put similar things together: whites here, darks there (group by similarity).
- Piles appear on their own; nobody named them first. These piles are clusters.
- Only afterwards do you name the piles: "office shirts", "gym wear" (meaning comes after).
Learning patterns from data with no labels is called unsupervised learning.
Demo 2: clustering in code
customers = [(14, 20), (12, 18), (15, 22), (2, 160),
(3, 150), (13, 21), (1, 170), (2, 140)]
def dist(p, q):
return ((p[0] - q[0]) ** 2 + (p[1] - q[1]) ** 2) ** 0.5
def mean(group):
return (sum(p[0] for p in group) / len(group),
sum(p[1] for p in group) / len(group))
centres = [customers[0], customers[1]] # 2 starting guesses
for step in range(5):
groups = [[], []]
for c in customers: # each customer joins the nearest centre
nearest = min([0, 1], key=lambda i: dist(c, centres[i]))
groups[nearest].append(c)
centres = [mean(g) for g in groups] # move centres to the middle
for g in groups:
print(len(g), "customers:", g)
Output:
4 customers: [(2, 160), (3, 150), (1, 170), (2, 140)]
4 customers: [(14, 20), (12, 18), (15, 22), (13, 21)]
How it works:
distmeasures how far apart two customers are, like measuring a straight line on a graph.- Start with two rough guesses for the middle of each pile (
centres). - Every customer joins the nearest centre; then each centre moves to the middle (
mean) of its group. Repeat five times.
One group comes rarely but spends around ₹150 each time; the other buys about 13 cups a week with a ₹20 bill. The code never knew the words regulars or party orders. It only found the groups. Raju looks at them and decides: daily regulars get a loyalty card, bulk buyers get a party discount. (This "join the nearest centre, move the centre, repeat" loop is the idea behind the well-known k-means algorithm, here with k = 2.)
What unsupervised learning gives you
| Input | Data with features only, no labels |
| Output | Structure, such as groups (clusters) |
| Who gives meaning | The machine finds the groups; a human decides what they mean |
| Used for | Customer segments, similar songs in a music app, transactions that don't fit any normal group (fraud checks) |
Problem 2: no data at all
Raju opens a second stall at the railway station. This time the notebook is completely empty: no data, no labels. He can make one snack per customer: samosa, vada pav or poha. Which sells best here? He has to find out by doing.
The obvious way: samosa sold on day one, so stick with samosa forever.
Demo 3: sticking with one choice fails
import random
random.seed(7)
# True chance that a customer buys each snack (Raju can't see this)
chance = {"samosa": 0.3, "vada pav": 0.8, "poha": 0.5}
def sell(snack): # reward: 1 if it sold, 0 if not
return 1 if random.random() < chance[snack] else 0
# Obvious way: samosa sold on day 1? Stick with it forever.
pick = "samosa"
sold = sum(sell(pick) for day in range(100))
print("always", pick, "-> sold", sold, "of 100")
Output:
always samosa -> sold 33 of 100
chanceis hidden from Raju: it is how the station's customers behave. Vada pav is secretly the favourite.sellreturns a reward: 1 if the customer buys, 0 if not.random.seed(7)makes the random numbers the same on every run, so you get the same output.
No error, no crash: Raju just never found out that vada pav sells far better, because he never tried it. To learn without data, you have to act, see the result, and sometimes try something new.
The fix: rewards (reinforcement learning)
Think of training a dog with treats:
| Training a dog | Reinforcement learning |
|---|---|
| The dog decides what to do | Agent: the learner that acts |
| Sit, jump, bark | Actions: the choices it can make |
| A treat (or nothing) after each try | Reward: a score for the action |
| Repeats what earned treats, but still tries new tricks | Explore vs exploit |
Nobody shows the dog the right answer; it is only told how well it did. Using the best-known choice is exploiting. Trying something new is exploring. Learning by acting and collecting rewards is reinforcement learning (RL).
Demo 4: reinforcement learning in code
import random
random.seed(7)
chance = {"samosa": 0.3, "vada pav": 0.8, "poha": 0.5}
def sell(snack):
return 1 if random.random() < chance[snack] else 0
value = {s: 0.0 for s in chance} # Raju's guess per snack
tries = {s: 0 for s in chance}
sold = 0
for day in range(100):
if random.random() < 0.1: # explore: try anything
snack = random.choice(list(chance))
else: # exploit: best so far
snack = max(value, key=value.get)
reward = sell(snack)
tries[snack] += 1
value[snack] += (reward - value[snack]) / tries[snack]
sold += reward
print("tries:", tries)
print("sold", sold, "of 100")
Output:
tries: {'samosa': 12, 'vada pav': 82, 'poha': 6}
sold 78 of 100
How it works:
valueis Raju's current guess of how often each snack sells;triescounts how often he has made it.- One time in ten (
random.random() < 0.1) he explores and picks any snack; otherwise he exploits the snack with the best guess so far. - After each sale,
value[snack] += (reward - value[snack]) / tries[snack]nudges the guess towards what just happened. (It keeps a running average of the rewards.)
78 sales instead of 33, and vada pav was made 82 times. Raju found the favourite himself, with no data and no labels, just rewards.
At a giant scale: AlphaGo
The same idea, at a giant scale, is behind AlphaGo. In March 2016 it beat the Go champion Lee Sedol four games to one. It first studied human games, then improved by playing against copies of itself and learning from wins and losses.
Problem 3: labels cost too much
Raju's nephew wants a keyboard app that guesses your next word. The supervised way: for every phrase, a person types the correct next word.
"one cutting" -> chai
"cutting chai" -> please
... a person types millions of these
For millions of sentences that is impossibly slow and expensive. But look at any sentence: the answer is already in it. After one cutting, the next word is chai. The text labels itself. Hide the next word and ask the model to guess it.
Demo 5: self-supervised learning in code
text = ("one cutting chai please . one cutting chai please . "
"one vada pav please . two cutting chai please .")
words = text.split()
# The labels come free: every next word is the answer
pairs = [(words[i], words[i + 1]) for i in range(len(words) - 1)]
print(len(pairs), "training pairs, 0 typed by hand")
print(pairs[:4])
# Learn: count which word follows which
follows = {}
for word, nxt in pairs:
follows.setdefault(word, []).append(nxt)
def predict(word):
options = follows[word]
return max(set(options), key=options.count)
print("cutting ->", predict("cutting"))
print("one ->", predict("one"))
Output:
19 training pairs, 0 typed by hand
[('one', 'cutting'), ('cutting', 'chai'), ('chai', 'please'), ('please', '.')]
cutting -> chai
one -> cutting
The code pairs every word with the word that follows it: the next word is the label, and nobody typed it. Creating the labels from the data itself is called self-supervised learning. Large language models learn this way in pretraining: guess the next word in huge amounts of text, check it against the real next word, and update.
All four types side by side
| Type | Where the answers come from | Example from this episode |
|---|---|---|
| Supervised | An answer key: labelled examples | Chai or cold coffee predictor (episode 2) |
| Unsupervised | No answers: find groups and structure | Raju's customer segments, laundry piles |
| Reinforcement | No data to start: act, get rewards, explore vs exploit | The station snack stall, AlphaGo |
| Self-supervised | Labels made from the data itself | Next-word keyboard, LLM pretraining |
The question to ask is always: where do the answers come from?
Glossary
| Term | Meaning |
|---|---|
| Supervised learning | Learning from examples that come with the right answers (labels) |
| Unsupervised learning | Finding structure in data that has no labels |
| Cluster | A group of similar data points found without labels |
| Clustering | Putting similar items into groups (here: nearest centre, move centre, repeat) |
| Centre (centroid) | The middle point of a cluster |
| Reinforcement learning | Learning by acting and receiving rewards |
| Agent | The learner that chooses actions |
| Action | A choice the agent can make (samosa, vada pav, poha) |
| Reward | A score after each action (1 if sold, 0 if not) |
| Explore | Try something new to learn about it |
| Exploit | Use the best-known choice so far |
| Self-supervised learning | The data creates its own labels (the next word is the answer) |
| Pretraining | An LLM's first training stage: predicting the next word in huge amounts of text |
Common mistakes
| Mistake | Code | Error |
|---|---|---|
| Supervised code on data with no label column | for cups, bill, label in customers: |
ValueError: not enough values to unpack (expected 3, got 2) |
| More starting cluster centres than data points | random.sample(customers, 5) with 3 customers |
ValueError: Sample larger than population or is negative |
| An agent that never explores | pick = "samosa" forever |
no error, just sold 33 of 100 |
The second one in full:
import random
customers = [(14, 20), (12, 18), (3, 150)]
centres = random.sample(customers, 5)
Traceback (most recent call last):
File "main.py", line 3, in <module>
centres = random.sample(customers, 5)
File ".../random.py", line 434, in sample
raise ValueError("Sample larger than population or is negative")
ValueError: Sample larger than population or is negative
The number of clusters must be at most the number of rows. For the first mistake: either label the data first, or switch to unsupervised learning. For the third: keep some exploration, for example try something random one time in ten.
Misconceptions
- "Unsupervised learning names the groups." It only finds them. A human gives them meaning.
- "Reinforcement learning is shown the right answer." It is only told how good its action was (a reward).
- "Self-supervised means no data is needed." It needs lots of raw data; it just doesn't need humans to label it.
Practice
- Add a customer
(8, 60)to the clustering demo. Which group does it join? - In Demo 4, change the explore chance from
0.1to0.0. What happens? - Add a fourth snack
"idli"with chance0.9to Demo 4 and rerun. Does Raju find it? - Add your own sentences to the text in Demo 5 and call
predict("chai"). - Name the type of learning: (a) a bank flags transactions that don't fit any normal group, (b) a game bot gets +1 for winning, (c) an email filter trained on emails marked spam/not spam.
Solutions and hints
1. It joins the frequent, low-bill group (it is much closer to bills around ₹20 than ₹150):
4 customers: [(2, 160), (3, 150), (1, 170), (2, 140)]
5 customers: [(14, 20), (12, 18), (15, 22), (13, 21), (8, 60)]
That group's centre moves to about (12.4, 28.2).
2. With 0.0 Raju never explores. All guesses start at 0.0, so max picks the first snack, samosa, and it stays the best guess for ever:
tries: {'samosa': 100, 'vada pav': 0, 'poha': 0}
sold 35 of 100
(35 rather than 33 because this version calls random.random() once more per day for the explore check, so the random sequence is different.)
3. Yes. With chance = {"samosa": 0.3, "vada pav": 0.8, "poha": 0.5, "idli": 0.9} and explore 0.1:
tries: {'samosa': 5, 'vada pav': 3, 'poha': 1, 'idli': 91}
sold 87 of 100
4. Example: add "chai and bun please . chai and bun please .". Now follows["chai"] is ['please', 'please', 'please', 'and', 'and'] and predict("chai") returns please. If two words tie for most common, the pick can change between runs, because the order of a set of strings is not fixed in Python.
5. (a) unsupervised, (b) reinforcement, (c) supervised.
Quick quiz
A music app has 1 crore (10 million) listeners. Nobody labels them. The app puts listeners with similar playlists into taste groups. Which type of learning is this?
- A. Supervised
- B. Unsupervised
- C. Reinforcement
- D. Self-supervised
Answer
B, unsupervised. There are no labels and the goal is to find groups of similar listeners: that is clustering, just like Raju's customers.
Interview questions
- What is the difference between supervised and unsupervised learning? Give one example of each.
- Explain how a simple clustering algorithm like k-means works.
- What are agent, action and reward in reinforcement learning?
- What is the explore vs exploit trade-off? What happens if an agent never explores?
- What is self-supervised learning, and why is it so useful for training LLMs?
- How would you decide which type of learning to use for a new problem? (Ask where the answers come from.)
Summary
- Supervised: learn from labelled examples (an answer key).
- No labels crashed the trainer, so we used unsupervised learning: clustering joins the nearest centre, moves it, and repeats.
- No data at all: reinforcement learning learns from actions and rewards, balancing explore and exploit.
- Labels too costly to type: self-supervised learning uses the next word as a free label, which is how LLMs pretrain.
Next episode: so far Raju's models answered with a choice: chai or cold coffee. But his milk supplier asks "how many cups will you sell tomorrow?" That's a number, not a choice. Classification versus regression.