SyntaxPilot
Courses/AI: Zero to Agents/Lesson 7

AI: Zero to Agents · Lesson 7 of 40 · 10 min

How a model improves

queued

These are the notes for AI episode 7. In episode 4 we trained a model by trying every possible line. Here you see why that is impossible for real AI, and invent the method almost every modern model uses to learn: gradient descent. Along the way: parameters, mean squared error, the slope (gradient), the learning rate, steps and epochs, the loss curve, local minima and batches. Every Python example below runs as-is in the SyntaxPilot compiler, and the outputs shown are real.

A model is a box of knobs

A music mixer In machine learning
Volume, bass and treble knobs Parameters: the numbers a model learns (our chai line had two: start and drop)
"Too loud!": how bad it sounds Loss: one number saying how wrong the model is
Turning knobs and listening, again and again Training: changing the parameters to lower the loss

Demo 1: trying everything fails

# Episode 4's training: try 41 values for each knob
values_per_knob = 41
tries_per_second = 1_000_000          # a fast computer
seconds_per_year = 60 * 60 * 24 * 365

for knobs in [2, 10, 100, 1000]:
    tries = values_per_knob ** knobs
    years = tries / tries_per_second / seconds_per_year
    print(f"{knobs:4} knobs: {years:.1e} years")

Output:

   2 knobs: 5.3e-11 years
  10 knobs: 4.3e+02 years
 100 knobs: 6.0e+147 years
Traceback (most recent call last):
  File "main.py", line 8, in <module>
    years = tries / tries_per_second / seconds_per_year
            ~~~~~~^~~~~~~~~~~~~~~~~~
OverflowError: integer division result too large for a float
  • 2 knobs: 1,681 tries, a tiny fraction of a second.
  • 10 knobs: about 430 years.
  • 100 knobs: 6 × 10¹⁴⁷ years.
  • 1,000 knobs: the number is too big for Python to even turn into a decimal (OverflowError).

Every new knob multiplies the number of tries by 41. Modern AI models have millions or billions of parameters. We need a way to improve without trying everything.

The fix: walk downhill on a foggy hill

On the foggy hill In machine learning
Thick fog: you can't see the valley We can't check every answer
Feel the ground: which way slopes down? The slope, or gradient
Take one step downhill, then feel again One step of training
Reach the bottom of the valley The lowest loss

The height of the hill is the loss; your position is the knob settings. Walking downhill on the loss is called gradient descent.

repeat:
    slope = how the loss changes if I nudge the knob
    knob  = knob - learning_rate × slope      # step AGAINST the slope = downhill

A smooth hill: mean squared error

In episode 4 the loss was the mean absolute error (the average |gap|). For walking downhill we usually square each gap: the mean squared error (MSE).

Gap Absolute error Squared error
1 1 1
5 5 25
10 10 100
  • Big mistakes cost much more.
  • The loss becomes a smooth bowl: steep far from the answer, gentle near the bottom. So the slope tells us which way to go and roughly how far.

Feeling the slope in code: nudge a knob by a tiny amount (0.001) and see how much the loss changes: slope = (loss(x + 0.001) - loss(x)) / 0.001. This is a numerical slope; real libraries compute exact slopes with calculus.

Demo 2: gradient descent on one knob

To keep it simple we measure temperature from 24 °C (t - 24) and fix base at the average number of cups, so there is only one knob to learn: drop.

# (temperature °C, cups sold); x = degrees above 24 °C
days = [(14, 96), (18, 88), (22, 81), (26, 72), (30, 63), (34, 57)]
base = 76.17                         # average cups (at 24 °C)

def loss(drop):                      # mean squared error
    return sum((base - drop * (t - 24) - cups) ** 2
               for t, cups in days) / len(days)

drop = 0.0                           # start anywhere
learning_rate = 0.002
print(f"start:   drop = {drop:.3f}   loss = {loss(drop):.2f}")
for step in range(1, 21):
    slope = (loss(drop + 0.001) - loss(drop)) / 0.001   # feel the ground
    drop = drop - learning_rate * slope                  # step downhill
    if step % 4 == 0:
        print(f"step {step:2}: drop = {drop:.3f}   loss = {loss(drop):.2f}")

Output:

start:   drop = 0.000   loss = 185.81
step  4: drop = 1.121   loss = 35.98
step  8: drop = 1.611   loss = 7.28
step 12: drop = 1.825   loss = 1.78
step 16: drop = 1.919   loss = 0.72
step 20: drop = 1.960   loss = 0.52

The loss falls step by step and drop climbs towards 2 cups per degree, with no brute force. Why subtract? If the loss goes up as drop grows, the slope is positive, so we move drop down; if the slope is negative, we move it up. Always against the slope.

The learning rate: how big is each step?

On the hill Learning rate Result
Tiny steps too small safe, but painfully slow
Sensible steps just right reaches the valley quickly
Giant leaps too big jumps over the valley, lands higher, and blows up

Demo 3: too small, too big

days = [(14, 96), (18, 88), (22, 81), (26, 72), (30, 63), (34, 57)]
base = 76.17

def loss(drop):
    return sum((base - drop * (t - 24) - cups) ** 2
               for t, cups in days) / len(days)

def train(learning_rate, steps=20):
    drop = 0.0
    for _ in range(steps):
        slope = (loss(drop + 0.001) - loss(drop)) / 0.001
        drop = drop - learning_rate * slope
    return drop

for lr in [0.0001, 0.002, 0.05]:
    drop = train(lr)
    print(f"learning rate {lr}: drop = {drop:.3g}   loss = {loss(drop):.3g}")

Output:

learning rate 0.0001: drop = 0.341   loss = 128
learning rate 0.002: drop = 1.96   loss = 0.519
learning rate 0.05: drop = -3.89e+11   loss = 7.06e+24
  • Too small: after 20 steps the loss is still 128. Crawling.
  • Just right: loss about 0.5.
  • Too big: no crash, no error, the loss is 7 × 10²⁴. Each step overshoots the bottom, the slope on the other side is steeper, and the next step is even bigger.

The learning rate is a setting we choose, not something the model learns. Such settings are called hyperparameters (episode 11 covers them in depth).

Demo 4: training both knobs, epochs and the loss curve

days = [(14, 96), (18, 88), (22, 81), (26, 72), (30, 63), (34, 57)]

def loss(base, drop):                # now both knobs are learned
    return sum((base - drop * (t - 24) - cups) ** 2
               for t, cups in days) / len(days)

base, drop = 0.0, 0.0                # start anywhere
lr, h = 0.02, 0.001
print(f"start      loss {loss(base, drop):7.1f}")
for epoch in range(1, 201):          # epoch = one pass over all the days
    slope_base = (loss(base + h, drop) - loss(base, drop)) / h
    slope_drop = (loss(base, drop + h) - loss(base, drop)) / h
    base -= lr * slope_base          # both knobs step downhill together
    drop -= lr * slope_drop
    if epoch % 25 == 0:
        bar = "#" * round(loss(base, drop) / 150)
        print(f"epoch {epoch:3}  loss {loss(base, drop):7.1f} {bar}")

print(f"learned: base {base:.1f}, drop {drop:.2f}")
print("20 °C ->", round(base - drop * (20 - 24)), "cups")

Output:

start      loss  5987.2
epoch  25  loss   754.1 #####
epoch  50  loss    98.3 #
epoch  75  loss    13.2 
epoch 100  loss     2.1 
epoch 125  loss     0.7 
epoch 150  loss     0.5 
epoch 175  loss     0.5 
epoch 200  loss     0.5 
learned: base 76.1, drop 1.99
20 °C -> 84 cups
  • One epoch = one pass over all the training data.
  • The falling shape of the loss is the loss curve. When it flattens, training has converged.
  • 20 °C → 84 cups: the same answer as episode 4, without trying every line. This exact loop scales to millions of knobs.
loss
6000 |#
     |
 750 |  #
 100 |     #
  13 |        #
 0.5 |           #  #  #  #      ← flat: converged
     +--------------------------- epochs

Three real-world details

Term What it means
Local minimum A small dip in a bumpy loss landscape that isn't the lowest valley. With huge models this usually matters less than people fear, and some randomness helps escape.
Batch / stochastic gradient descent (SGD) With millions of examples, each step uses a small random group (a batch) instead of the whole dataset. One epoch is then many steps.
Backpropagation Real libraries compute all the slopes at once with calculus instead of nudging each knob. In neural networks this is called backpropagation (coming in the neural-network episodes).

Glossary

Term Meaning
Parameter A number the model learns (a knob), e.g. base, drop
Loss One number measuring how wrong the model is
Mean squared error (MSE) The average of the squared gaps between guess and truth
Loss landscape The "hill": loss for every possible setting of the knobs
Slope / gradient Which way, and how steeply, the loss changes when a knob moves
Gradient descent Repeatedly stepping each knob against its slope to lower the loss
Learning rate How big each step is (a hyperparameter)
Hyperparameter A setting we choose before training, not learned (learning rate, number of epochs)
Step One update of the knobs
Epoch One full pass over the training data
Loss curve The loss plotted over training steps or epochs
Converged Training has settled: the loss stops going down
Local minimum A dip that is lower than its surroundings but not the lowest point
Batch A small group of examples used for one step
Stochastic gradient descent (SGD) Gradient descent where each step uses a random batch
Backpropagation Computing all the gradients of a neural network at once with calculus

Common mistakes

Mistake Code Error
Brute force on many knobs 41 ** 1000 / 1_000_000 OverflowError: integer division result too large for a float
Learning rate too big learning_rate = 0.05 no error, loss 7.06e+24
Adding the slope instead of subtracting drop = drop + learning_rate * slope no error, loss keeps rising
Learning rate too small learning_rate = 0.0001 no error, loss still 128 after 20 steps

Fixes: use gradient descent; if the loss grows, try a learning rate 10× smaller; always step against the slope; if the loss falls too slowly, raise the learning rate or train for more epochs.

Practice

  1. Find the largest learning rate that still works for Demo 3 (20 steps).
  2. Start drop at 10.0 instead of 0.0 (learning rate 0.002). Does it still head for 2?
  3. Swap - for + in the step on purpose. What happens?
  4. Print the loss every epoch in Demo 4 and sketch the loss curve.
Solutions

1. Try several values (the train function also takes a starting drop and a direction sign, used in questions 2 and 3):

days = [(14, 96), (18, 88), (22, 81), (26, 72), (30, 63), (34, 57)]
base = 76.17

def loss(drop):
    return sum((base - drop * (t - 24) - cups) ** 2
               for t, cups in days) / len(days)

def train(learning_rate, steps=20, drop=0.0, sign=-1):
    for _ in range(steps):
        slope = (loss(drop + 0.001) - loss(drop)) / 0.001
        drop = drop + sign * learning_rate * slope
    return drop

for lr in [0.01, 0.02, 0.021, 0.022, 0.03]:
    d = train(lr)
    print(f"learning rate {lr}: drop = {d:.3g}   loss = {loss(d):.3g}")
d = train(0.002, drop=10.0)
print(f"start at 10: drop = {d:.3f}   loss = {loss(d):.2f}")
d = train(0.002, sign=+1)
print(f"plus instead of minus: drop = {d:.3g}   loss = {loss(d):.3g}")
learning rate 0.01: drop = 1.99   loss = 0.47
learning rate 0.02: drop = 1.88   loss = 1.08
learning rate 0.021: drop = 1.11   loss = 36.7
learning rate 0.022: drop = -3.64   loss = 1.48e+03
learning rate 0.03: drop = -2.54e+05   loss = 3.01e+12
start at 10: drop = 2.121   loss = 1.23
plus instead of minus: drop = -59.1   loss = 1.74e+05

0.01 is the best here; around 0.02 it still works but zig-zags; from about 0.022 it blows up. (For this loss the limit is 2 ÷ the bowl's curvature ≈ 0.0214.)

2. Yes: from 10 it walks down to 2.121 in 20 steps (loss 1.23) and would reach 2 with more steps. Gradient descent can start anywhere on a smooth bowl.

3. It walks uphill: drop goes to -59.1 and the loss to 174,000.

4. Change if epoch % 25 == 0: to print every epoch; the loss drops fast at first, then flattens near 0.5.

Quick quiz 1

The loss printed every epoch is 120, 340, 1900, 25000, 410000 ... What should you change first?

  • A. Train for more epochs
  • B. Lower the learning rate
  • C. Add more knobs
  • D. Use the test set for training
Answer

B, lower the learning rate. A loss that grows faster and faster means the steps overshoot the valley. More epochs would only make it worse.

Quick quiz 2

At drop = 1.0 the slope of the loss is -40, and learning_rate = 0.01. What is the new drop?

  • A. 0.6
  • B. 1.0
  • C. 1.4
  • D. -39.0
Answer

C, 1.4. new = old − learning_rate × slope = 1.0 − 0.01 × (−40) = 1.0 + 0.4. A negative slope means downhill is towards a bigger drop.

Interview questions

  • What is gradient descent? Explain it without maths.
  • Why can't we train a large model by trying every combination of parameters?
  • What happens if the learning rate is too small? Too large? How do you spot each one?
  • What is the difference between a step and an epoch?
  • Why is mean squared error popular as a loss for gradient descent?
  • What is a local minimum? What is stochastic (mini-batch) gradient descent and why is it used?

Summary

  • A model is a box of knobs: its parameters. Training lowers the loss by turning them.
  • Trying every setting takes centuries for 10 knobs and overflows for 1,000.
  • Gradient descent: feel the slope of the loss, step against it, repeat.
  • MSE squares the gaps, giving a smooth bowl to walk down.
  • The learning rate sets the step size: too small crawls, too big explodes.
  • Watch the loss curve over epochs; when it flattens, training has converged.

Next episode: our model is still a straight line. What if the pattern isn't a line at all? The artificial neuron: weights, bias and activation.