These are the notes for Data Analytics with Python, episode 40. Episode 39 ended with a warning: P(A | B) is not P(B | A). Today you see how badly that can go wrong, and the rule that fixes it. You will learn true and false positives and negatives, sensitivity and specificity, the law of total probability, Bayes' theorem with its prior, likelihood and posterior, the base rate and the base rate fallacy, and how to check all of it by simulation with NumPy. Every term is defined the first time it appears and collected in the Glossary.
Every program below was run for real (Python 3.13, NumPy 2.5.3, pandas 3.0.5, matplotlib 3.11.2, on a server with no screen). Outputs and images are from those runs. All numbers (the condition, the test, the fraud detector) are made up for teaching; no real test, disease or bank is described. Programs start with sys.tracebacklimit = 0 so an error shows only its message, as in the video.
The problem: Asha's positive test
Asha goes to a free screening camp. The (made-up) condition is rare: 1 in 100 people have it. The poster says the test catches 99% of people who have it; the small print says 5% of healthy people also test positive by mistake. Asha tests positive and concludes: "the test is 99% right, so I'm 99% likely to have it."
Simulating a million people
import sys; sys.tracebacklimit = 0
import numpy as np, pandas as pd
rng = np.random.default_rng(40); n = 1_000_000 # made-up screening test
ill = rng.random(n) < 0.01 # 1 in 100 has it
positive = np.where(ill, rng.random(n) < 0.99, # ill: 99% test +
rng.random(n) < 0.05) # healthy: 5% test +
print("people tested: ", n)
print("tested positive: ", positive.sum())
print("P(ill | positive) =", round(ill[positive].mean(), 3))
people tested: 1000000
tested positive: 59415
P(ill | positive) = 0.166
Only 16.6% of the people who tested positive are ill. 99% is P(positive | ill); Asha needs P(ill | positive). Flipping the condition gives a very different number.
Why: count real people
| Group | People | Test positive |
|---|---|---|
| Ill (1%) | 10,000 | 99% → 9,900 |
| Healthy (99%) | 990,000 | 5% → 49,500 |
| Total | 1,000,000 | 59,400 |
P(ill | +) = 9,900 ÷ 59,400 = 0.167. The test is good; the condition is rare. A small false-alarm rate on a huge healthy crowd produces far more false alarms (49,500) than true ones (9,900). Counting a concrete population like this ("natural frequencies") is the easiest way to get Bayes problems right.
True and false positives, sensitivity and specificity
t = pd.crosstab(pd.Series(ill, name="ill"), pd.Series(positive, name="positive"))
print(t)
print("sensitivity P(+ | ill) =", round(t.loc[True, True] / t.loc[True].sum(), 3))
print("specificity P(- | healthy) =", round(t.loc[False, False] / t.loc[False].sum(), 3))
positive False True
ill
False 940480 49581
True 105 9834
sensitivity P(+ | ill) = 0.989
specificity P(- | healthy) = 0.95
| Test positive | Test negative | |
|---|---|---|
| Ill | true positive (TP): correct alarm (9,834) | false negative (FN): missed case (105) |
| Healthy | false positive (FP): false alarm (49,581) | true negative (TN): correctly cleared (940,480) |
- Sensitivity = TP ÷ (TP + FN) = P(+ | ill): the share of ill people the test catches (0.989 here; 0.99 by design).
- Specificity = TN ÷ (TN + FP) = P(− | healthy): the share of healthy people the test correctly clears (0.95).
- Both describe the test. Neither answers "given my result, am I ill?".
Real-life example: a kitchen smoke alarm. Real fire and it rings: true positive. Burnt toast and it rings: false positive. Real fire and it stays silent: false negative (the dangerous miss). If real fires are rare, most rings are toast.
The law of total probability
A positive can happen two ways: ill and positive, or healthy and positive. Add every path:
P(+) = P(+ | ill) · P(ill) + P(+ | healthy) · P(healthy)
prior, sens, spec = 0.01, 0.99, 0.95
p_pos = sens * prior + (1 - spec) * (1 - prior) # law of total probability
print("P(+) =", round(p_pos, 4))
print("P(ill | +) =", round(sens * prior / p_pos, 3))
P(+) = 0.0594
P(ill | +) = 0.167
P(+) = 0.99 × 0.01 + 0.05 × 0.99 = 0.0099 + 0.0495 = 0.0594. Note that P(+ | healthy) = 1 − specificity.
Bayes' theorem
P(ill | +) = P(+ | ill) × P(ill) ÷ P(+)
| Part | Name | Value | Meaning |
|---|---|---|---|
| P(ill) | prior | 0.01 | belief before the evidence; for a population this is the base rate |
| P(+ | ill) | likelihood | 0.99 | how well the evidence fits the hypothesis |
| P(+) | evidence (normaliser) | 0.0594 | total probability of the evidence |
| P(ill | +) | posterior | 0.167 | belief after the evidence |
You could have invented it. The general multiplication rule (episode 39) says P(A and B) = P(B) × P(A | B) = P(A) × P(B | A). Set the two equal and divide by P(B):
P(A | B) = P(B | A) × P(A) ÷ P(B)
It is named after Thomas Bayes; his essay on the problem was published in 1763, two years after his death, by his friend Richard Price (source).
The base rate decides everything
def bayes(prior, sens, spec):
p_pos = sens * prior + (1 - spec) * (1 - prior)
return sens * prior / p_pos
for prior in [0.001, 0.01, 0.1, 0.5]:
print(f"base rate {prior:>5}: P(ill | +) = {bayes(prior, 0.99, 0.95):.3f}")
print(bayes(0, 0.99, 1.0))
base rate 0.001: P(ill | +) = 0.019
base rate 0.01: P(ill | +) = 0.167
base rate 0.1: P(ill | +) = 0.687
base rate 0.5: P(ill | +) = 0.952
ZeroDivisionError: float division by zero
Same test, wildly different meaning of a positive: 1.9% when the condition affects 1 in 1,000, 16.7% at 1 in 100, 68.7% at 1 in 10, 95.2% at 1 in 2.
The last call raises ZeroDivisionError: float division by zero: with a base rate of 0 and no false alarms (specificity 1.0), nobody can test positive, P(+) = 0, and "what does a positive mean?" has no answer. In real code, check the denominator first:
def bayes(prior, sens, spec):
p_pos = sens * prior + (1 - spec) * (1 - prior)
if p_pos == 0:
return None # no one can test positive
return sens * prior / p_pos
print(bayes(0, 0.99, 1.0), round(bayes(0.01, 0.99, 0.95), 3))
None 0.167
import numpy as np
import matplotlib.pyplot as plt
def bayes(prior, sens, spec):
p_pos = sens * prior + (1 - spec) * (1 - prior)
return sens * prior / p_pos
prior = np.logspace(-4, 0, 200) # 0.0001 to 1
fig, ax = plt.subplots(figsize=(11, 4.5))
ax.plot(prior, bayes(prior, 0.99, 0.95), lw=3)
ax.set_xscale("log"); ax.set_xlabel("base rate (prior)"); ax.set_ylabel("P(ill | +)")
ax.axvline(0.01, color="#e4572e", ls="--")
fig.tight_layout(); fig.savefig("curve.png")
print("at 1%:", round(bayes(0.01, 0.99, 0.95), 3))
at 1%: 0.167

The x axis is on a log scale (each step ×10). For rare conditions the curve hugs zero; at Asha's 1% (dashed line) it is about 0.17.
The base rate fallacy is ignoring the base rate and trusting only the test's accuracy. A fallacy is a mistake in reasoning that feels right. It shows up in lab reports, spam scores, fraud alerts, security screening and "AI detectors".
Updating with a second test
Yesterday's posterior is today's prior:
def bayes(prior, sens, spec):
p_pos = sens * prior + (1 - spec) * (1 - prior)
return sens * prior / p_pos
first = bayes(0.01, 0.99, 0.95)
second = bayes(first, 0.99, 0.95) # yesterday's posterior = today's prior
print(f"after 1 positive: {first:.3f}")
print(f"after 2 positives: {second:.3f}")
after 1 positive: 0.167
after 2 positives: 0.798
Check with the simulation: give every person a second, independent test.
second = np.where(ill, rng.random(n) < 0.99, rng.random(n) < 0.05)
both = positive & second
print("positive twice:", both.sum())
print("P(ill | + and +) =", round(ill[both].mean(), 3))
positive twice: 12199
P(ill | + and +) = 0.798
Formula and simulation agree: two positives raise the chance from 16.7% to about 80%. This is the idea behind confirming a screening result with a second, independent test. (Both tests must make independent errors; repeating the same faulty test twice adds little.)
Bayes in analytics: fraud alerts
def bayes(prior, sens, spec):
p_pos = sens * prior + (1 - spec) * (1 - prior)
return sens * prior / p_pos
p = bayes(prior=0.002, sens=0.95, spec=0.98) # made-up UPI fraud alert
print(f"P(fraud | alert) = {p:.3f}")
print(f"alerts per 100,000 payments: {100_000 * (0.95*0.002 + 0.02*0.998):.0f}")
print(f"real frauds among them: {100_000 * 0.95 * 0.002:.0f}")
P(fraud | alert) = 0.087
alerts per 100,000 payments: 2186
real frauds among them: 190
A made-up UPI fraud detector: 0.2% of payments are fraud, it catches 95% of them, and it flags 2% of normal payments. Only 8.7% of alerts are real fraud: about 2,186 alerts per 100,000 payments, of which about 190 are fraud. If each alert blocks a payment, more than 9 in 10 blocked customers are innocent.
When someone says a model is "95% accurate", ask: how rare is the thing it looks for (base rate), and how often does it raise false alarms (1 − specificity)? With Bayes, those give the share of alerts that are real, called precision (episode 53).
Common mistakes
| Code / habit | Real error / symptom | Why | Fix |
|---|---|---|---|
bayes(0, 0.99, 1.0) |
ZeroDivisionError: float division by zero |
P(+) = 0 | check p_pos > 0 first |
| "sensitivity 0.99, so P(ill | +) = 0.99" | no error; 0.99 vs 0.167 | P(A | B) ≠ P(B | A) | Bayes' theorem |
| ignoring how rare the event is | no error; base rate fallacy | the prior decides the answer | always ask for the base rate |
| quoting one accuracy number | no error; hides false alarms | rare events → most alarms false | report sensitivity, specificity, precision |
| treating two runs of the same test as independent | no error; overconfidence | errors may repeat | use a different, independent test |
Mini project: a Bayes calculator that also counts people
def explain(prior, sens, spec, people=100_000):
ill = prior * people; healthy = people - ill
tp = sens * ill; fp = (1 - spec) * healthy
print(f"Of {people:,} people: {ill:,.0f} ill, {healthy:,.0f} healthy")
print(f"Positives: {tp:,.0f} true + {fp:,.0f} false = {tp + fp:,.0f}")
print(f"P(ill | +) = {tp / (tp + fp):.3f}")
explain(0.01, 0.99, 0.95)
print()
explain(0.002, 0.95, 0.98)
Of 100,000 people: 1,000 ill, 99,000 healthy
Positives: 990 true + 4,950 false = 5,940
P(ill | +) = 0.167
Of 100,000 people: 200 ill, 99,800 healthy
Positives: 190 true + 1,996 false = 2,186
P(ill | +) = 0.087
Practice
1. A condition affects 1 in 1,000. A test has sensitivity 99% and specificity 99%. Count 100,000 people: what is P(ill | +)?
Solution
ill, healthy = 100, 99_900
tp, fp = 0.99 * ill, 0.01 * healthy
print(round(tp), round(fp), round(tp / (tp + fp), 3))
99 999 0.09
99 true positives against 999 false ones: only 9% of positives are ill, even with a 99%/99% test.
2. Simulate the spam quiz with NumPy: 10% spam, the filter flags 90% of spam and 10% of good mail. Estimate P(spam | flag).
Solution
import numpy as np
rng = np.random.default_rng(1); n = 1_000_000
spam = rng.random(n) < 0.10
flag = np.where(spam, rng.random(n) < 0.90, rng.random(n) < 0.10)
print(round(spam[flag].mean(), 3))
0.5
3. Which is the prior, the likelihood and the posterior in: "Before checking, I thought 30% of my emails from this sender were promotions. Promotions contain the word 'offer' 80% of the time. This email contains 'offer'; now I think it's 60% likely to be a promotion"?
Solution
Prior: P(promo) = 0.3. Likelihood: P("offer" | promo) = 0.8. Posterior: P(promo | "offer") = 0.6. (To compute the posterior exactly you'd also need P("offer" | not promo).)
Quick quiz
P(spam) = 0.10
P(flag | spam) = 0.90
P(flag | not spam) = 0.10
P(spam | flag) = ?
A. 0.90 B. 0.50 C. 0.10 D. 0.99
Answer
B. 0.50. P(flag) = 0.9 × 0.1 + 0.1 × 0.9 = 0.18; P(spam | flag) = 0.09 ÷ 0.18 = 0.5. Out of 100 messages: 9 spam flagged, 9 good mail flagged.
Interview questions
- State Bayes' theorem and name its parts. P(A | B) = P(B | A) P(A) / P(B): prior P(A), likelihood P(B | A), evidence P(B), posterior P(A | B).
- Sensitivity vs specificity? Sensitivity = TP / (TP + FN) = P(+ | condition); specificity = TN / (TN + FP) = P(− | no condition).
- What is the base rate fallacy? Ignoring the prior probability and judging only by the evidence's accuracy.
- Why can a 99%-sensitive test give mostly false positives? When the condition is rare, false positives from the large healthy group outnumber true positives.
- What is the law of total probability? P(B) = Σ P(B | Aᵢ) P(Aᵢ) over a set of mutually exclusive, exhaustive events Aᵢ.
- How does a second test change the result? The first posterior becomes the new prior; with an independent second test the posterior rises sharply.
- How would you check a Bayes calculation? Simulate a large population with NumPy, or count natural frequencies.
Glossary
| Term | Meaning | Example |
|---|---|---|
| Screening test | A test given to many people to find a condition early | the camp's test |
| True positive (TP) | Has the condition, test positive | 9,834 |
| False positive (FP) | Doesn't have it, test positive: a false alarm | 49,581 |
| False negative (FN) | Has it, test negative: a missed case | 105 |
| True negative (TN) | Doesn't have it, test negative | 940,480 |
| Sensitivity | P(+ | condition) = TP / (TP + FN) | 0.99 |
| Specificity | P(− | no condition) = TN / (TN + FP) | 0.95 |
| Law of total probability | Add every path to an event: P(B) = Σ P(B | Aᵢ) P(Aᵢ) | P(+) = 0.0594 |
| Bayes' theorem | P(A | B) = P(B | A) P(A) / P(B) | 0.167 |
| Prior | Belief before the evidence | P(ill) = 0.01 |
| Base rate | How common something is in the whole population; the prior for a random member | 1 in 100 |
| Likelihood | Probability of the evidence if the hypothesis is true | P(+ | ill) = 0.99 |
| Posterior | Belief after the evidence | P(ill | +) = 0.167 |
| Base rate fallacy | Ignoring the base rate when judging evidence | "99% test → 99% ill" |
| Fallacy | A mistake in reasoning that feels right | the base rate fallacy |
| Natural frequencies | Counting a concrete population instead of using percentages | 9,900 of 59,400 |
| Simulation | Imitating a random process many times with random numbers | a million people |
| Precision | Share of alerts that are real: TP / (TP + FP) (episode 53) | 8.7% of fraud alerts |
ZeroDivisionError |
Python error when dividing by zero | bayes(0, 0.99, 1.0) |
Summary
- P(ill | +) ≠ P(+ | ill). A 99%-sensitive test gave a 16.7% chance of being ill.
- Count people: rare condition + small false-alarm rate on many healthy people = mostly false positives.
- Sensitivity and specificity describe the test; the posterior needs the base rate too.
- Law of total probability: P(+) adds every path. Bayes' theorem: posterior = likelihood × prior ÷ evidence.
- The base rate fallacy is ignoring the prior; a second independent test turns the posterior into the next prior.
- In analytics, alerts for rare events (fraud, defects, churn) are mostly false alarms: check the base rate.
- Next episode: random variables and discrete distributions.