These are the notes for Python episode 29. Raju's offer board says bills of Rs 100 or more get 10% off. One evening he tidies his code and, without noticing, >= becomes >. He "checks" by printing a few bills, every number looks believable, and a week later a customer with a bill of exactly Rs 100 complains. In this episode you invent automated tests: small programs that check your program for you, every time it changes. You start with the assert keyword, move to Python's built-in unittest module, and finish with pytest, which most Python teams use. Every compiler example below runs as-is in the SyntaxPilot compiler, with its real output from Python 3.13; pytest output is from pytest 9 run in a terminal.
The problem: checking by eye
def final_bill(amount):
if amount > 100:
return amount - amount // 10
return amount
print(final_bill(50))
print(final_bill(100))
print(final_bill(200))
Output:
50
100
180
Does that look right? At a glance, yes. But Rs 100 should give 90. Nothing on the screen says "wrong".
Why it fails: printing shows what the code does, not what it should do. You have to remember the right answers and compare by eye, after every change, forever. Nobody keeps doing that.
assert: write the right answer down
A teacher doesn't re-solve every question for every paper; she has an answer key. A test is the same idea: next to each call, write the answer it should give, and let Python compare.
assert condition does nothing if the condition is true, and stops the program with an AssertionError if it is false. An optional message goes after a comma.
def final_bill(amount):
if amount > 100:
return amount - amount // 10
return amount
assert final_bill(50) == 50
assert final_bill(200) == 180
assert final_bill(100) == 90, "Rs 100 should get 10% off"
print("all checks passed")
Output:
Traceback (most recent call last):
File "main.py", line 8, in <module>
assert final_bill(100) == 90, "Rs 100 should get 10% off"
^^^^^^^^^^^^^^^^^^^^^
AssertionError: Rs 100 should get 10% off
Caught in a second, not a week. But: the first failing assert stops everything (with three bugs you'd see one at a time), and there's no report of what passed.
assertis for tests and for "this can never happen" checks while developing. Don't use it to validate user input in real programs: Python skips everyassertwhen run withpython -O.
unittest: Python's built-in tester
| Piece | What it does |
|---|---|
import unittest |
the testing module that comes with Python |
class TestX(unittest.TestCase): |
a group of tests (inheritance, episode 22) |
a method named test_... |
one test; only methods starting with test are run |
self.assertEqual(actual, expected) |
the check: what the code gave vs what it should give |
unittest.main() |
finds every test, runs each one even if others fail, prints a report |
In the report: . = passed, F = failed (an assert was false), E = error (the test crashed with an unexpected exception).
import unittest
def final_bill(amount):
if amount > 100:
return amount - amount // 10
return amount
class TestFinalBill(unittest.TestCase):
def test_small_bill(self):
self.assertEqual(final_bill(50), 50)
def test_exactly_100(self):
self.assertEqual(final_bill(100), 90)
def test_big_bill(self):
self.assertEqual(final_bill(200), 180)
unittest.main()
Output:
.F.
======================================================================
FAIL: test_exactly_100 (__main__.TestFinalBill.test_exactly_100)
----------------------------------------------------------------------
Traceback (most recent call last):
File "main.py", line 13, in test_exactly_100
self.assertEqual(final_bill(100), 90)
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
AssertionError: 100 != 90
----------------------------------------------------------------------
Ran 3 tests in 0.001s
FAILED (failures=1)
It names the test, the line, what we got (100) and what we wanted (90). Fix the bug (>=) and add a test that a negative bill is refused. with self.assertRaises(ValueError): means "the code inside this block must raise ValueError, or the test fails".
import unittest
def final_bill(amount):
if amount < 0:
raise ValueError("amount cannot be negative")
if amount >= 100:
return amount - amount // 10
return amount
class TestFinalBill(unittest.TestCase):
def test_small_bill(self):
self.assertEqual(final_bill(50), 50)
def test_exactly_100(self):
self.assertEqual(final_bill(100), 90)
def test_big_bill(self):
self.assertEqual(final_bill(200), 180)
def test_negative(self):
with self.assertRaises(ValueError):
final_bill(-20)
unittest.main()
Output:
....
----------------------------------------------------------------------
Ran 4 tests in 0.000s
OK
More assert methods
| Method | Passes when |
|---|---|
assertEqual(a, b) / assertNotEqual(a, b) |
a == b / a != b |
assertTrue(x) / assertFalse(x) |
x is truthy / falsy |
assertIn(a, b) / assertNotIn(a, b) |
a in b / a not in b |
assertIsNone(x) / assertIsNotNone(x) |
x is None / x is not None |
assertAlmostEqual(a, b) |
a and b are equal to 7 decimal places |
assertRaises(Error) |
the block raises Error |
Decimals: assertAlmostEqual
import unittest
class TestMath(unittest.TestCase):
def test_total(self):
self.assertEqual(0.1 + 0.2, 0.3)
unittest.main()
Output (key lines):
AssertionError: 0.30000000000000004 != 0.3
FAILED (failures=1)
Floats are stored in binary and are slightly off (episode 4). Use self.assertAlmostEqual(0.1 + 0.2, 0.3), which prints OK.
setUp: a fresh start for every test
setUp runs before each test, so every test gets fresh data and tests can't affect each other. verbosity=2 prints one line per test.
import unittest
class TestBill(unittest.TestCase):
def setUp(self):
self.menu = {"chai": 10, "samosa": 15}
def test_chai(self):
self.assertEqual(self.menu["chai"], 10)
def test_add_item(self):
self.menu["poha"] = 25
self.assertEqual(len(self.menu), 3)
def test_menu_fresh(self):
self.assertEqual(len(self.menu), 2)
unittest.main(verbosity=2)
Output:
test_add_item (__main__.TestBill.test_add_item) ... ok
test_chai (__main__.TestBill.test_chai) ... ok
test_menu_fresh (__main__.TestBill.test_menu_fresh) ... ok
----------------------------------------------------------------------
Ran 3 tests in 0.000s
OK
pytest: less ceremony
pytest is a third-party package (install it with pip install pytest, episode 30). Tests are plain functions whose names start with test, using the plain assert keyword. Run pytest in the project folder: it finds files named test_*.py (or *_test.py) and runs every test function in them.
Project layout:
shop/
├── bill.py
└── test_bill.py
bill.py (still with the > bug):
def final_bill(amount):
if amount < 0:
raise ValueError("amount cannot be negative")
if amount > 100:
return amount - amount // 10
return amount
test_bill.py:
import pytest
from bill import final_bill
def test_small_bill():
assert final_bill(50) == 50
def test_exactly_100():
assert final_bill(100) == 90
def test_negative():
with pytest.raises(ValueError):
final_bill(-20)
$ pytest
collected 3 items
test_bill.py .F. [100%]
=================================== FAILURES ===================================
_______________________________ test_exactly_100 _______________________________
def test_exactly_100():
> assert final_bill(100) == 90
E assert 100 == 90
E + where 100 = final_bill(100)
test_bill.py:8: AssertionError
=========================== short test summary info ============================
FAILED test_bill.py::test_exactly_100 - assert 100 == 90
========================= 1 failed, 2 passed in 0.03s ==========================
pytest shows the actual values inside a plain assert (where 100 = final_bill(100)). pytest.raises does what assertRaises does.
Many cases at once: @pytest.mark.parametrize
Bugs love edges. Instead of one function per case, give one test a list of inputs and expected answers:
import pytest
from bill import final_bill
@pytest.mark.parametrize("amount, expected", [
(50, 50), (99, 99), (100, 90), (200, 180), (0, 0),
])
def test_final_bill(amount, expected):
assert final_bill(amount) == expected
def test_negative():
with pytest.raises(ValueError):
final_bill(-20)
After fixing the bug (>=), pytest -v (verbose):
test_bill.py::test_final_bill[50-50] PASSED [ 16%]
test_bill.py::test_final_bill[99-99] PASSED [ 33%]
test_bill.py::test_final_bill[100-90] PASSED [ 50%]
test_bill.py::test_final_bill[200-180] PASSED [ 66%]
test_bill.py::test_final_bill[0-0] PASSED [ 83%]
test_bill.py::test_negative PASSED [100%]
============================== 6 passed in 0.02s ===============================
Boundary values (99 and 100, right on either side of the rule, and 0) are where bugs hide. Always test them. For floats in pytest: assert 0.1 + 0.2 == pytest.approx(0.3).
unittest vs pytest
| unittest | pytest | |
|---|---|---|
| comes with Python | yes | no: pip install pytest |
| a test is | a test_ method in a TestCase class |
a test_ function |
| checking | self.assertEqual, assertTrue, ... |
plain assert |
| expected errors | with self.assertRaises(E): |
with pytest.raises(E): |
| many cases | a loop with self.subTest |
@pytest.mark.parametrize |
| run | unittest.main() or python -m unittest |
pytest |
pytest also runs unittest tests, so you can switch any time.
Common mistakes
| Mistake | What you see | Why | Fix |
|---|---|---|---|
a test method named check_bill |
Ran 0 tests / NO TESTS RAN |
only test... names are run |
def test_bill(self): |
assertEqual(final_bill(100), 90) fails |
AssertionError: 100 != 90 |
the code (or the expected value) is wrong | read got vs wanted, fix the code |
assertEqual(0.1 + 0.2, 0.3) |
AssertionError: 0.30000000000000004 != 0.3 |
floats are inexact | assertAlmostEqual / pytest.approx |
calling final_bill(-20) in a test without assertRaises |
E and ERROR: ... ValueError: amount cannot be negative |
an exception the test didn't expect | with self.assertRaises(ValueError): |
| only testing 50 and 200 | no error: the > bug slips through |
edges not tested | test 99, 100, 0 |
| tests that depend on each other's data | a test passes alone, fails with others | shared state | build fresh data in setUp |
Mini project: testing last episode's phone-number checker
Two numbers must pass; four must fail (too short, starts with 5, eleven digits, contains a letter). with self.subTest(n=n): runs each number as its own small test, so the report names the one that failed. This version has a planted bug: re.search instead of re.fullmatch.
import re
import unittest
def is_mobile(text):
return re.search(r"[6-9]\d{9}", text) is not None
class TestIsMobile(unittest.TestCase):
def test_valid(self):
for n in ["9876543210", "6000000000"]:
with self.subTest(n=n):
self.assertTrue(is_mobile(n))
def test_invalid(self):
for n in ["12345", "5876543210", "98765432101", "98765 4321a"]:
with self.subTest(n=n):
self.assertFalse(is_mobile(n))
unittest.main()
Output:
F.
======================================================================
FAIL: test_invalid (__main__.TestIsMobile.test_invalid) (n='98765432101')
----------------------------------------------------------------------
Traceback (most recent call last):
File "main.py", line 16, in test_invalid
self.assertFalse(is_mobile(n))
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
AssertionError: True is not false
----------------------------------------------------------------------
Ran 2 tests in 0.001s
FAILED (failures=1)
search found ten good digits inside the eleven-digit number. Change re.search to re.fullmatch and run again: Ran 2 tests ... OK.
The habit: when you find a bug, first write a test that fails because of it, then fix the code until the test passes. Writing tests before the code, on purpose, is called test-driven development (TDD).
Test first: red, green, refactor
Raju's friends want to split a bill so that no rupee is lost. Write the tests before the function exists:
import unittest
class TestSplitBill(unittest.TestCase):
def test_even(self):
self.assertEqual(split_bill(300, 3), [100, 100, 100])
def test_uneven(self):
self.assertEqual(split_bill(100, 3), [34, 33, 33])
unittest.main()
Output (key lines): EE, then twice NameError: name 'split_bill' is not defined, and FAILED (errors=2). This is the red step: the tests say what we want, and nothing passes yet.
The simplest code that could work:
def split_bill(total, people):
return [total // people] * people
Output (key lines):
.F
FAIL: test_uneven (__main__.TestSplitBill.test_uneven)
AssertionError: Lists differ: [33, 33, 33] != [34, 33, 33]
First differing element 0:
33
34
- [33, 33, 33]
+ [34, 33, 33]
FAILED (failures=1)
One rupee went missing. divmod(a, b) divides and returns both the quotient and the remainder, so the first extra people pay one rupee more:
def split_bill(total, people):
share, extra = divmod(total, people)
return [share + 1] * extra + [share] * (people - extra)
Output: .., Ran 2 tests in 0.000s, OK. This is green. Now you can tidy the code (refactor) and the tests tell you at once if you broke anything. Red, green, refactor is the cycle of test-driven development.
Practice
1. Write five tests for a grade(marks) function, including the edges and an out-of-range value.
Solution
import unittest
def grade(marks):
if marks < 0 or marks > 100:
raise ValueError("marks must be 0 to 100")
if marks >= 90:
return "A"
if marks >= 60:
return "B"
return "C"
class TestGrade(unittest.TestCase):
def test_top(self):
self.assertEqual(grade(95), "A")
def test_edge_90(self):
self.assertEqual(grade(90), "A")
def test_edge_89(self):
self.assertEqual(grade(89), "B")
def test_low(self):
self.assertEqual(grade(10), "C")
def test_out_of_range(self):
with self.assertRaises(ValueError):
grade(101)
unittest.main()
Output:
.....
----------------------------------------------------------------------
Ran 5 tests in 0.000s
OK
2. Test that int("abc") raises ValueError, and that the message mentions "invalid literal".
Solution
import unittest
class TestInt(unittest.TestCase):
def test_bad_text(self):
with self.assertRaises(ValueError) as cm:
int("abc")
self.assertIn("invalid literal", str(cm.exception))
unittest.main()
Output:
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
as cm keeps the caught exception in cm.exception.
3. Write parametrized pytest tests for the fixed is_mobile.
Solution
mobile.py:
import re
def is_mobile(text):
return re.fullmatch(r"[6-9]\d{9}", text) is not None
test_mobile.py:
import pytest
from mobile import is_mobile
@pytest.mark.parametrize("number", ["9876543210", "6000000000"])
def test_valid(number):
assert is_mobile(number)
@pytest.mark.parametrize("number", ["12345", "5876543210", "98765432101", "98765 4321a"])
def test_invalid(number):
assert not is_mobile(number)
$ pytest -q
...... [100%]
6 passed in 0.01s
Quick quiz
import unittest
class T(unittest.TestCase):
def test_a(self): self.assertEqual(1, 1)
def check_b(self): self.assertEqual(1, 2)
def test_c(self): self.assertTrue(0)
unittest.main()
How many tests run, and what is the result? A. 3 tests, OK B. 2 tests, OK C. 2 tests, 1 failure D. 3 tests, 2 failures
Answer
C. check_b doesn't start with test, so it never runs. test_c fails with AssertionError: 0 is not true. The report ends with Ran 2 tests and FAILED (failures=1).
Interview questions
- What is a unit test? A small automated check of one unit (a function or class) in isolation: given this input, expect this output.
- Failure vs error in unittest? A failure (
F) is an assertion that was false; an error (E) is an unexpected exception inside the test. - Why test boundary values? Off-by-one mistakes (
>vs>=,range(n)vsrange(n + 1)) only show at the edges. - What does
setUpdo? Runs before each test so every test starts from fresh data;tearDownruns after each test. - Why is pytest popular? Plain
assertwith detailed failure messages, less boilerplate,parametrize, fixtures, and plugins; it also runs unittest tests. - What is TDD? Test-driven development: write a failing test, write just enough code to pass it, then clean up (red, green, refactor).
- Why not use
assertfor input validation in production code?python -Oremoves asserts; raise a real exception (ValueError) instead.
Glossary
| Term | Meaning |
|---|---|
| test | code that checks other code by comparing actual results with expected ones |
| automated test | a test the computer runs for you, as often as you like |
| unit test | a test of one small piece (a function or class) |
assert |
a keyword: do nothing if the condition is true, raise AssertionError if false |
AssertionError |
the exception raised when an assertion is false |
| expected vs actual | what the code should give vs what it gave |
unittest |
Python's built-in testing module |
TestCase |
the unittest class your test classes inherit from |
| test method | a method whose name starts with test; one test |
| test runner | the tool that finds and runs tests and prints a report (unittest.main(), pytest) |
failure (F) |
a test whose assertion was false |
error (E) |
a test that crashed with an unexpected exception |
assertRaises / pytest.raises |
checks that a block raises a given exception |
assertAlmostEqual / pytest.approx |
compares floats allowing a tiny difference |
subTest |
runs one case of a loop as its own reported sub-test |
setUp / tearDown |
methods run before / after each test |
| pytest | a popular third-party test runner using plain functions and assert |
| parametrize | running one test function with many input/expected pairs |
| edge case / boundary value | an input at the limit of a rule (99, 100, 0, empty list) |
| regression | a bug that comes back after a change; tests catch regressions |
| red, green, refactor | the TDD cycle: a failing test, code that passes it, then tidy-up |
divmod(a, b) |
returns (a // b, a % b): quotient and remainder |
| test-driven development (TDD) | writing a failing test first, then the code that makes it pass |
Summary
- Printing shows what code does; a test states what it should do.
assertis silent when true, raisesAssertionErrorwhen false.unittest: aTestCaseclass,test_methods,self.assertEqual,unittest.main();.pass,Ffail,Eerror.assertRaisesfor expected errors,assertAlmostEqualfor floats,subTestfor loops,setUpfor fresh data.pytest: plain functions andassert, great failure messages,@pytest.mark.parametrize,pytest.raises.- Test the edges, and write a failing test for every bug before fixing it.
- Next episode: two projects need two versions of the same package. You'll invent virtual environments, and learn pip and project structure.