These are the notes for Python episode 26. Since episode 20 you've been writing with open(...) as f:. In this episode you find out what with really does: a bad price crashes Raju's save halfway, and the file ends up empty and still open, because the clean-up line never ran. We fix it with try/finally, then invent context managers: objects with __enter__ (setup) and __exit__ (clean-up that always runs). We cover what __exit__ receives when there's an error, three traps (using a file after the block, objects that don't support with, and __exit__ swallowing errors), the @contextmanager shortcut with yield, suppress(), managing two files in one with, and a mini project: a stock transaction that rolls back. Every example below runs as-is in the SyntaxPilot compiler, with its real output from Python 3.13.
The problem: a crash empties the file
Every morning Raju opens the shop shutter, and every night he pulls it down. A file is the same: open, use, close. Then one day there's a power cut at noon and everyone runs out: the shutter is left open. In code, an error is that power cut.
def save_sales(sales):
f = open("sales.txt", "w")
for item, price in sales:
f.write(f"{item},{int(price)}\n")
f.close()
try:
save_sales([("chai", "10"), ("samosa", "ten")])
except ValueError as e:
print("Error:", e)
print("File says:", repr(open("sales.txt").read()))
Output:
Error: invalid literal for int() with base 10: 'ten'
File says: ''
int("ten") fails (a ValueError, episode 18), which is fair. But the file is empty, even though chai,10 was written before the crash. (repr() shows the text with its quotes, so an empty string appears as ''.)
Why it fails: Python doesn't write to the disk on every write(). It collects text in a buffer, a waiting area in memory, and saves it when the buffer fills up or the file is closed. The error jumped out of the loop, so f.close() never ran: the file is still open and chai,10 is still sitting in the buffer. In a long-running billing app, files left open pile up and data is lost. Clean-up must run no matter what.
The old fix: try / finally
From episode 18: code in finally runs whether the try block finished normally or raised an error.
f = open("sales.txt", "w")
try:
f.write("chai,10\n")
price = int("ten")
finally:
f.close()
print("closed?", f.closed)
Output:
closed? True
Traceback (most recent call last):
File "main.py", line 4, in <module>
price = int("ten")
ValueError: invalid literal for int() with base 10: 'ten'
The file was closed, and then the error was still reported. It works, but every open now needs a try, a finally and a close, in every function. Forget one and the bug is back. We'd rather package the setup and the clean-up together, once.
Inventing context managers: __enter__ and __exit__
Think of a hotel. At check-in you get a key card (setup). Then your stay (whatever you do in the room). At check-out the card stops working, whether you had a great stay or the AC broke (clean-up, always).
| At a hotel | In Python |
|---|---|
| Check in, get the key card | __enter__() runs at the start of with; its return value goes into as |
| Your stay | the indented block under with |
| Check out, the card stops working | __exit__() runs when the block ends, even after an error |
class Shift:
def __init__(self, cashier):
self.cashier = cashier
def __enter__(self):
print(f"{self.cashier} opens the drawer")
return self
def __exit__(self, exc_type, exc, tb):
print(f"{self.cashier} locks the drawer")
print(" error was:", exc_type)
with Shift("Ravi") as s:
print("selling chai...")
with Shift("Asha") as s:
print("selling samosa...")
price = int("ten")
Output:
Ravi opens the drawer
selling chai...
Ravi locks the drawer
error was: None
Asha opens the drawer
selling samosa...
Asha locks the drawer
error was: <class 'ValueError'>
Traceback (most recent call last):
File "main.py", line 18, in <module>
price = int("ten")
ValueError: invalid literal for int() with base 10: 'ten'
__enter__returnsself, soas sreceives theShiftobject.__exit__gets three extra arguments about any error: its type (exc_type), the error object (exc) and the traceback (tb). With no error, all three areNone.- Asha's shift crashed, but the drawer was still locked, and then the error carried on as normal.
Any object with __enter__ and __exit__ is a context manager, and with is the statement that uses one. You just invented context managers.
What with open(...) does
A file object is a context manager too.
with open("sales.txt", "w") as f:
f.write(...)
open(...)makes the file object (the context manager).- Python calls its
__enter__(); the return value (the file itself) goes intof. - Your block runs.
- When the block ends, normally or because of an error,
__exit__()runs: it closes the file, which also writes the buffer to the disk.
Roughly, with is this try/finally written for you:
manager = open("sales.txt", "w")
f = manager.__enter__()
try:
f.write(...)
finally:
manager.__exit__(...) # with the error details, if there was one
The same save, with with:
def save_sales(sales):
with open("sales.txt", "w") as f:
for item, price in sales:
f.write(f"{item},{int(price)}\n")
print("closed?", f.closed)
try:
save_sales([("chai", "10"), ("samosa", "ten")])
except ValueError as e:
print("Error:", e)
print("File says:", repr(open("sales.txt").read()))
save_sales([("chai", "10"), ("samosa", "15")])
Output:
Error: invalid literal for int() with base 10: 'ten'
File says: 'chai,10\n'
closed? True
Same error, but this time chai,10 was saved, because the file was closed on the way out.
Three traps
Trap 1: using the file after the block
with open("sales.txt", "w") as f:
f.write("chai,10\n")
f.write("samosa,15\n")
Output:
Traceback (most recent call last):
File "main.py", line 4, in <module>
f.write("samosa,15\n")
~~~~~~~^^^^^^^^^^^^^^^
ValueError: I/O operation on closed file.
The variable f still exists, but the file was closed when the block ended. (I/O = input/output.) Do all file work inside the block.
Trap 2: an object that isn't a context manager
class Shop:
pass
with Shop() as s:
print("hi")
Output:
Traceback (most recent call last):
File "main.py", line 4, in <module>
with Shop() as s:
~~~~^^
TypeError: 'Shop' object does not support the context manager protocol
A protocol here means the agreed set of methods: __enter__ and __exit__. Shop has neither.
Trap 3: __exit__ returning True swallows the error
If __exit__ returns a true value, Python treats the error as handled and hides it.
class Quiet:
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
print("cleaning up")
return True
with Quiet():
total = 100 / 0
print("this never runs")
print("total is", total)
Output:
cleaning up
Traceback (most recent call last):
File "main.py", line 13, in <module>
print("total is", total)
^^^^^
NameError: name 'total' is not defined
No ZeroDivisionError at all: it was swallowed. The program fails later with a confusing NameError, because total was never set. Return True only when you really mean to ignore the error; normally return nothing (None).
The shortcut: @contextmanager
Writing a class every time is a lot. The contextlib module ("tools for context managers") has the contextmanager decorator (episode 25), used on a generator (episode 24):
- everything before
yieldis the setup (__enter__), - the value you yield goes into
as, - everything after
yieldis the clean-up (__exit__); put it infinallyso it runs even on an error (the error comes out of the function at theyieldline).
from contextlib import contextmanager
@contextmanager
def shift(cashier):
print(f"{cashier} opens the drawer")
try:
yield cashier.upper()
finally:
print(f"{cashier} locks the drawer")
with shift("Ravi") as name:
print("serving as", name)
with shift("Asha"):
int("ten")
Output:
Ravi opens the drawer
serving as RAVI
Ravi locks the drawer
Asha opens the drawer
Asha locks the drawer
Traceback (most recent call last):
File "main.py", line 15, in <module>
int("ten")
~~~^^^^^^^
ValueError: invalid literal for int() with base 10: 'ten'
A whole context manager in seven lines. Without the try/finally, Asha's drawer would not be locked.
Handy built-ins
import time
from contextlib import contextmanager, suppress
@contextmanager
def timer(label):
start = time.perf_counter()
yield
print(f"{label} took {time.perf_counter() - start:.1f}s")
with timer("closing report"):
time.sleep(0.3)
with suppress(FileNotFoundError):
open("old_bills.txt")
print("no old bills, carrying on")
Output:
closing report took 0.3s
no old bills, carrying on
timeris a context manager we wrote; a plainyield(no value) is fine whenasisn't needed.suppress(FileNotFoundError)ignores exactly that one kind of error inside its block. It's the safe version of trap 3: you name exactly which error to ignore.
Two files in one with
with open("in.txt", "w") as f:
f.write("chai\nsamosa\n")
with open("in.txt") as src, open("out.txt", "w") as dst:
for line in src:
dst.write(line.upper())
print(open("out.txt").read())
Output:
CHAI
SAMOSA
Separate the managers with a comma. Both files are closed at the end, in reverse order (dst, then src). Thread locks (episode 31), database connections and network connections all work with with too.
Common mistakes
| Mistake | What you see | Why | Fix |
|---|---|---|---|
f = open(...), crash before f.close() |
no error: the file is empty ('') and still open |
close() was skipped; the buffer was never written |
with open(...) as f: |
f.write(...) after the with block |
ValueError: I/O operation on closed file. |
with already closed it |
do the work inside the block |
with Shop() as s: |
TypeError: 'Shop' object does not support the context manager protocol |
no __enter__ / __exit__ |
add both, or use @contextmanager |
__exit__ returns True |
no error: the exception is swallowed (later NameErrors) |
True means "handled" |
return None, or use suppress(SpecificError) |
yield without try/finally in @contextmanager |
clean-up skipped when the block raises | the error comes out at yield |
try: yield ... finally: clean up |
forgetting return self in __enter__ |
as s gives None (AttributeError: 'NoneType' object has no attribute ...) |
__enter__ returned nothing |
return self |
__exit__(self) with no extra parameters |
TypeError: Shift.__exit__() takes 1 positional argument but 4 were given |
Python always passes three error arguments | def __exit__(self, exc_type, exc, tb): |
Mini project: stock rollback
An order takes milk, tea and sugar from the stock. If the sugar runs out halfway, the milk already taken must go back: all or nothing. Banks and databases call this a transaction; undoing it is a rollback. Setup copies the stock; if anything fails, clean-up puts the copy back and re-raises the error so it isn't hidden.
from contextlib import contextmanager
stock = {"milk": 5, "tea": 10, "sugar": 3}
@contextmanager
def transaction(store):
backup = store.copy()
try:
yield store
except Exception as e:
store.clear()
store.update(backup)
print("Rolled back:", e)
raise
def sell(store, item, qty):
if store[item] < qty:
raise ValueError(f"only {store[item]} {item} left")
store[item] -= qty
with transaction(stock):
sell(stock, "milk", 2)
sell(stock, "tea", 2)
print("After order 1:", stock)
try:
with transaction(stock):
sell(stock, "milk", 2)
sell(stock, "sugar", 5)
except ValueError:
pass
print("After order 2:", stock)
Output:
After order 1: {'milk': 3, 'tea': 8, 'sugar': 3}
Rolled back: only 3 sugar left
After order 2: {'milk': 3, 'tea': 8, 'sugar': 3}
Order 2 took 2 milk, then failed on sugar, and the milk came back: the stock is exactly as before. store.clear() + store.update(backup) restores the same dictionary object (so stock outside sees it); a bare raise inside except re-raises the current error.
Practice
1. A Shift class that counts sales and prints the count on exit.
Solution
class Shift:
def __init__(self, cashier):
self.cashier = cashier
self.sales = 0
def __enter__(self):
print(f"{self.cashier} starts")
return self
def sell(self, item):
self.sales += 1
print(" sold", item)
def __exit__(self, exc_type, exc, tb):
print(f"{self.cashier} ends after {self.sales} sales")
with Shift("Ravi") as s:
s.sell("chai")
s.sell("samosa")
Output:
Ravi starts
sold chai
sold samosa
Ravi ends after 2 sales
2. A @contextmanager called indent() so that nested blocks print further to the right.
Solution
from contextlib import contextmanager
level = 0
@contextmanager
def indent():
global level
level += 1
try:
yield
finally:
level -= 1
def say(text):
print(" " * level + text)
say("Menu")
with indent():
say("Drinks")
with indent():
say("chai")
say("coffee")
say("Snacks")
say("End")
Output:
Menu
Drinks
chai
coffee
Snacks
End
3. Copy a file into a backup file using a single with.
Solution
with open("bills.txt", "w") as f:
f.write("chai,10\nsamosa,15\n")
with open("bills.txt") as src, open("backup.txt", "w") as dst:
for line in src:
dst.write(line)
with open("backup.txt") as f:
print(f.read())
Output:
chai,10
samosa,15
4. Run several orders through the rollback project, and print the stock inside the failing order before the rollback.
Solution
from contextlib import contextmanager
stock = {"milk": 5, "tea": 10, "sugar": 3}
@contextmanager
def transaction(store):
backup = store.copy()
try:
yield store
except Exception as e:
print(" inside, before rollback:", store)
store.clear()
store.update(backup)
print("Rolled back:", e)
raise
def sell(store, item, qty):
if store[item] < qty:
raise ValueError(f"only {store[item]} {item} left")
store[item] -= qty
for order in [[("milk", 2)], [("tea", 4), ("sugar", 9)], [("sugar", 3)]]:
try:
with transaction(stock):
for item, qty in order:
sell(stock, item, qty)
except ValueError:
pass
print("Stock:", stock)
Output:
Stock: {'milk': 3, 'tea': 10, 'sugar': 3}
inside, before rollback: {'milk': 3, 'tea': 6, 'sugar': 3}
Rolled back: only 3 sugar left
Stock: {'milk': 3, 'tea': 10, 'sugar': 3}
Stock: {'milk': 3, 'tea': 10, 'sugar': 0}
Quick quiz
1. The video's quiz: what does this print?
class Door:
def __enter__(self):
print("open")
def __exit__(self, *err):
print("close")
with Door():
print("inside")
print("after")
Answer
open
inside
close
after
__enter__ runs first, then the block, then __exit__ as soon as the block ends, and only then the next line. (*err collects the three error arguments.)
2. What does this print?
from contextlib import contextmanager
@contextmanager
def tag(name):
print(f"<{name}>")
yield
print(f"</{name}>")
with tag("b"):
print("chai")
Answer
<b>
chai
</b>
3. If the block inside with open(...) as f: raises an error, is the file closed?
Answer
Yes. __exit__ always runs, so the file is closed and the buffer is written; then the error carries on (a file's __exit__ doesn't swallow it).
Interview questions
- What is a context manager? An object that defines
__enter__and__exit__, used withwithto guarantee setup and clean-up around a block of code. - What does the
withstatement do? Calls__enter__, binds its result to theasname, runs the block, then always calls__exit__with the exception details (orNones), like a built-intry/finally. - What are
__exit__'s parameters?exc_type,exc_value,traceback: allNoneif the block finished normally. - How can a context manager suppress an exception? By returning a true value from
__exit__. Use this deliberately and narrowly, e.g.contextlib.suppress(FileNotFoundError). - How does
@contextmanagerwork? It turns a generator into a context manager: code beforeyieldis setup, the yielded value is bound byas, and code afteryieldis clean-up. Exceptions in the block are re-raised at theyield, so wrap it intry/finally. - Why is
with open(...)better thanopen()+close()? The file is always closed (and flushed) even if an error occurs, with less code and no way to forget. - Name some real context managers. Files,
threading.Lock, database connections/transactions,decimal.localcontext(),tempfile.TemporaryDirectory(),unittest.mock.patch,contextlib.suppress,contextlib.redirect_stdout. - Can one
withmanage several resources? Yes:with a() as x, b() as y:; they are exited in reverse order. For a variable number, usecontextlib.ExitStack. - What is a transaction? A group of changes that happens completely or not at all; on failure, the changes are rolled back.
Glossary
| Term | Meaning | Example |
|---|---|---|
| Context manager | An object with __enter__ and __exit__, used with with |
a file object |
with statement |
Runs setup, the block, then always the clean-up | with open("a.txt") as f: |
__enter__ |
Setup; its return value goes into as |
return self |
__exit__ |
Clean-up; always runs when the block ends | def __exit__(self, exc_type, exc, tb): |
as |
Names the value __enter__ returned |
as f |
| Clean-up | Code that releases a resource: close, unlock, disconnect | f.close() |
| Resource | Something that must be given back: a file, lock, connection | an open file |
| Buffer | A waiting area in memory where written text sits until it's saved to disk | text before close() |
try / finally |
finally runs whether or not there was an error |
finally: f.close() |
f.closed |
True if the file has been closed |
print(f.closed) |
exc_type, exc, tb |
The error's type, the error object, the traceback (None if no error) |
<class 'ValueError'> |
| Traceback | The report of where an error happened | Traceback (most recent call last): |
| Context manager protocol | The rule that an object must have __enter__ and __exit__ |
'Shop' object does not support... |
| Swallowing an error | Hiding an exception so it's never reported | __exit__ returning True |
contextlib |
Standard-library module of context-manager tools | from contextlib import contextmanager |
@contextmanager |
Turns a generator (setup, yield, clean-up) into a context manager |
@contextmanager def timer(...) |
suppress() |
Ignores one named kind of error inside its block | with suppress(FileNotFoundError): |
FileNotFoundError |
The error for opening a file that doesn't exist | open("old_bills.txt") |
| I/O | Input/output: reading and writing files, the screen, the network | I/O operation on closed file. |
repr() |
Shows a value as code, e.g. text with its quotes | repr("") → "''" |
| Transaction | A group of changes that happens completely or not at all | an order |
| Rollback | Undoing a failed transaction's changes | restoring the stock backup |
| Re-raise | Raising the same error again after handling it partly | bare raise in except |
Summary
- An error can skip
f.close(): the file stays open and the buffer is lost (''). try/finallyfixes it but is easy to forget; a context manager packages setup and clean-up together.__enter__sets up (its return value goes intoas);__exit__cleans up and always runs, receiving the error details.with open(...)closes the file for you, even after a crash.- Traps: using a file after the block, objects without the protocol, and
__exit__returningTrue(swallowed errors). @contextmanager: setup,yield, clean-up infinally.suppress(SpecificError)ignores one named error; onewithcan manage several resources.- A context manager can make changes all or nothing (a transaction with rollback).
Next episode: a price arrives as text and two samosas cost 1,515 rupees. We learn type hints, a type checker, and dataclasses that write __init__ for you.