SyntaxPilot
Courses/Python/Lesson 28

Python · Lesson 28 of 40 · 11 min

Regular expressions

queued

These are the notes for Python episode 28. Raju's tea shop gets hundreds of customer messages, and he wants every phone number in them. But people write numbers every way: 9876543210, 98765-43210, +91 98765 43210. String methods like find(), in and split() look for exact text, and a phone number is not exact text: it is a shape (ten digits, maybe with a space or a dash). In this episode you invent regular expressions: a tiny language for describing the shape of text, so Python can find it, check it and change it. Every example below runs as-is in the SyntaxPilot compiler, with its real output from Python 3.13.

The problem: find every phone number

msgs = ["Call me on 9876543210 for 2 chai",
        "My number: 98765-43210, order 4521",
        "+91 98765 43210 please, table 7"]
for m in msgs:
    for word in m.split():
        if word.isdigit() and len(word) == 10:
            print("Found:", word)

Output:

Found: 9876543210

One number out of three. 98765-43210, has a dash and a comma stuck to it, so isdigit() is False; 98765 43210 is split into two words of five digits. We could add more ifs (strip commas, join pairs of words, check for +91), but every new way of writing a number means more code.

Why it fails: string methods match exact text. We don't know the number; we only know what it looks like.

The idea: describe the shape

On a paper form, the phone field is ten little boxes, one digit in each. The form doesn't contain a number; it describes its shape. A regular expression (regex, or pattern) is a string that describes a shape of text:

On the form In a regex Meaning
one box for a digit \d any one digit, 0 to 9
"10 boxes" \d{10} exactly ten digits
a clerk scanning the page re.findall(pattern, text) every match, as a list

Python's regex tools are in the standard-library module re (episode 19: modules).

import re
text = "Call 9876543210 now. Order id 123456789012. Old: 98765-43210"
print(re.findall(r"\d{10}", text))

Output:

['9876543210', '1234567890']

The phone number is found, but so is 1234567890: the first ten digits of the 12-digit order id. A pattern matches anywhere in the text, including inside a longer number.

Word boundaries \b and raw strings r"..."

\b is a word boundary: the position between a "word character" (letter, digit or _) and anything else (a space, a dot, a comma, the start or end of the text). It doesn't match a character; it matches a position. So \b\d{10}\b means "exactly ten digits, with nothing word-like stuck to either side".

The trap: in a normal Python string, \b already means the backspace character, and Python converts it before re sees the pattern. The fix is a raw string: an r before the quotes. In a raw string, backslashes stay as they are. Always write patterns as raw strings.

import re
text = "Call 9876543210 now. Order id 123456789012. Old: 98765-43210"
print(re.findall(r"\b\d{10}\b", text))
print(re.findall("\b\d{10}\b", text))

Output:

<string>:4: SyntaxWarning: invalid escape sequence '\d'
['9876543210']
[]

With r: only the real phone number. Without r: an empty list, and Python warns invalid escape sequence '\d'. The pattern was quietly broken: \b became a backspace character, which never appears in the text.

Character classes and quantifiers

Pattern Matches Example
\d one digit 7
\w one letter, digit or _ a, Z, 5, _
\s one whitespace character (space, tab, newline) " "
. any one character except a newline x, -, @
[ -] one character: a space or a dash (a character class) " ", -
[6-9] one character in the range 6 to 9 6, 9
[a-z] one lowercase letter q
[^0-9] one character that is not a digit (^ inside [ ] = not) a
\D, \W, \S the opposite of \d, \w, \s

Quantifiers say how many times the thing just before them repeats:

Quantifier Meaning
? 0 or 1 times (optional)
+ 1 or more times
* 0 or more times
{5} exactly 5 times
{2,4} from 2 to 4 times
{2,} 2 or more times

To match a special character literally, put a backslash before it: \. is a real dot, \+ a real plus, \( a real bracket.

A phone number written three ways is: five digits, an optional space or dash, five digits, boundaries on both sides:

import re
text = "Call 9876543210. Old: 98765-43210. Also +91 98765 43210"
pattern = r"\b\d{5}[ -]?\d{5}\b"
print(re.findall(pattern, text))

Output:

['9876543210', '98765-43210', '98765 43210']

All three found, with one line of pattern instead of a pile of if statements. The +91 is not part of the match because the pattern only describes the ten digits.

Finding vs checking: search, match, fullmatch

Function Question it answers Returns
re.findall(p, s) every match in s a list of strings (or tuples, with groups)
re.search(p, s) is there a match anywhere in s? the first match object, or None
re.match(p, s) does s start with a match? a match object, or None
re.fullmatch(p, s) does the whole of s match? a match object, or None

A match object holds what matched and where (span = start and end positions). It counts as True in an if; None counts as False.

import re
for pin in ["110001", "1100", "110001abc"]:
    print(pin, re.search(r"\d{6}", pin), re.fullmatch(r"\d{6}", pin))

Output:

110001 <re.Match object; span=(0, 6), match='110001'> <re.Match object; span=(0, 6), match='110001'>
1100 None None
110001abc <re.Match object; span=(0, 6), match='110001'> None

For 110001abc, search says "yes, I found six digits inside", but the input is not a valid PIN code. To check user input, use fullmatch.

re.match only looks at the start:

import re
print(re.match(r"\d+", "chai 10"), re.search(r"\d+", "chai 10"))

Output:

None <re.Match object; span=(5, 7), match='10'>

Indian mobile numbers start with 6, 7, 8 or 9:

import re
for p in ["9876543210", "98765 43210", "12345", "5876543210"]:
    ok = re.fullmatch(r"[6-9]\d{4} ?\d{5}", p)
    print(p, "valid" if ok else "invalid")

Output:

9876543210 valid
98765 43210 valid
12345 invalid
5876543210 invalid

Groups: taking parts out

Round brackets ( ) make a group: a part of the match you can take out on its own. Groups are numbered from the left, starting at 1; group() (or group(0)) is the whole match.

import re
msg = "Mail the bill to raju.chai@gmail.com today"
m = re.search(r"([\w.]+)@([\w.]+)", msg)
print(m.group())
print(m.group(1))
print(m.group(2))

Output:

raju.chai@gmail.com
raju.chai
gmail.com

When there is no match, search returns None, and None has no groups:

import re
m = re.search(r"([\w.]+)@([\w.]+)", "no email here")
print(m.group(1))

Output:

Traceback (most recent call last):
  File "main.py", line 3, in <module>
    print(m.group(1))
          ^^^^^^^
AttributeError: 'NoneType' object has no attribute 'group'

Always check the match first:

import re
m = re.search(r"([\w.]+)@([\w.]+)", "no email here")
if m:
    print(m.group(1))
else:
    print("no email found")

Output:

no email found

Named groups make long patterns readable: (?P<name>...).

import re
m = re.search(r"(?P<day>\d{2})-(?P<month>\d{2})-(?P<year>\d{4})", "Order on 04-11-2026 at 7 pm")
print(m.group("day"), m.group("month"), m.group("year"))
print(m.groupdict())

Output:

04 11 2026
{'day': '04', 'month': '11', 'year': '2026'}

With groups, findall returns the groups instead of the whole match: one string per match for one group, a tuple per match for two or more.

Greedy .* vs lazy .*?

import re
html = "<b>chai</b> and <b>samosa</b>"
print(re.findall(r"<b>(.*)</b>", html))
print(re.findall(r"<b>(.*?)</b>", html))

Output:

['chai</b> and <b>samosa']
['chai', 'samosa']

*, + and ? are greedy: they take as many characters as they can and still match, so .* runs to the last </b>. Adding ? after them (*?, +?) makes them lazy: as few characters as possible.

re.sub: replacing matches

re.sub(pattern, replacement, text) replaces every match. In the replacement, \1, \2 put back what groups 1 and 2 matched (a back-reference). Raju masks phone numbers before sharing messages with his staff:

import re
text = "Raju: 9876543210, Asha: 98765-43210"
print(re.sub(r"\b(\d{2})\d{3}[ -]?\d{3}(\d{2})\b", r"\1XXXXXX\2", text))

Output:

Raju: 98XXXXXX10, Asha: 98XXXXXX10

Other useful tools:

import re
print(re.split(r"[,;]\s*", "chai, samosa;vada pav,  poha"))
pat = re.compile(r"\b[6-9]\d{9}\b")
print(pat.findall("9876543210 and 5123456789 and 7000000001"))
print(re.findall(r"^\w+", "chai\nsamosa", re.MULTILINE))

Output:

['chai', 'samosa', 'vada pav', 'poha']
['9876543210', '7000000001']
['chai', 'samosa']
  • re.split splits on a pattern (here: a comma or semicolon, then any spaces).
  • re.compile turns a pattern into a reusable object, handy when you use it many times.
  • ^ means "start of the text" and $ "end of the text"; with re.MULTILINE they mean start and end of each line.
  • Flags: re.IGNORECASE (re.I) ignores upper/lower case; re.MULTILINE (re.M); re.DOTALL lets . match newlines too.

A broken pattern

import re
print(re.findall(r"(\d+", "2 chai"))

Output (last line):

re.PatternError: missing ), unterminated subpattern at position 0

Python checks the pattern itself before searching. position 0 is where the unclosed ( is. (Before Python 3.13 this error was called re.error; the old name still works.)

Common mistakes

Mistake What you see Why Fix
re.findall("\b\d{10}\b", text) SyntaxWarning: invalid escape sequence '\d' and [] \b became a backspace raw string: r"\b\d{10}\b"
re.findall(r"\d{10}", text) no error: '1234567890' from a 12-digit id no boundaries r"\b\d{10}\b"
re.search(r"\d{6}", "110001abc") to check input no error: a match for bad input search looks anywhere re.fullmatch(r"\d{6}", s)
re.search(...).group(1) with no match AttributeError: 'NoneType' object has no attribute 'group' no match gives None if m: m.group(1)
r"(\d+" re.PatternError: missing ), unterminated subpattern at position 0 unclosed group close it: r"(\d+)"
r"<b>(.*)</b>" one huge match across both tags * is greedy lazy: .*?
r"raju.chai@gmail.com" also matches rajuXchai@gmailYcom . means any character escape it: \.

Mini project: an order message into a bill

Customers order by message: "2 chai, 3 Samosa, 1 vada pav". The pattern (\d+) ([a-z ]+) is: a group of one or more digits, a space, then a group of letters and spaces. A comma is not a letter, so each item stops at the comma. re.IGNORECASE makes Samosa match too. With two groups, findall gives a list of (qty, item) pairs.

import re
menu = {"chai": 10, "samosa": 15, "vada pav": 20}
order = "2 chai, 3 Samosa, 1 vada pav"
total = 0
for qty, item in re.findall(r"(\d+) ([a-z ]+)", order, re.IGNORECASE):
    item = item.lower()
    cost = int(qty) * menu[item]
    total += cost
    print(qty, "x", item, "= Rs", cost)
print("Total: Rs", total)

Output:

2 x chai = Rs 20
3 x samosa = Rs 45
1 x vada pav = Rs 20
Total: Rs 85

Try it: what happens if a customer orders coffee? (Solution in Practice 5.)

Practice

1. Find every email address in a paragraph.

Solution
import re
text = "Write to raju.chai@gmail.com or orders@rajustall.in, not to raju@"
print(re.findall(r"[\w.]+@[\w]+\.[\w.]+", text))

Output:

['raju.chai@gmail.com', 'orders@rajustall.in']

raju@ is skipped: after the @ the pattern needs a word, a real dot, and more. Real-world email rules are much more complex; this pattern is good enough for finding emails in messages.

2. Check that an OTP is exactly six digits.

Solution
import re
for otp in ["482913", "48291", "4829134", "48a913"]:
    print(otp, "OK" if re.fullmatch(r"\d{6}", otp) else "invalid")

Output:

482913 OK
48291 invalid
4829134 invalid
48a913 invalid

3. Find all prices written like Rs 120 and add them up.

Solution
import re
msg = "Chai Rs 10, samosa Rs 15 and a thali for Rs 120"
prices = [int(p) for p in re.findall(r"Rs (\d+)", msg)]
print(prices, "total:", sum(prices))

Output:

[10, 15, 120] total: 145

4. Mask every email so that only its first letter shows: r***@gmail.com.

Solution
import re
text = "Mail raju.chai@gmail.com or neha@yahoo.in"
print(re.sub(r"\b(\w)[\w.]*@", r"\1***@", text))

Output:

Mail r***@gmail.com or n***@yahoo.in

5. Make the mini project print "not on the menu" instead of crashing with KeyError for an unknown item.

Solution
import re
menu = {"chai": 10, "samosa": 15, "vada pav": 20}
order = "2 chai, 1 coffee, 3 samosa"
total = 0
for qty, item in re.findall(r"(\d+) ([a-z ]+)", order, re.IGNORECASE):
    item = item.lower()
    if item not in menu:
        print(item, "is not on the menu")
        continue
    cost = int(qty) * menu[item]
    total += cost
    print(qty, "x", item, "= Rs", cost)
print("Total: Rs", total)

Output:

2 x chai = Rs 20
coffee is not on the menu
3 x samosa = Rs 45
Total: Rs 65

Quick quiz

import re
print(re.findall(r"\b\d{3}\b", "pin 560 001 and 1234"))

A. ['560', '001', '123'] B. ['560', '001'] C. ['560001'] D. []

Answer

B. \b needs the three digits to stand alone. In 1234 there is another digit after the third one, so there is no boundary there and it is skipped.

Interview questions

  1. What is the difference between re.search, re.match and re.fullmatch? search finds the first match anywhere; match only at the start of the string; fullmatch only if the whole string matches. Use fullmatch to validate input.
  2. Why use raw strings for patterns? Backslashes are escape characters in normal strings ("\b" is a backspace, "\n" a newline). In r"..." they reach the regex engine unchanged.
  3. What does findall return when the pattern has groups? The groups, not the whole match: a list of strings for one group, a list of tuples for several.
  4. Greedy vs lazy? .* takes as much as possible and backs off only as far as needed; .*? takes as little as possible.
  5. What does \b match? A position between a word character (\w) and a non-word character or the start/end of the string. It consumes no characters.
  6. When should you not use a regex? For a fixed piece of text (in, startswith, replace are simpler and faster), and for nested formats like HTML or JSON (use a real parser such as json, episode 20).

Glossary

Term Meaning
regular expression (regex) a string that describes a shape of text, used to find, check or change text
pattern the regex string itself, e.g. r"\d{10}"
re module Python's standard-library module for regular expressions
match a piece of text that fits the pattern
match object what search/match/fullmatch return: the matched text, its span and its groups
span the start and end positions of a match
character class [...]: one character from a set, e.g. [ -] or [6-9]
range a-z or 6-9 inside a class: every character between the two
quantifier ?, +, *, {n}, {m,n}: how many times the previous item repeats
\d, \w, \s a digit; a letter, digit or underscore; a whitespace character
. any one character except a newline
word boundary \b the position between a word character and a non-word character
anchor ^ (start) or $ (end): matches a position, not a character
raw string r"...": a string where backslashes are kept as they are
escape sequence a backslash code in a normal string, e.g. \n (newline), \b (backspace)
escape (in a regex) \. \+ \(: a backslash that makes a special character literal
group ( ... ): a part of the pattern whose text you can take out
named group (?P<name> ... ): a group you take out by name
back-reference \1, \2 in a replacement: the text group 1 or 2 matched
greedy a quantifier that takes as much text as possible
lazy a quantifier with ? after it (*?): as little text as possible
flag an option such as re.IGNORECASE or re.MULTILINE
re.compile turns a pattern into a reusable pattern object
re.PatternError the error raised for an invalid pattern (called re.error before Python 3.13)
validation checking that input has the right form before using it

Summary

  • String methods match exact text; a regex matches a shape.
  • Building blocks: \d \w \s ., classes [6-9] [ -], quantifiers ? + * {n} {m,n}, anchors \b ^ $.
  • Always write patterns as raw strings: r"...".
  • findall = every match; search = first match anywhere; fullmatch = check the whole input.
  • ( ) groups take parts out; check for None before calling .group().
  • .* is greedy, .*? is lazy; re.sub replaces, with \1 back-references.
  • Next episode: Raju's code keeps growing and every change breaks something. You'll invent automated tests with unittest and pytest.