SyntaxPilot
Courses/System Design/Lesson 3
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

System Design · Lesson 3 of 40

DNS deep dive

scheduled (upload queued)

These are the notes for System Design episode 3. In episode 2 we built client–server and HTTP, and quietly assumed the client already knows the server's address. Here you find out how it gets that address, by inventing DNS yourself, one failure at a time: a shared file that goes stale, a central server that crashes, the root → TLD → authoritative split, a recursive resolver with a cache, and TTL. Then the record types (A, AAAA, CNAME, MX, NS, TXT) and the day Facebook vanished from DNS. Every Python demo below is self-contained and runs as-is; the outputs shown are real (timings vary by a millisecond or two per run).

The 203.0.113.x addresses in these notes are the documentation range (RFC 5737), not real servers.

The six problems, and the fix for each

Problem Fix (a piece of DNS)
People remember names, computers need numbers A lookup: name → IP address
One shared file goes stale Ask a server, don't copy
One server crashes Split the job: root, TLD, authoritative
Three trips every time A resolver with a cache
The cache goes stale TTL
DNS disappears Design DNS like the rest of your system

Problem 1: names for people, numbers for machines

You don't call your mum by typing ten digits. You tap Mummy and the phone dials the number.

On your phone On the internet
"Mummy": the name you remember Domain name: syntaxpilot.in
98XXXXXX01: the number the network dials IP address: 203.0.113.10
The contacts app turns the name into the number DNS (Domain Name System): name → IP address

Your browser can only connect to a number. Something has to turn the name into the number, and that is DNS.

Demo 1: ask for a lookup from Python

import socket

print("localhost ->", socket.gethostbyname("localhost"))
print("syntaxpilot.inn ->", socket.gethostbyname("syntaxpilot.inn"))

Output:

localhost -> 127.0.0.1
Traceback (most recent call last):
  File "main.py", line 4, in <module>
    print("syntaxpilot.inn ->", socket.gethostbyname("syntaxpilot.inn"))
                                ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known
  • socket.gethostbyname takes a name and returns the IPv4 address.
  • localhost (the name every computer uses for itself) comes back as 127.0.0.1. It is answered by a tiny file on your own computer: the hosts file (/etc/hosts on Linux and macOS).
  • The typo syntaxpilot.inn crashes with socket.gaierror: Name or service not known: no phone book anywhere has that name. You will see this error every time a domain is misspelled.

Problem 2: one shared file goes stale (HOSTS.TXT)

The obvious answer: one big file with every name and its number. Every computer downloads a copy and looks names up locally.

This really happened. The early internet used a file called HOSTS.TXT, kept on one computer at the SRI Network Information Center in California and copied to every host on the network. Every new computer made the file bigger, and every change meant everyone had to download it again. The DNS designers wrote that the cost of distributing it grew much faster than the number of hosts.

Demo 2: a copied phone book goes stale

# HOSTS.TXT: one master file, copied to every computer
master = {"syntaxpilot.in": "203.0.113.10",
          "irctc.co.in":    "203.0.113.20"}
my_copy = dict(master)                     # downloaded last week

master["syntaxpilot.in"] = "203.0.113.99"  # a site moves
master["swiggy.com"] = "203.0.113.30"      # a new site appears

print("syntaxpilot.in ->", my_copy["syntaxpilot.in"], "(old!)")
print("swiggy.com     ->", my_copy["swiggy.com"])

Output:

syntaxpilot.in -> 203.0.113.10 (old!)
Traceback (most recent call last):
  File "main.py", line 10, in <module>
    print("swiggy.com     ->", my_copy["swiggy.com"])
                               ~~~~~~~^^^^^^^^^^^^^^
KeyError: 'swiggy.com'

Two failures: the copy still has the old address for syntaxpilot.in (the browser would knock on an empty server), and it has never heard of swiggy.com. Copies go stale the moment the original changes.

Don't send copies; let people ask.

Problem 3: one central server is a single point of failure

Fine, no copies: put the phone book on one server and everyone asks it, like one enquiry counter for the whole country. It is always correct, because there is only one copy. But every phone, laptop and app on the planet asks this one machine every second, and if it goes down, nobody can find anything.

Demo 3: the only server goes down

import socket, threading

book = {"syntaxpilot.in": "203.0.113.10"}

def central_server(sock):
    conn, _ = sock.accept()
    name = conn.recv(100).decode()
    conn.send(book.get(name, "unknown").encode())
    conn.close()
    sock.close()                 # the one and only server goes down

sock = socket.socket()
sock.bind(("127.0.0.1", 8101)); sock.listen()
t = threading.Thread(target=central_server, args=(sock,))
t.start()

def ask(name):
    c = socket.create_connection(("127.0.0.1", 8101))
    c.send(name.encode())
    return c.recv(100).decode()

print("Person 1:", ask("syntaxpilot.in"))
t.join()
print("Person 2:", ask("syntaxpilot.in"))

Output:

Person 1: 203.0.113.10
Traceback (most recent call last):
  File "main.py", line 24, in <module>
    print("Person 2:", ask("syntaxpilot.in"))
                       ~~~^^^^^^^^^^^^^^^^^^
  File "main.py", line 18, in ask
    c = socket.create_connection(("127.0.0.1", 8101))
  ...
ConnectionRefusedError: [Errno 111] Connection refused
  • The server thread listens on port 8101, answers one question, then closes its socket, like a machine on a bad day.
  • Person 1 gets the address. Person 2 gets ConnectionRefusedError: the phone book is fine, but the only place it lives is gone.

This is a single point of failure (SPOF). One machine can't carry the whole world, and it must never be the only one.

The fix: split the job (root, TLD, authoritative)

How do you find a building in a city you've never visited? Nobody knows every address in Mumbai, but you still get there:

New to Mumbai In DNS What it knows
Station enquiry: "Andheri? Take the western line" Root server Only the big direction: "ask the .in servers"
Andheri auto stand: "Lokhandwala is that side" TLD server (top-level domain, e.g. .in) The area: "ask ns1.syntaxpilot.in"
Building watchman: "Flat 4B, third floor" Authoritative server The exact answer: 203.0.113.10
Your local friend who does all the asking Recursive resolver (your ISP, 8.8.8.8, 1.1.1.1) Asks on your behalf and remembers

Read a domain from right to left

root "."  ->  .in (TLD server)  ->  syntaxpilot.in (authoritative)
  • Root: knows who runs each TLD (.in, .com, .org).
  • TLD: knows which name server owns each domain.
  • Authoritative: knows the final answer (the IP address).

No single machine holds the whole phone book, and each level can be run by different people. That also fixes the crash problem.

The real root: there are 13 root server identities, named a to m, run by 12 independent organisations. Each identity is not one machine but is copied to many places: according to root-servers.org there were about 2,000 copies around the world in August 2026, answering roughly 1.6 million questions every second.

Demo 4: simulate root → TLD → authoritative

import time

ROOT = {"in": "TLD .in", "com": "TLD .com"}
TLD  = {"TLD .in":  {"syntaxpilot.in": "ns1.syntaxpilot.in"},
        "TLD .com": {"swiggy.com": "ns1.swiggy.com"}}
AUTH = {"ns1.syntaxpilot.in": {"syntaxpilot.in": "203.0.113.10"},
        "ns1.swiggy.com":     {"swiggy.com": "203.0.113.30"}}

def ask(server, answer):
    time.sleep(0.03)                       # ~30 ms per trip
    print(f"  ask {server:<19} -> {answer}")
    return answer

def resolve(name):
    start = time.perf_counter()
    tld = ask("root", ROOT[name.split(".")[-1]])
    ns  = ask(tld, TLD[tld][name])
    ip  = ask(ns, AUTH[ns][name])
    ms = (time.perf_counter() - start) * 1000
    print(f"{name} = {ip}  ({ms:.0f} ms)")
    return ip

resolve("syntaxpilot.in")
resolve("swiggy.com")
resolve("syntaxpilot.in")

Output:

  ask root                -> TLD .in
  ask TLD .in             -> ns1.syntaxpilot.in
  ask ns1.syntaxpilot.in  -> 203.0.113.10
syntaxpilot.in = 203.0.113.10  (91 ms)
  ask root                -> TLD .com
  ask TLD .com            -> ns1.swiggy.com
  ask ns1.swiggy.com      -> 203.0.113.30
swiggy.com = 203.0.113.30  (91 ms)
  ask root                -> TLD .in
  ask TLD .in             -> ns1.syntaxpilot.in
  ask ns1.syntaxpilot.in  -> 203.0.113.10
syntaxpilot.in = 203.0.113.10  (90 ms)
  • name.split(".")[-1] takes the last part (in, com): reading right to left.
  • Every question is a trip across the network, so ask sleeps 30 ms; each lookup takes about 90 ms.

It works, but look at the last lookup: the exact same question again costs three trips and 90 ms. A page with fifty images and scripts could do this dozens of times.

Problem 4: three trips every time → a resolver with a cache

Your local friend does the running around once, and the next time anyone asks, they just remember.

  • That friend is the recursive resolver, usually run by your internet provider, or a public one like Google's 8.8.8.8 or Cloudflare's 1.1.1.1. Your phone asks it one question and it walks the chain for you.
  • It keeps a cache: a notebook of recent answers. Your browser and operating system keep small caches too. The second time, there are zero trips.

Demo 5: a cache makes it fast (and wrong forever)

import time

AUTH = {"syntaxpilot.in": "203.0.113.10"}  # the site's own DNS server

def resolve(name):                         # root -> TLD -> authoritative
    time.sleep(0.09)                       # 3 trips x ~30 ms
    return AUTH[name]

cache = {}
def lookup(name):
    if name in cache:
        print(f"{name} = {cache[name]}  cache hit, 0 ms")
        return
    start = time.perf_counter()
    ip = resolve(name)
    cache[name] = ip                       # keep it... forever?
    ms = (time.perf_counter() - start) * 1000
    print(f"{name} = {ip}  asked the servers, {ms:.0f} ms")

lookup("syntaxpilot.in")
lookup("syntaxpilot.in")
AUTH["syntaxpilot.in"] = "203.0.113.99"    # the site moves servers
lookup("syntaxpilot.in")                   # still the old address!

Output:

syntaxpilot.in = 203.0.113.10  asked the servers, 90 ms
syntaxpilot.in = 203.0.113.10  cache hit, 0 ms
syntaxpilot.in = 203.0.113.10  cache hit, 0 ms

First lookup: 90 ms. Second: a cache hit, 0 ms. But the third is a cache hit too, with the old address. The site moved, and this cache will hand out the wrong address forever. We fixed speed and brought back the stale-copy problem from HOSTS.TXT.

Problem 5: the cache goes stale → TTL

Every milk packet says use within 2 days. The dairy decides how long it stays good; you keep it in the fridge until then, and buy fresh after.

Milk packet DNS record
"Use within 2 days", printed by the dairy TTL (time to live), set by the site's own DNS server
Keep it in the fridge until the date The cached answer is used until the TTL runs out
Expired? Buy fresh Ask again: walk the chain once more

RFC 1035, the DNS rule book from November 1987, defines TTL as the number of seconds a record may be cached before it should be discarded. The site owner picks the number: a long TTL means fewer trips; a short TTL means changes spread faster.

Demo 6: a cache with TTL

import time

AUTH = {"syntaxpilot.in": "203.0.113.10"}  # the site's own DNS server

def resolve(name):                         # root -> TLD -> authoritative
    time.sleep(0.09)                       # 3 trips x ~30 ms
    return AUTH[name]

cache = {}
def lookup(name, ttl=2):
    if name in cache and time.time() < cache[name][1]:
        print(f"{name} = {cache[name][0]}  cache hit, 0 ms")
        return
    start = time.perf_counter()
    ip = resolve(name)
    cache[name] = (ip, time.time() + ttl)  # keep it for ttl seconds
    ms = (time.perf_counter() - start) * 1000
    print(f"{name} = {ip}  asked the servers, {ms:.0f} ms")

lookup("syntaxpilot.in")
lookup("syntaxpilot.in")
AUTH["syntaxpilot.in"] = "203.0.113.99"    # the site moves servers
lookup("syntaxpilot.in")                   # still the old address!
time.sleep(2.1)                            # wait for the TTL to run out
lookup("syntaxpilot.in")

Output:

syntaxpilot.in = 203.0.113.10  asked the servers, 91 ms
syntaxpilot.in = 203.0.113.10  cache hit, 0 ms
syntaxpilot.in = 203.0.113.10  cache hit, 0 ms
syntaxpilot.in = 203.0.113.99  asked the servers, 92 ms
  • The cache now stores (ip, expiry time), where expiry = now + ttl.
  • A hit only counts if time.time() is still before the expiry.
  • Inside the TTL we still get the old address: that is the price of caching, and why DNS changes take time to reach everyone.
  • After the TTL, the cache misses, we ask again and get the new address 203.0.113.99. Stale for at most the TTL, never forever.

Choosing a TTL

TTL Good Bad
Long (1 day) Fewer lookups, faster pages A change can take up to a day to reach everyone
Short (60 s) Changes spread fast Resolvers must ask much more often

The engineer's trick: before moving a site to a new server, lower the TTL a day early. Then switch the IP; everyone picks it up within minutes. Then raise the TTL again.

DNS record types

Record What it says Example
A name → IPv4 address syntaxpilot.in → 203.0.113.10
AAAA name → IPv6 address (the longer kind)
CNAME this name is just another name www.syntaxpilot.in → syntaxpilot.in
MX which server receives this domain's email
NS which name servers are authoritative (the arrows in the chain) handed out by the TLD server
TXT free text often used to prove to Google or others that you own a domain

One lookup, end to end

Browser cache -> OS (hosts file, cache) -> Recursive resolver (cache?) -> Root -> TLD -> Authoritative (only on a miss)
  1. The browser checks its own cache, then the operating system: the hosts file, then its cache.
  2. The recursive resolver: a cache hit? Done in a few milliseconds. This is what happens most of the time.
  3. Only on a miss does it walk root → TLD → authoritative.
  4. The answer is cached at each step for its TTL.

DNS messages usually travel as small UDP messages on port 53, not over HTTP.

When DNS disappears: Facebook, 4 October 2021

  • At 15:39 UTC on 4 October 2021, a maintenance command at Facebook accidentally disconnected all of its data centres from the internet.
  • Facebook's own DNS servers were built to withdraw their routes when they lost the data centres, so nobody in the world could reach Facebook's authoritative name servers.
  • Facebook, Instagram, WhatsApp and Messenger were down for six to seven hours. The servers were healthy, but phones couldn't even find the address to connect to.

The lesson: DNS is not someone else's problem. It is part of your system and needs the same care: many servers, many places, no single path.

Common mistakes (and the real errors)

Mistake Error Why Fix
socket.gethostbyname("syntaxpilot.inn") socket.gaierror: [Errno -2] Name or service not known No DNS server has that name: a typo, or the domain doesn't exist Check the spelling; test with nslookup or dig
my_copy["swiggy.com"] KeyError: 'swiggy.com' A copied phone book only knows what it had when you copied it Ask a server; don't rely on a local copy
socket.create_connection(("127.0.0.1", 8101)) ConnectionRefusedError: [Errno 111] Connection refused Nothing is listening there: the server is down Start the server, or have more than one
"I changed my DNS, why do I still see the old site?" no error: an old cached answer Resolvers keep the old record until its TTL runs out Wait out the TTL; lower it before the next change

Re-running Demo 3 quickly

If you run the central-server demo twice in a row, the second run may fail at bind with:

OSError: [Errno 98] Address already in use

The operating system keeps the just-closed port reserved for a short while. Wait a minute, use another port, or allow reuse before binding:

sock = socket.socket()
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sock.bind(("127.0.0.1", 8101)); sock.listen()

Practice

  1. In Demo 6, change ttl=2 to ttl=5 but keep time.sleep(2.1). Predict all four lines, then run it.
  2. Add a cache to Demo 4's resolve so the third lookup is instant.
  3. Your A record has TTL 300. A resolver cached it at 9:58. You change the IP at 10:00. Until when can that resolver serve the old IP?
  4. Which record type would you add for each: (a) email for your domain, (b) www pointing to your main name, (c) an IPv6 address, (d) proving domain ownership to a search console?
  5. Run nslookup syntaxpilot.in (or dig syntaxpilot.in) on your computer. Find the TTL in dig's answer, run it again, and watch it drop.
Solutions and hints

1. The TTL hasn't run out after 2.1 seconds, so the last lookup is still a cache hit with the old address:

syntaxpilot.in = 203.0.113.10  asked the servers, 90 ms
syntaxpilot.in = 203.0.113.10  cache hit, 0 ms
syntaxpilot.in = 203.0.113.10  cache hit, 0 ms
syntaxpilot.in = 203.0.113.10  cache hit, 0 ms

2. Keep the same ROOT, TLD, AUTH and ask as Demo 4, and replace resolve:

cache = {}
def resolve(name):
    start = time.perf_counter()
    if name in cache:
        ip = cache[name]
    else:
        tld = ask("root", ROOT[name.split(".")[-1]])
        ns  = ask(tld, TLD[tld][name])
        ip  = ask(ns, AUTH[ns][name])
        cache[name] = ip
    ms = (time.perf_counter() - start) * 1000
    print(f"{name} = {ip}  ({ms:.0f} ms)")
    return ip

The first two lookups print the same three ask lines as before; the third prints only:

syntaxpilot.in = 203.0.113.10  (0 ms)

(This cache has no TTL yet, so it has the stale problem from Demo 5.)

3. Until 10:03. 300 seconds = 5 minutes, counted from when it was cached (9:58), not from when you changed it.

4. (a) MX, (b) CNAME, (c) AAAA, (d) TXT.

5. The number in dig's answer section next to the record is the remaining TTL in seconds. Your resolver serves it from cache, so it counts down between runs. (Your exact numbers will differ.)

Quick quiz

Your A record has TTL = 3600. At 10:00 a resolver caches it. At 10:05 you move the site to a new IP. Until when can that resolver give out the old IP?

  • A. 10:05, the moment you change it
  • B. 10:06, one minute later
  • C. 11:00, when its TTL runs out
  • D. Forever, until you restart it
Answer

C, 11:00. TTL 3600 = one hour from when it was cached (10:00). After that, the resolver must ask again.

Interview questions

  • What happens when you type a URL into the browser, up to the moment you have an IP address?
  • What do root, TLD and authoritative name servers each know?
  • What is the difference between a recursive resolver and an authoritative server?
  • What is TTL? What are the trade-offs of a long vs a short TTL? How would you migrate a site to a new IP with minimal impact?
  • What is the difference between an A record and a CNAME? What are MX and NS records for?
  • Why is DNS a potential single point of failure, and how do you design around it? (Multiple servers in many places; the Facebook 2021 outage.)
  • Why does DNS usually use UDP on port 53?

Summary

  • DNS turns names (syntaxpilot.in) into IP addresses.
  • Copies go stale: HOSTS.TXT didn't scale. One central server is a single point of failure.
  • Root → TLD → authoritative split the job; a domain is read right to left. 13 root identities, about 2,000 copies.
  • A recursive resolver asks for you and caches answers; most lookups end at a cache.
  • TTL = how many seconds an answer may be cached: stale for at most the TTL.
  • Record types: A, AAAA, CNAME, MX, NS, TXT. DNS is part of your system: design it with no single path.

Next episode: we kept saying "90 milliseconds" and "0 milliseconds". Next we measure speed properly: latency vs throughput, and why a faster internet plan doesn't make every app faster.