SyntaxPilot
Courses/System Design/Lesson 17

System Design · Lesson 17 of 40 · 12 min

Monolith vs microservices

queued

These are the notes for System Design episode 17. ChaiNow started as one program: the menu, orders, payments and loyalty points are just modules calling each other. That is a monolith, and it's a great way to start. Then one bug takes the whole program down, the Diwali rush forces you to copy everything, and every small change waits for everyone else's. You'll split ChaiNow into microservices, then discover their price: network calls are slow and can fail, so every call needs a timeout and a fallback. Finally, the honest part: when splitting is worth it, and when it isn't. Every demo is plain Python and runs as-is; outputs are real (Python 3.13). RAM sizes, copy counts, the 5 ms network time and the 99.9% uptime are made-up numbers.

The problems

Problem What it teaches
One bug, whole app down Monolith, process, shared fate, fault isolation
Diwali: only the menu is busy Scaling a monolith means copying all of it
Every change waits for everyone Tied change cycles, release trains, independent deploys
Split it up Microservices: own process, own database, own team; Conway's law
Network calls are slow and flaky Latency, compound availability, timeouts, fallbacks
Was it worth it? Distributed data, operations cost, Prime Video's 90% cut, Monolith First, modular monolith

The start: a monolith

A monolith is an application "built as a single unit" (Lewis & Fowler): one codebase, built, deployed and run as one program. Think of a small dhaba where one kitchen does everything.

  • Modules call each other with ordinary function calls, in the same memory: instant and never lost.
  • There is usually one shared database: any module can read any table, and a SQL JOIN across orders and points is easy.
  • For a small team it is simple to build, test, run and debug. This is not a mistake. It's the right place to start.

A process is one running program, with its own memory, managed by the operating system. A monolith runs as one process (or several identical copies of it).

Problem 1: one bug, whole app down

Three teams work in one program. The points team ships a "streak bonus": average points per visit.

menu   = {"chai": 20, "samosa": 15}       # menu team's code
points = {"asha": 40, "ravi": 0}          # points team's code
visits = {"asha": 4,  "ravi": 0}

def price(items):                         # menu team
    return sum(menu[i] for i in items)

def add_points(user, amount):             # points team, new "streak bonus"
    streak = points[user] // visits[user] # average points per visit
    points[user] += amount // 10 + streak

def place_order(user, items):             # orders team
    total = price(items)
    add_points(user, total)               # a normal function call
    print(f"{user}: order of Rs {total} placed")

place_order("asha", ["chai", "samosa"])
place_order("ravi", ["chai"])             # ravi's first visit
print("menu still works?", price(["chai"]))

Output:

asha: order of Rs 35 placed
Traceback (most recent call last):
  File "main.py", line 18, in <module>
    place_order("ravi", ["chai"])             # ravi's first visit
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
  File "main.py", line 14, in place_order
    add_points(user, total)               # a normal function call
    ~~~~~~~~~~^^^^^^^^^^^^^
  File "main.py", line 9, in add_points
    streak = points[user] // visits[user] # average points per visit
             ~~~~~~~~~~~~~^^~~~~~~~~~~~~~
ZeroDivisionError: integer division or modulo by zero

Ravi has zero visits, so // 0 raises ZeroDivisionError. The order fails because of loyalty points, the least important feature. And the last line, the menu check, never runs: the whole program stopped. In one process every module shares one fate. (A real web server usually turns an exception into a failed request rather than exiting, but a crash, a memory leak or an infinite loop in any module still takes the whole process, and every feature in it, down.)

Fault isolation means one part failing doesn't take the others with it. A monolith has little of it.

Problem 2: Diwali, only the menu is busy

In episode 5 we scaled out by running more copies behind a load balancer. A monolith is one unit, so every copy carries every module.

ram_gb = {"menu": 0.5, "orders": 0.5, "payments": 0.5, "points": 0.5}
need   = {"menu": 10,  "orders": 2,   "payments": 2,   "points": 1}   # copies for the Diwali rush

one_app = sum(ram_gb.values())            # a monolith copy carries every module
mono = max(need.values()) * one_app       # copy ALL of it for the busiest part
print(f"monolith: {max(need.values())} copies x {one_app} GB = {mono} GB")

split = sum(need[s] * ram_gb[s] for s in ram_gb)
for s in ram_gb:
    print(f"  {s:8} {need[s]:2} copies x {ram_gb[s]} GB")
print(f"separate services: {split} GB")

Output:

monolith: 10 copies x 2.0 GB = 20.0 GB
  menu     10 copies x 0.5 GB
  orders    2 copies x 0.5 GB
  payments  2 copies x 0.5 GB
  points    1 copies x 0.5 GB
separate services: 7.5 GB

Lewis & Fowler: in a monolith, "Scaling requires scaling of the entire application rather than parts of it that require greater resource."

Problem 3: every change waits for everyone

With forty developers in one codebase, a one-word typo fix on the menu still means rebuilding, retesting and redeploying the whole app. Lewis & Fowler: "Change cycles are tied together - a change made to a small part of the application, requires the entire monolith to be rebuilt and deployed." Teams batch changes into a weekly release train; one team's failing test makes everyone miss it.

The fix: microservices

From one dhaba to a food court: separate stalls, each with its own store room and its own staff.

Lewis & Fowler (2014): "the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API."

Idea Meaning
Service A separate program, in its own process, doing one business job (menu, orders, points)
Own database Each service manages its own data ("letting each service manage its own database"); others ask through its API
Independent deploys A team ships its service without waiting for others
Independent scaling 10 copies of menu, 1 of points
Fault isolation If points crashes, orders keep working (if orders handles that well, see below)

Conway's law: "Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure." Microservices use it on purpose: one team, one service.

Problem 4: network calls are slow and flaky

A function call becomes a network call (a remote call): a request sent to another machine and a reply sent back. It takes time (latency) and can be lost.

import time

def points_local(user):                   # inside the monolith
    return 40

def points_remote(user):                  # points is now another service
    time.sleep(0.005)                     # ~5 ms there and back (made up)
    return 40

for call in [points_local, points_remote]:
    start = time.perf_counter()
    for _ in range(100):
        call("asha")
    ms = (time.perf_counter() - start) * 1000
    print(f"{call.__name__:13}: 100 calls in {ms:7.2f} ms")

print("each service is up 99.9% of the time:")
for hops in [1, 5, 10, 20]:
    up = 0.999 ** hops                    # every hop must work
    print(f"  order needs {hops:2} services -> works {up:.1%} of the time")

Output (the remote timing varies a little per run):

points_local : 100 calls in    0.01 ms
points_remote: 100 calls in  515.38 ms
each service is up 99.9% of the time:
  order needs  1 services -> works 99.9% of the time
  order needs  5 services -> works 99.5% of the time
  order needs 10 services -> works 99.0% of the time
  order needs 20 services -> works 98.0% of the time

Availability (the share of time something works) multiplies along a chain: if every hop must work, the chain works 0.999 × 0.999 × ... of the time. Twenty services at 99.9% each give 98.0%: 1 order in 50 fails. A chatty design, where one request hops through many services, gets slow and fragile.

Timeouts and fallbacks

The worst failure is the service that never answers. A timeout is the longest you'll wait for a reply. A fallback is what you do instead when the call fails.

import time
from concurrent.futures import ThreadPoolExecutor, TimeoutError
pool = ThreadPoolExecutor()
later = []                                 # a queue, like episode 16

def points_service(user, amount):          # another team's service
    time.sleep(2)                          # it's stuck today
    return "ok"

def place_order(user, total, timeout=None):
    start = time.perf_counter()
    answer = pool.submit(points_service, user, total)   # send the request
    try:
        answer.result(timeout=timeout)     # wait for the reply
    except TimeoutError:
        print("   points: no reply, add them later")
        later.append((user, total))        # fallback
    took = time.perf_counter() - start
    print(f"{user}: order placed in {took:.1f} s")

place_order("asha", 35)                    # wait as long as it takes
place_order("ravi", 20, timeout=0.3)       # wait at most 0.3 s
print("points to add later:", later)

Output:

asha: order placed in 2.0 s
   points: no reply, add them later
ravi: order placed in 0.3 s
points to add later: [('ravi', 20)]

ThreadPoolExecutor.submit runs the call on a background thread and returns a future (a placeholder for the result); result(timeout=...) waits at most that long and raises TimeoutError otherwise. Rule: every network call gets a timeout and a plan for failure; otherwise one slow service makes all its callers slow too (a cascading failure). For points, the plan is a queue (episode 16): the order succeeds and points are added later.

Problem 5: was it worth it?

The bill for microservices:

  • Data is split. No SQL JOIN across services; keeping orders and payments consistent across two databases is hard (later episodes cover patterns for this).
  • Debugging spreads out. One order touches many services; you follow it through many logs (episode 25: observability).
  • Operations multiply. Twenty programs to deploy, monitor, secure and version.

Real example. In 2023 Amazon's Prime Video team described its audio/video quality-monitoring service, built from distributed serverless components (AWS Step Functions, Lambda, S3, SNS). Costs came from "a high volume of read/writes to the S3 bucket storing intermediate work items" and "several state transitions for each second of the analyzed audio/video stream". They moved the work into a single application process on ECS/EC2, and operational costs fell by 90% (InfoQ, May 2023). For that workload, fewer network hops won.

When to split

Martin Fowler, "MonolithFirst" (2015): "Almost all the successful microservice stories have started with a monolith that got too big and was broken up", and "Almost all the cases where I've heard of a system that was built as a microservice system from scratch, it has ended up in serious trouble."

Situation Choice
Small team, new product A monolith
Growing codebase A modular monolith: one deploy, clean modules that only talk through clear interfaces
One part keeps breaking others Split that part out
One part needs 10x the scale Split that part out
Teams keep blocking each other's releases Split along team lines
One request hops through many services You split too far; merge or use a queue

The whole chain

  1. Monolith: one program, fast function calls, one database. A good start.
  2. One bug (ZeroDivisionError) → the whole program stops.
  3. Diwali → copy everything: 20 GB vs 7.5 GB.
  4. Every change waits for the release train.
  5. Microservices: own process, own database, own team; fault isolation, independent scaling and deploys.
  6. Network calls: ~5 ms each, 99.9%^20 = 98.0% → timeouts and fallbacks.
  7. Real cost (Prime Video: −90% after moving to one process) → Monolith First, split when you can name the pain.

Common mistakes (and the real errors)

Mistake Code Error / result Fix
Risky code in the same process as everything points[user] // visits[user] ZeroDivisionError: integer division or modulo by zero (and the program stops) Test edge cases; isolate risky or low-priority parts
Calling a service with no timeout answer.result() asha: order placed in 2.0 s result(timeout=0.3) + a fallback
Long chains of service calls 20 services in a row works 98.0% of the time Fewer hops; move side jobs to a queue
Microservices sharing one database every service reads every table a schema change breaks other teams Each service owns its data; others use its API
Splitting a new product on day one 20 services for 3 developers slow delivery, many moving parts Monolith first; keep it modular

Mini project: a safe call to a flaky service

A timeout on every call, one retry, and a fallback list that is drained once the service is healthy.

# Mini project: an order service that calls a flaky points service safely:
# a timeout on every call, one retry, and a fallback list that is drained later.
import time
from concurrent.futures import ThreadPoolExecutor, TimeoutError

pool = ThreadPoolExecutor(max_workers=8)
pending, points = [], {}
calls = {"n": 0}
HANGS = {1, 3, 4}                             # which calls hang (made up)

def points_service(user, amount):             # another team's service
    calls["n"] += 1
    if calls["n"] in HANGS:
        time.sleep(1.0)                       # stuck, then gives up
        raise ConnectionError("points service reset the connection")
    points[user] = points.get(user, 0) + amount // 10
    return "ok"

def call_with_timeout(fn, *args, timeout=0.2, retries=1):
    for attempt in range(1, retries + 2):
        try:
            return pool.submit(fn, *args).result(timeout=timeout)
        except TimeoutError:
            print(f"   attempt {attempt}: no reply in {timeout} s")
    raise TimeoutError("points service unavailable")

def place_order(user, total):
    start = time.perf_counter()
    try:
        call_with_timeout(points_service, user, total)
    except TimeoutError:
        pending.append((user, total))         # never fail the order for points
    print(f"{user}: order placed in {time.perf_counter() - start:.1f} s")

for user, total in [("asha", 35), ("ravi", 20), ("meena", 50)]:
    place_order(user, total)
print("pending:", pending)

HANGS.clear()                                 # later: the service is healthy again
while pending:
    call_with_timeout(points_service, *pending.pop(0))
print("points:", points)

Output:

   attempt 1: no reply in 0.2 s
asha: order placed in 0.2 s
   attempt 1: no reply in 0.2 s
   attempt 2: no reply in 0.2 s
ravi: order placed in 0.4 s
meena: order placed in 0.0 s
pending: [('ravi', 20)]
points: {'asha': 3, 'meena': 5, 'ravi': 2}

No order ever waited more than 0.4 s, and Ravi's points arrived later. Careful: here a hung call fails without adding points. If a "timed-out" call could still succeed later, a retry would add points twice, so real retries need the idempotency keys from episode 16. Extend it: add an order id to every call and skip ids already applied.

Practice

  1. Homework: fix the streak bonus in Demo 1 so new users don't crash it. What does the program print now?
  2. Homework: in Demo 3, what is the success rate with 50 services in a row? And with 20 services at 99.99% each?
  3. Homework: in Demo 4, set Ravi's timeout to 3 s. What changes?
  4. ChaiNow has 3 developers and 200 orders a day. Monolith or microservices? Why?
  5. Two services both read and write the same orders table. Which microservice benefit did you lose?
  6. Give two reasons Prime Video's monitoring workload got cheaper in one process.
Solutions and hints

1. Guard the division: streak = points[user] // visits[user] if visits[user] else 0:

asha: order of Rs 35 placed
ravi: order of Rs 20 placed
menu still works? 20

2. print(f"{0.999**50:.1%}", f"{0.9999**20:.1%}") prints 95.1% 99.8%. Fifty hops at 99.9% fail 1 order in 20; making each service ten times more reliable (99.99%) brings 20 hops back to 99.8%.

3. With timeout=3 the reply (2 s) arrives in time:

asha: order placed in 2.0 s
ravi: order placed in 2.0 s
points to add later: []

A timeout longer than the slow case protects nothing. Pick it from how fast the service normally answers, and what the caller can afford.

4. A monolith (keep it modular). None of the pains exist yet: no team blocking, no part needing separate scale. Microservices would add network calls and operations work for nothing.

5. Independence: a schema change by one team can break the other, and they can't deploy or scale their data separately. Each service should own its data.

6. No more reading and writing intermediate frames to S3 between components (data passed in memory), and no more orchestration state transitions for every second of video. Network hops and per-call charges disappeared.

Quick quiz

# every service is up 99.9% of the time
# an order needs these, in a row:
#   gateway, menu, orders, payments, points

print(f"{0.999 ** 5:.1%}")
  • A. 99.9%
  • B. 99.5%
  • C. 95.0%
  • D. 100.0%
Answer

B. Every hop must work, so multiply: 0.999⁵ ≈ 0.995, printed as 99.5%.

Interview questions

  • What is a monolith, and when is it the right choice? (One deployable unit; small teams and new products.)
  • What do microservices give you? (Independent deploys, independent scaling, fault isolation, team ownership.)
  • What do they cost? (Network latency and failures, distributed data, harder debugging, more operations.)
  • What is Conway's law, and how does it relate to microservices? (Systems mirror the organisation's communication; one team per service.)
  • Why must every remote call have a timeout? (Otherwise one stuck service blocks every caller: cascading failure.)
  • What is a modular monolith? (One deploy, but strictly separated modules; easy to split later.)
  • Should each microservice have its own database? Why? (Yes: otherwise schema changes and load couple the services again.)
  • What happened at Prime Video, and what's the lesson? (A distributed design moved into one process cut cost 90%: pick the architecture per workload.)

Glossary

Term Meaning
Monolith An application built, deployed and run as a single unit
Module A part of a program with one job (menu, points), inside the same codebase
Process One running program with its own memory
Function call Calling code in the same process: in memory, instant
Shared fate When one part fails, everything in the same process fails with it
Fault isolation One part failing doesn't take down the others
Scale out Run more copies of a program behind a load balancer (episode 5)
Release train A fixed, shared release (e.g. weekly) that all changes wait for
Deploy Put a new version of a program into production
Microservice A small service in its own process, with its own data, owned by one team
Microservice architecture Building one application as a suite of small services talking over the network
API Application programming interface: how one program asks another for something
Conway's law A system's structure copies the organisation's communication structure
Network call / remote call A request to another machine and its reply
Latency The time a request takes to get its answer
Availability The share of time a system works (99.9% ≈ 43 minutes down a month)
Compound availability Chained parts multiply: 0.999²⁰ = 98.0%
Chatty design One request hopping through many services
Timeout The longest a caller waits for a reply before giving up
Fallback What you do instead when a call fails (a default, a queue, a cached value)
Cascading failure One slow or failed service making its callers slow or fail too
Future A placeholder for a result that isn't ready yet (concurrent.futures)
ThreadPoolExecutor Python's pool of threads that runs functions in the background
Retry Trying a failed call again (safe only if the call is idempotent, episode 16)
JOIN A SQL operation combining rows from two tables; impossible across separate service databases
Modular monolith One deployable program with strictly separated modules
Monolith First Fowler's advice: start with a monolith, split when it gets too big
Serverless Cloud functions and workflows you don't run servers for (e.g. AWS Lambda, Step Functions)

Summary

  • A monolith is one program: simple, fast function calls, one database. A great start.
  • Its pains grow with size: one bug can stop it all (ZeroDivisionError), you scale it as a whole (20 GB vs 7.5 GB), and every change waits for one release.
  • Microservices give each part its own process, data and team: fault isolation, independent scaling and deploys.
  • The price is the network: slow (≈5 ms vs 0.0001 ms here), and failures multiply (99.9%²⁰ = 98.0%). Every call needs a timeout and a fallback.
  • Splitting costs real money and effort (Prime Video cut 90% by going back to one process for one workload). Monolith first, keep it modular, split when you can name the pain.

Next episode: our services talk over the network, but in what language? API design: REST, GraphQL and gRPC.

Sources: James Lewis & Martin Fowler, "Microservices" (martinfowler.com, 2014); Martin Fowler, "MonolithFirst" (2015); InfoQ, "Prime Video Switched from Serverless to EC2 and ECS to Save Costs" (2023), on the Prime Video Tech blog post "Scaling up the Prime Video audio/video monitoring service and reducing costs by 90%".