SyntaxPilot
Courses/System Design/Lesson 20

System Design · Lesson 20 of 40 · 11 min

Proxies and API gateways

queued

These are the notes for System Design episode 20. ChaiNow is now split into small services (menu, orders, payments, users), and the app on every phone has each service's address built in. The menu service moves to a new machine, and every old app crashes with a real ConnectionRefusedError. You'll invent the reverse proxy (one public address with a routing table behind it), tell it apart from a forward proxy, find out why every request suddenly seems to come from one IP address and fix it with X-Forwarded-For / Forwarded (RFC 7239), watch a payment service that forgot its login check let a stranger pay, move the shared checks into the front door to get an API gateway, and stop that gateway from becoming a single point of failure. Every demo is plain Python and runs as-is; outputs are real (Python 3.13). Service names, ports, IP addresses and limits are made up for the demos.

The problems

Problem What it teaches
The app knows every server Clients coupled to the inside layout; ConnectionRefusedError
Which server gets this request? The reverse proxy, a routing table (path prefix → servers), NGINX
Forward or reverse? Who the proxy works for: clients or servers
Everyone has the same IP The proxy hides the client's IP; X-Forwarded-For, Forwarded (RFC 7239)
Payments forgot the login check Shared checks at the door: the API gateway (auth, rate limit, routing, logging)
One door can break Single point of failure; many copies behind a load balancer; keep it thin

Problem 1: one server moves, the app crashes

Since episode 17 ChaiNow is a set of small services, and the app talks to each one directly, so the address of every service is built into the app. The team moves the menu service to a bigger machine: new port, 9011. Millions of phones still have the old address, port 9001, and most people won't update the app today.

This is a real network test: a real socket listens on the new port, and the app connects to the old one.

import socket

# the menu service moved to a new machine: it now listens on port 9011
menu = socket.socket()
menu.bind(("127.0.0.1", 9011))
menu.listen()

# but every phone has the OLD address built into the app
APP_MENU_ADDRESS = ("127.0.0.1", 9001)

print("app: loading the menu...")
conn = socket.create_connection(APP_MENU_ADDRESS, timeout=2)
print("menu loaded")

Output (the middle of the traceback trimmed):

app: loading the menu...
Traceback (most recent call last):
  ...
ConnectionRefusedError: [Errno 111] Connection refused

Nothing is listening at the old address, so the operating system refuses the connection and the app can't even load the menu. The real cause: the clients know how the inside of the system is laid out. Every change inside (a move, a new copy, a split) breaks people outside.

An IP address (like 127.0.0.1, "this machine") identifies a machine on a network; a port (like 9011) identifies one program listening on that machine. Together they are an address a client can connect to.

Problem 2: one front door (the reverse proxy)

At a big hotel, guests never wander the corridors looking for the laundry. They go to one reception desk. Reception keeps a list ("laundry → room 12"), and when the laundry moves to room 20, reception updates its list and the guests never notice.

At a hotel Reverse proxy
One reception desk, every guest goes there first One public address: api.chainow.in
Reception's room list A routing table: /menu → menu servers
Staff change rooms, guests never notice Servers are hidden: move them freely
from itertools import cycle

# the proxy's routing table: path prefix -> the servers behind it
ROUTES = {
    "/menu":   cycle(["menu-1:9011", "menu-2:9012"]),
    "/orders": cycle(["orders-1:9021", "orders-2:9022"]),
    "/pay":    cycle(["pay-1:9031"]),
}

def proxy(path):                     # the app only knows api.chainow.in
    for prefix, servers in ROUTES.items():
        if path.startswith(prefix):
            return f"{path:<14} -> {next(servers)}"
    return f"{path:<14} -> 404 Not Found"

for path in ["/menu/today", "/menu/today", "/orders/42", "/pay/42", "/admin"]:
    print(proxy(path))

ROUTES["/menu"] = cycle(["menu-3:9013"])    # menu moved: change ONE line here
print("after the move:", proxy("/menu/today"))

Output:

/menu/today    -> menu-1:9011
/menu/today    -> menu-2:9012
/orders/42     -> orders-1:9021
/pay/42        -> pay-1:9031
/admin         -> 404 Not Found
after the move: /menu/today    -> menu-3:9013
  • ROUTES is the routing table: each path prefix (the start of the URL path, like /menu) maps to the servers behind it.
  • cycle hands out those servers in turn, like the load balancer from episode 6: menu requests alternate between menu-1 and menu-2.
  • No prefix matches /admin, so the proxy answers 404 Not Found itself; no service is bothered.
  • When the menu moves, one line changes, in the proxy only. The same app request lands on menu-3, and not a single phone needs an update.

A server that stands in front of your servers, takes every request, and forwards it to the right one is a reverse proxy. You just invented it.

The real thing: NGINX

In real life you rarely write this yourself; a popular choice is NGINX. Its docs ("NGINX Reverse Proxy") say: "Proxying is typically used to distribute the load among several servers, seamlessly show content from different websites, or pass requests for processing to application servers over protocols other than HTTP." The video's config (an NGINX config file, not Python):

server {
    listen 443 ssl;
    server_name api.chainow.in;

    location /menu/ {
        proxy_set_header X-Real-IP $remote_addr;
        proxy_pass http://menu_servers;
    }
    location /orders/ {
        proxy_pass http://order_servers;
    }
}
  • Each location block is one row of the routing table: paths starting with /menu/ are passed, with proxy_pass, to the group of menu servers. (The docs' own example is location /some/path/ { proxy_pass http://www.example.com/link/; }.)
  • listen 443 ssl means the proxy handles the encryption (HTTPS) once, at the door, and the servers behind it don't each need to. This is called TLS termination (often still called SSL termination).
  • proxy_set_header X-Real-IP $remote_addr; passes the client's address on to the server (Problem 4).
  • The docs also note: "By default, NGINX modifies two header fields in proxied requests, Host and Connection."

What a reverse proxy commonly does for you: one public address, routing, load balancing, TLS termination, hiding the servers' addresses, and a single place to add caching, compression or limits.

Problem 3: forward or reverse?

Both are a middleman. The difference is who it works for.

Forward proxy Reverse proxy
Real life You send a friend to buy something; the shop only sees your friend The counter boy fetches boxes from the godown; you never see the godown
Sits next to The clients The servers
Works for The clients The servers
Hides Who is asking Which server answers
Typical uses Office or school networks: every request must pass through it; block sites, hide or log who is asking One public address, routing, load balancing, TLS termination

Problem 4: everyone has the same IP

Four customers, each on their own phone with their own IP address. But every request now goes through the proxy, and the proxy opens its own connection to the server. RFC 7239 describes exactly this: proxying makes requests "appear as if they originated from the proxy's IP address". And the rate limit from episode 19 is five requests per IP address per minute.

LIMIT = 5                            # 5 requests per IP per minute (episode 19)
PROXY_IP = "10.0.0.5"

# 4 customers, 3 requests each; every one passes through the proxy
customers = {"asha": "49.36.1.10", "ravi": "49.36.7.22",
             "meena": "117.9.3.4", "john": "157.4.8.9"}
requests = [{"remote_addr": PROXY_IP,             # what the server sees
             "X-Forwarded-For": ip}               # added by the proxy
            for ip in customers.values() for _ in range(3)]

def served(key):
    counts, ok = {}, 0
    for r in requests:
        ip = r[key]
        counts[ip] = counts.get(ip, 0) + 1
        ok += counts[ip] <= LIMIT
    return ok

print("limit by remote_addr:    ", served("remote_addr"), "of 12 served")
print("limit by X-Forwarded-For:", served("X-Forwarded-For"), "of 12 served")

Output:

limit by remote_addr:     5 of 12 served
limit by X-Forwarded-For: 12 of 12 served
  • remote_addr is the address of the connection the server actually sees: always the proxy, 10.0.0.5.
  • X-Forwarded-For is a header (a Name: value line sent with an HTTP request) that the proxy adds, carrying the customer's real IP.
  • Keyed by remote_addr, all four customers look like one client and share one limit: 5 of 12. Keyed by the forwarded IP, everyone gets their own fair share: 12 of 12.

So the proxy must pass on what it hides. RFC 7239, "Forwarded HTTP Extension" (June 2014), defines a header "that allows proxy components to disclose information lost in the proxying process, for example, the originating IP address of a request". Its example: Forwarded: for=192.0.2.43. Its parameters are for (the client), by (the proxy that received the request), host (the original Host header) and proto (http or https). The older X-Forwarded-* headers are non-standard, but you'll still see them everywhere.

Forwarded: for=49.36.1.10            (RFC 7239, the standard)
X-Forwarded-For: 49.36.1.10          (older, very common)
X-Real-IP: 49.36.1.10                (set by NGINX's proxy_set_header in our config)

Trust it carefully

It's just a header, and anyone can type a header. A bot could claim a new IP on every request and dodge the limit. So:

  • Servers should trust it only when their own proxy set it (requests from the proxy's address), never directly from the internet.
  • The proxy should overwrite (or carefully append to) whatever the client sent, not pass a fake one through.

Problem 5: payments forgot the login check

Every service must check that the user is logged in, so each one has a copy of the same check. And the same rate limiting, and the same logging: four services, four copies of everything. Then, in a busy week, a new developer writes the payment service and forgets one line.

VALID_TOKENS = {"tok-asha"}          # logged-in users

def check_login(req):
    if req.get("token") not in VALID_TOKENS:
        raise PermissionError("401 Unauthorized")

# every service copies the same login check... almost
def menu(req):   check_login(req); return "menu sent"
def orders(req): check_login(req); return "order placed"
def users(req):  check_login(req); return "profile sent"
def pay(req):    return "payment of Rs 500 done"     # someone forgot!

stranger = {"token": None}           # not logged in
for service in [menu, orders, users, pay]:
    try:
        print(f"{service.__name__:<7}", service(stranger))
    except PermissionError as e:
        print(f"{service.__name__:<7}", e)

Output:

menu    401 Unauthorized
orders  401 Unauthorized
users   401 Unauthorized
pay     payment of Rs 500 done

check_login raises 401 Unauthorized when the token (a string the app got at login and sends with each request) isn't valid. Three services refuse the stranger. Payments takes 500 rupees from someone who isn't even logged in. A check that every request needs was copied into every service, and one missed copy is a security hole.

The fix: do the shared checks once, at the door

Nobody checks your ID card on every floor of an office building. Security checks it once, at the gate.

At an office API gateway
ID card check at the gate, once for every floor Authentication: no valid token → 401
Visitor register; stop someone who keeps coming back Logging + rate limits: too many → 429
"Accounts? 3rd floor" Routing: /pay → pay service
VALID_TOKENS, LIMIT = {"tok-asha", "tok-ravi"}, 3
counts = {}

def auth(req):                       # 1. who are you?
    if req["token"] not in VALID_TOKENS: return "401 Unauthorized"
def rate_limit(req):                 # 2. too many? (episode 19)
    counts[req["token"]] = counts.get(req["token"], 0) + 1
    if counts[req["token"]] > LIMIT: return "429 Too Many Requests"
def route(req):                      # 3. which service?
    return "-> " + req["path"].split("/")[1] + " service"

def gateway(req):                    # every request, the same checks
    for step in [auth, rate_limit, route]:
        answer = step(req)
        if answer: return answer

for token, path in [(None, "/pay/42"), ("tok-asha", "/pay/42"),
                    ("tok-ravi", "/menu"), ("tok-ravi", "/menu"),
                    ("tok-ravi", "/menu"), ("tok-ravi", "/orders/7")]:
    print(f"{str(token):<9} {path:<10}", gateway({"token": token, "path": path}))

Output:

None      /pay/42    401 Unauthorized
tok-asha  /pay/42    -> pay service
tok-ravi  /menu      -> menu service
tok-ravi  /menu      -> menu service
tok-ravi  /menu      -> menu service
tok-ravi  /orders/7  429 Too Many Requests
  • Three steps, in order: auth (who are you?), rate_limit (too many? three per user here), route (which service?).
  • Each step either returns an answer that stops the request, or None, so the next step runs. This chain of small steps that every request passes through is often called middleware.
  • The stranger is stopped at the door with 401 before payments even sees the request. Asha is routed to payments. Ravi's first three go through; his fourth gets 429 Too Many Requests.

A reverse proxy that also does the shared jobs every request needs (login checks, rate limits, logging, routing) is an API gateway. You just invented it.

A real one: the AWS docs ("Amazon API Gateway quotas") give an account-level throttle per Region of "10,000 requests per second (RPS) with an additional burst capacity provided by the token bucket algorithm, using a maximum bucket capacity of 5,000 requests". That's the token bucket you invented in episode 19.

Reverse proxy vs API gateway vs load balancer

Main job Knows about
Load balancer Spread requests across copies of the same server Servers and their health
Reverse proxy One front door: route, balance, TLS, hide servers Paths, hosts, servers
API gateway A reverse proxy plus the shared API jobs: auth, rate limits, logging, routing Users, tokens, limits, APIs

The lines blur: NGINX can do all three, and many gateways are built on a reverse proxy.

Problem 6: one door can break

Now every single request passes through the gateway. If it goes down, all of ChaiNow goes down: it's a single point of failure.

App (phones)  →  Load balancer  →  Gateway x3 (same config)  →  Services (menu, pay...)
  • Run several identical copies of it behind a load balancer, exactly like any other server since episode 6. Because the copies share the same config, any one can handle any request. (Shared state, like the rate-limit counters, lives in a shared store such as Redis, as in episode 19.)
  • Keep it thin. The gateway checks, counts and routes. Prices, orders and payments stay in the services. A gateway full of business logic just becomes a new monolith, and every change to it risks every service.
  • It is also one more hop on every request, so it should be fast.

The whole chain

  1. The app knew every address → the menu moved → Connection refused.
  2. One front door + a routing table: the reverse proxy; moving a service is one line.
  3. Forward proxy works for clients; reverse proxy works for servers.
  4. Every request had the proxy's IP → limit served 5 of 12 → X-Forwarded-For / Forwarded → 12 of 12 (trust it only from your own proxy).
  5. Payments forgot the login check → shared checks at the door: the API gateway (auth, rate limit, route, log).
  6. The gateway is a single point of failure → many copies, kept thin.

Common mistakes (and the real errors)

Mistake Code Error / result Fix
Building server addresses into the client APP_MENU_ADDRESS = ("127.0.0.1", 9001) ConnectionRefusedError: [Errno 111] Connection refused One public address: a reverse proxy
Limiting by the connection's address behind a proxy ip = request.remote_addr limit by remote_addr: 5 of 12 served Read X-Forwarded-For / Forwarded set by your proxy
Copying the login check into every service def pay(req): return "payment of Rs 500 done" pay payment of Rs 500 done Check logins once, in the gateway
Trusting X-Forwarded-For straight from the internet ip = headers["X-Forwarded-For"] a bot sends a new fake IP each time and is never limited Trust it only from your proxy; the proxy overwrites it
Prefix without a / boundary path.startswith("/pay") /payroll -> pay-1:9031 Match path == p or path.startswith(p + "/")
One gateway copy a single gateway server it goes down, every service is unreachable Several copies behind a load balancer
Business logic in the gateway order totals computed in the gateway a new monolith every team must change Keep it thin: check, count, route

Mini project: a tiny API gateway with health checks

Auth, rate limiting, routing with a proper prefix match, load balancing that skips unhealthy servers, and an access log with the client's IP.

# Mini project: a tiny API gateway with health checks
from itertools import count

VALID_TOKENS, LIMIT = {"tok-asha", "tok-ravi"}, 3
SERVERS = {"/menu": ["menu-1", "menu-2"], "/orders": ["orders-1"], "/pay": ["pay-1"]}
healthy = {s: True for group in SERVERS.values() for s in group}
turn, counts, log = count(), {}, []

def pick(prefix):                    # take turns, but skip unhealthy servers
    alive = [s for s in SERVERS[prefix] if healthy[s]]
    return alive[next(turn) % len(alive)] if alive else None

def gateway(token, path, client_ip):
    if token not in VALID_TOKENS:
        status = "401 Unauthorized"
    elif counts.setdefault(token, 0) >= LIMIT:
        status = "429 Too Many Requests"
    else:
        counts[token] += 1
        prefix = next((p for p in SERVERS if path == p or path.startswith(p + "/")), None)
        server = pick(prefix) if prefix else None
        status = ("200 OK via " + server) if server else \
                 "404 Not Found" if not prefix else "503 Service Unavailable"
    log.append(f"{client_ip:<11} {str(token):<9} {path:<12} {status}")
    return status

requests = [("tok-asha", "/menu/today", "49.36.1.10"),
            ("tok-asha", "/menu/today", "49.36.1.10"),
            (None,       "/pay/42",     "157.4.8.9"),
            ("tok-ravi", "/payroll",    "49.36.7.22")]
for r in requests: gateway(*r)

healthy["menu-2"] = False            # the health check finds menu-2 down
healthy["pay-1"] = False             # and pay-1 too
for r in [("tok-asha", "/menu/today", "49.36.1.10"),
          ("tok-ravi", "/pay/42",     "49.36.7.22"),
          ("tok-asha", "/orders/7",   "49.36.1.10"),
          ("tok-ravi", "/menu",       "49.36.7.22")]:
    gateway(*r)

print("\n".join(log))

Output:

49.36.1.10  tok-asha  /menu/today  200 OK via menu-1
49.36.1.10  tok-asha  /menu/today  200 OK via menu-2
157.4.8.9   None      /pay/42      401 Unauthorized
49.36.7.22  tok-ravi  /payroll     404 Not Found
49.36.1.10  tok-asha  /menu/today  200 OK via menu-1
49.36.7.22  tok-ravi  /pay/42      503 Service Unavailable
49.36.1.10  tok-asha  /orders/7    429 Too Many Requests
49.36.7.22  tok-ravi  /menu        200 OK via menu-1
  • Menu requests alternate between menu-1 and menu-2 until the health check marks menu-2 down; after that every menu request goes to menu-1.
  • /payroll is a 404: the prefix must be followed by / (or be the whole path), so it doesn't fall into /pay.
  • With pay-1 down there is no healthy server left, so the gateway answers 503 Service Unavailable instead of hanging.
  • Asha's fourth request is 429: three per token.
  • Every request is logged with the client's IP, which in a real deployment would come from X-Forwarded-For set by the load balancer in front.

Extend it: add a /users route, make the rate limit a token bucket (episode 19), or add a by= and proto= to a Forwarded header the gateway sends to the services.

Practice

  1. Homework: in the PROXY demo, add a /users route to users-1:9041 and send /users/7. What prints?
  2. Homework: in the CLIENT_IP demo, set LIMIT = 2. What prints?
  3. Homework: in the GATEWAY demo, add a logging step that prints every request. Where should it go in the list, and why?
  4. In the PROXY demo, what do /menu, /menus-secret, /payroll and /Menu/today return?
  5. A bot sends X-Forwarded-For: 1.2.3.<n> with a new n on every request, straight to your server. What happens to the per-IP limit, and how do you stop it?
  6. Homework: should the gateway calculate the order total? Why, or why not?
Solutions and hints

1. Add "/users": cycle(["users-1:9041"]), to ROUTES and "/users/7" to the list:

/menu/today    -> menu-1:9011
/menu/today    -> menu-2:9012
/orders/42     -> orders-1:9021
/pay/42        -> pay-1:9031
/users/7       -> users-1:9041
/admin         -> 404 Not Found
after the move: /menu/today    -> menu-3:9013

2.

limit by remote_addr:     2 of 12 served
limit by X-Forwarded-For: 8 of 12 served

By remote_addr all 12 requests share one limit of 2. By the forwarded IP each customer gets 2 of their 3: 4 × 2 = 8.

3. Put it first, so even rejected requests are logged (a log of 401s and 429s is exactly what you want when someone attacks):

def log(req):                        # 0. write it down
    print("  log:", req["token"], req["path"])

and for step in [log, auth, rate_limit, route]:. log returns None, so the chain continues:

  log: None /pay/42
None      /pay/42    401 Unauthorized
  log: tok-asha /pay/42
tok-asha  /pay/42    -> pay service
  log: tok-ravi /menu
tok-ravi  /menu      -> menu service
  log: tok-ravi /menu
tok-ravi  /menu      -> menu service
  log: tok-ravi /menu
tok-ravi  /menu      -> menu service
  log: tok-ravi /orders/7
tok-ravi  /orders/7  429 Too Many Requests

4. After the PROXY demo's definitions:

for path in ["/menu", "/menus-secret", "/payroll", "/Menu/today"]:
    print(proxy(path))
/menu          -> menu-1:9011
/menus-secret  -> menu-2:9012
/payroll       -> pay-1:9031
/Menu/today    -> 404 Not Found

startswith is too loose (/menus-secret, /payroll slip into the wrong service) and case-sensitive (/Menu is a 404). Match on a / boundary, as the mini project does.

5. Each fake IP looks like a new customer, so the bot is never limited. Only accept the header from your own proxy's address, have the proxy overwrite whatever the client sent, and limit by the API token or user id as well as by IP.

6. No. The gateway does the jobs every request shares (who are you, how often, where to). Prices and order totals are the order service's business logic; putting them in the gateway makes it a new monolith that every team must change and redeploy, on the path of every request.

Quick quiz

# Your college Wi-Fi sends every student's
# request through one machine.
# It blocks some websites, and the websites
# only ever see the college's IP address.

# What is that machine?
  • A. A reverse proxy
  • B. A forward proxy
  • C. An API gateway
  • D. A database
Answer

B. A forward proxy. It sits next to the clients, works for them, and hides who is asking from the websites. A reverse proxy would sit next to the servers.

Interview questions

  • What is a reverse proxy, and why put one in front of your services? (One public address; route, load balance, TLS termination; hide and move servers freely.)
  • Forward proxy vs reverse proxy? (Forward works for clients and hides them; reverse works for servers and hides them.)
  • Behind a proxy, how does a server learn the client's IP? (X-Forwarded-For, or the standard Forwarded: for= from RFC 7239, or X-Real-IP; trust it only from your own proxy.)
  • What does an API gateway do that a plain reverse proxy may not? (Authentication, rate limiting, logging, request/response changes, per-API routing.)
  • API gateway vs load balancer? (A load balancer spreads requests across copies of one service; a gateway is the front door for many services and does shared API checks.)
  • Isn't the gateway a single point of failure? (Yes, unless you run several copies behind a load balancer, keep shared state in a shared store, and keep it thin and fast.)
  • What should never go into a gateway? (Business logic: prices, orders, payments.)
  • Where does TLS end, and is traffic behind the proxy encrypted? (At the proxy; behind it, it may be plain or re-encrypted, depending on how much you trust the inside network.)

Glossary

Term Meaning
Service A small program that does one job (menu, orders, payments, users)
Client The program that sends requests, e.g. the ChaiNow app
Server A machine or program that answers requests
IP address A number that identifies a machine on a network, e.g. 49.36.1.10; 127.0.0.1 means "this machine"
Port A number that picks one program on a machine, e.g. 9011; 443 is the usual HTTPS port
Socket The operating system's handle for one network connection
ConnectionRefusedError Python's error when nothing is listening at the address you connect to
Proxy A middleman that receives a request and sends it on for someone else
Forward proxy A proxy next to the clients that works for them, e.g. an office or college network; can block sites and hide who is asking
Reverse proxy A proxy next to the servers that works for them: one public address, forwarding each request to the right server
Routing table A list that says which servers handle which requests
Path The part of a URL after the host, e.g. /menu/today
Path prefix The start of a path used for routing, e.g. /menu
404 Not Found HTTP status: nothing matches this path
401 Unauthorized HTTP status: no valid login (authentication failed)
429 Too Many Requests HTTP status: rate limit exceeded (episode 19)
503 Service Unavailable HTTP status: the server can't handle the request right now, e.g. no healthy server
Load balancer Spreads requests across several copies of a server (episode 6)
Health check Regularly testing each server and sending traffic only to healthy ones
NGINX A popular web server often used as a reverse proxy and load balancer
location / proxy_pass NGINX config: match a path / forward the request to these servers
TLS / SSL The encryption behind HTTPS; SSL is the older name, still widely used
TLS (SSL) termination Decrypting HTTPS at the proxy, so the servers behind it don't each have to
HTTP header A Name: value line sent with a request or response
remote_addr The address of the connection the server sees; behind a proxy, the proxy's address
X-Forwarded-For A common, non-standard header in which a proxy passes on the client's IP
X-Real-IP A similar header, often set by NGINX with proxy_set_header X-Real-IP $remote_addr;
Forwarded The standard header from RFC 7239, e.g. Forwarded: for=192.0.2.43; parameters for, by, host, proto
RFC "Request for Comments": the documents that define internet standards
RFC 7239 "Forwarded HTTP Extension" (June 2014), which standardised the Forwarded header
Rate limit A cap on requests per client per period of time (episode 19)
Token (login token) A string a client gets at login and sends with each request to prove who it is
Token bucket The rate-limit algorithm from episode 19 (bucket size = burst, refill rate = average)
Authentication Checking who the caller is
Authorization Checking what the caller is allowed to do
API gateway A reverse proxy that also does the shared jobs every request needs: authentication, rate limits, logging, routing
Middleware A step that every request passes through before (or after) the real handler
Logging Writing down each request (who, what, when, result)
Single point of failure One part whose failure takes the whole system down
Business logic The rules of the business itself: prices, orders, payments
Monolith One big program that does everything (episode 17)
Redis A fast in-memory store, used for shared counters (episode 19)

Summary

  • When clients know your servers' addresses, every change inside breaks them (Connection refused).
  • A reverse proxy gives clients one public address and hides the servers behind a routing table; moving a service is one line in the proxy. NGINX is a common choice, and it can also end TLS at the door.
  • A forward proxy works for the clients; a reverse proxy works for the servers.
  • Behind a proxy every request seems to come from the proxy's IP; pass the client's IP on in X-Forwarded-For or Forwarded (RFC 7239), and trust it only from your own proxy.
  • Shared checks copied into every service get forgotten; an API gateway does authentication, rate limits, logging and routing once, at the door.
  • The gateway is a single point of failure: run several copies behind a load balancer and keep it thin.

Next episode: ChaiNow's menu now has a photo of every dish. Where do millions of photos live, and how do they load fast in Chennai and in Delhi? Storage and CDNs.

Sources: NGINX docs, "NGINX Reverse Proxy"; RFC 7239, "Forwarded HTTP Extension" (June 2014); AWS docs, "Amazon API Gateway quotas".