SyntaxPilot
Courses/AI: Zero to Agents/Lesson 37
Video coming soonThis lesson’s video is being made

Start with the notes below; the video appears here as soon as it is on YouTube.

AI: Zero to Agents · Lesson 37 of 40

Agent design patterns and frameworks

queued

These are the notes for AI episode 37. Real teams rarely build one huge prompt or one all-powerful agent. They arrange LLM calls in a few patterns that come up again and again. Anthropic's guide "Building effective agents" (19 December 2024) names five: prompt chaining, routing, parallelization, orchestrator-workers and evaluator-optimizer. In this episode you invent each one by watching the simple way fail on screen, then learn when a framework helps and when it hurts. Every Python example runs as-is in the SyntaxPilot compiler; outputs are real (Python 3.13; timings rounded to 0.1 s). The LLMs are toys (if-rules and fixed lists) standing in for real model calls.

Real-life example: a wedding caterer's kitchen

In the kitchen Pattern
Chop, cook, taste, serve: one station after another prompt chaining: steps with checks between
The head waiter sends sweets to the halwai, rotis to the tandoor routing: classify, then a specialist
Five cooks roll rotis together; three tasters vote on a new dish parallelization: sectioning and voting
The head chef plans today's work and assigns cooks orchestrator-workers: subtasks decided at run time

Workflows vs agents

The guide splits "agentic systems" into two kinds:

Workflow Agent
Who decides the next step? your code the model
Definition (Anthropic) "LLMs and tools are orchestrated through predefined code paths" LLMs "dynamically direct their own processes and tool usage"
Good for tasks whose steps you know open-ended tasks whose steps you can't know
Trade-offs predictable, cheaper, easier to test flexible, but slower, costlier, harder to test

The guide also notes: "Agentic systems often trade latency and cost for better task performance." So start with workflows.

Problem 1: one giant prompt

def llm(prompt):                       # toy LLM: one big prompt, one big guess
    return ("Goa trip plan: train Rs 1450 x 2, hotel Rs 2500 x 2 nights, "
            "scooter Rs 400. Have fun!")

request = "Plan a 2-night Goa trip for 2 people, total under Rs 5000"
print(llm(request))
costs = [1450 * 2, 2500 * 2, 400]      # we add it up ourselves...
print("total: Rs", sum(costs))

Output:

Goa trip plan: train Rs 1450 x 2, hotel Rs 2500 x 2 nights, scooter Rs 400. Have fun!
total: Rs 8300

One call had to choose, price, add up, check and write, and nobody checked the total.

Pattern 1: prompt chaining

Split the job into small steps; each LLM call does one thing and feeds the next. Between steps, a gate checks the result in code.

def llm(step, items):                  # toy LLM: one small job per call
    if step == "costs":
        return [("train", 2900), ("hotel", 5000), ("scooter", 400)]
    if step == "cheaper":
        return [("train", 900), ("hostel", 1800), ("bus", 200)]
    if step == "write":
        return "Goa plan: " + ", ".join(name for name, _ in items)

def total(items): return sum(cost for _, cost in items)
BUDGET = 5000

costs = llm("costs", None)             # step 1: list the costs
print("step 1: Rs", total(costs))
if total(costs) > BUDGET:              # gate: checked in CODE
    costs = llm("cheaper", costs)      # step 2: fix it
    print("step 2: Rs", total(costs))
print("step 3:", llm("write", costs), "| Rs", total(costs))

Output:

step 1: Rs 8300
step 2: Rs 2900
step 3: Goa plan: train, hostel, bus | Rs 2900

Each step is simpler, so the model gets it right more often, and the gate catches what it misses. The cost: more calls, so it's slower (the guide: chaining trades latency for accuracy).

Problem 2: one prompt for every message

def llm(message):                      # one general prompt for everything
    return "Thanks! We value you. See our FAQ."

for msg in ["Where is my refund for PNR 4521?",
            "The app crashes when I pay",
            "Do trains to Goa have WiFi?"]:
    print(f"{msg:34} -> {llm(msg)}")

Output:

Where is my refund for PNR 4521?   -> Thanks! We value you. See our FAQ.
The app crashes when I pay         -> Thanks! We value you. See our FAQ.
Do trains to Goa have WiFi?        -> Thanks! We value you. See our FAQ.

A refund needs the refund tool; a crash needs a support ticket. One prompt trying to be good at everything ends up good at nothing.

Pattern 2: routing

Step 1, a small and cheap call classifies the input. Step 2 sends it to a specialist with its own prompt and tools.

def classify(message):                 # step 1: a small, cheap LLM call
    m = message.lower()
    if "refund" in m: return "refund"
    if "crash" in m or "error" in m: return "tech"
    return "general"

HANDLERS = {                           # step 2: a specialist for each kind
    "refund":  lambda m: "Refund for PNR 4521: Rs 450, sent on 3 Nov",
    "tech":    lambda m: "Ticket #88 opened, logs attached",
    "general": lambda m: "WiFi: see the amenities page",
}
for msg in ["Where is my refund for PNR 4521?",
            "The app crashes when I pay",
            "Do trains to Goa have WiFi?"]:
    kind = classify(msg)
    print(f"{kind:8} -> {HANDLERS[kind](msg)}")

Output:

refund   -> Refund for PNR 4521: Rs 450, sent on 3 Nov
tech     -> Ticket #88 opened, logs attached
general  -> WiFi: see the amenities page

This matches the guide's example: general questions, refund requests and technical support go to different downstream processes. You can also route easy questions to a small, cheap model and hard ones to a bigger model.

Problem 3: slow checks, one by one

Before booking, the agent runs three checks (date, fare, seat); each is one LLM call that takes about a second. One after another they take 3 s (the first output line below); with sectioning they take 1 s.

Pattern 3a: parallelization by sectioning

import time
from concurrent.futures import ThreadPoolExecutor

def check(name):                       # each check = one slow LLM call
    time.sleep(1)
    return f"{name}: OK"

CHECKS = ["date", "fare", "seat"]

start = time.perf_counter()            # one after another
print([check(c) for c in CHECKS], f"{time.perf_counter() - start:.1f}s")

start = time.perf_counter()            # all at the same time
with ThreadPoolExecutor() as pool:
    print(list(pool.map(check, CHECKS)), f"{time.perf_counter() - start:.1f}s")

Output:

['date: OK', 'fare: OK', 'seat: OK'] 3.0s
['date: OK', 'fare: OK', 'seat: OK'] 1.0s

Sectioning splits a job into independent parts and runs them at the same time. ThreadPoolExecutor runs the calls in threads, so their waiting overlaps. It only works when no part needs another part's answer.

Pattern 3b: parallelization by voting

from collections import Counter
CODE = "sql = 'SELECT * FROM users WHERE name = ' + name"

def reviewer(i, code):                 # toy LLM run 3 times: answers vary
    return ["SQL injection!", "looks fine", "SQL injection!"][i]

votes = [reviewer(i, CODE) for i in range(3)]
print("votes :", votes)
answer, count = Counter(votes).most_common(1)[0]
print("result:", answer, f"({count} of {len(votes)})")

Output:

votes : ['SQL injection!', 'looks fine', 'SQL injection!']
result: SQL injection! (2 of 3)

Voting asks the same question several times and combines the answers (Counter(...).most_common(1) takes the majority). One reviewer alone would have said "looks fine". The guide's example: several prompts reviewing code for vulnerabilities. More calls and cost, fewer misses.

Problem 4: fixed steps for an open-ended request

WORKERS = {
    "trains": lambda req: "train 12127, 8 Nov",
    "hotel":  lambda req: "Sea View Hotel, 2 nights",
}
def fixed_pipeline(request):           # the same steps for EVERY request
    return [WORKERS[step](request) for step in ["trains", "hotel"]]

request = "Goa trip with Dadi: she uses a wheelchair, and our dog comes"
for line in fixed_pipeline(request):
    print("-", line)
print("On arrival: 3 floors, no lift, no pets allowed!")

Output:

- train 12127, 8 Nov
- Sea View Hotel, 2 nights
On arrival: 3 floors, no lift, no pets allowed!

The steps were decided before the request was read, so they can't add what this trip needs.

Pattern 4: orchestrator-workers

An orchestrator (a model) reads the request and plans the subtasks at run time; workers do each one; the results are combined.

WORKERS = {
    "trains": lambda req: "train 12127, 8 Nov, step-free coach",
    "hotel":  lambda req: f"Beach Inn: {req}",
    "pets":   lambda req: "Beach Inn allows dogs: pet room booked",
}
def orchestrator(request):             # toy LLM: plans subtasks at RUN time
    plan = [("trains", "")]
    if "wheelchair" in request: plan.append(("hotel", "ground floor, ramp"))
    if "dog" in request:        plan.append(("pets", ""))
    return plan

request = "Goa trip with Dadi: she uses a wheelchair, and our dog comes"
plan = orchestrator(request)
print("plan:", [name for name, _ in plan])
for name, detail in plan:              # workers do the subtasks
    print("-", WORKERS[name](detail))

Output:

plan: ['trains', 'hotel', 'pets']
- train 12127, 8 Nov, step-free coach
- Beach Inn: ground floor, ramp
- Beach Inn allows dogs: pet room booked

The guide's example: coding tools that make changes across many files, where you can't know in advance which files. Difference from parallelization: there, you decide the parts; here, the model decides them.

Pattern 5: evaluator-optimizer

writer: draft ──► evaluator: checks + feedback ──► writer: improves ──► ... ──► done (passes)

One call writes, another evaluates and gives feedback, and they loop until the result passes (you built this as reflection in episode 31). It works best when you can state clearly what "good" means. The guide's example: literary translation, where a critic can point out nuances the first draft missed.

A tiny version with a toy writer (three drafts) and a critic with two clear rules:

DRAFTS = [{"items": ["train", "scooter"], "total": 3300},          # toy writer:
          {"items": ["train", "hotel"], "total": 7900},            # one draft
          {"items": ["train", "hostel", "bus"], "total": 2900}]    # per try

def evaluate(plan, budget):            # the critic: clear rules for "good"
    feedback = []
    if plan["total"] > budget:
        feedback.append(f"over budget by Rs {plan['total'] - budget}")
    if not {"hotel", "hostel"} & set(plan["items"]):
        feedback.append("no place to stay")
    return feedback

for attempt, plan in enumerate(DRAFTS, 1):     # write -> evaluate -> improve
    feedback = evaluate(plan, 5000)
    print(f"draft {attempt}: {plan['items']} ->", feedback or "PASS")
    if not feedback:
        break

Output:

draft 1: ['train', 'scooter'] -> ['no place to stay']
draft 2: ['train', 'hotel'] -> ['over budget by Rs 2900']
draft 3: ['train', 'hostel', 'bus'] -> PASS

In a real system the feedback text goes into the writer's next prompt, so each draft fixes what the critic found.

Which pattern? Start simple

  1. One good LLM call (with retrieval and examples). Often enough.
  2. A workflow (chain, route, parallel) when you know the steps.
  3. An orchestrator or full agent only when the steps can't be known in advance.

The guide: "we recommend finding the simplest solution possible, and only increasing complexity when needed." Prove the need with your evals (episode 36): a more complex design must show a better score.

Pattern Use when Cost
Prompt chaining the task splits into fixed steps more calls, slower
Routing inputs fall into clear categories one extra (cheap) call
Sectioning parts are independent same calls, less waiting
Voting you need more confidence k times the calls
Orchestrator-workers subtasks depend on the input planning call + workers
Evaluator-optimizer clear criteria for "good" exist loops of 2 calls
Agent open-ended, unknown number of steps most expensive, hardest to test

Frameworks: what they give, what they hide

Frameworks are libraries that give you these patterns ready-made. The current version of the guide names examples such as the Claude Agent SDK, Strands Agents SDK by AWS, Rivet and Vellum.

They save work They can hide
calling LLMs the real prompts sent
defining and parsing tools the real responses received
chaining calls together what happens between steps

Problem: a prompt hidden inside a framework

class EasyAgent:                       # a toy "framework": one line to use!
    TEMPLATE = ["You are a travel agent.",          # hidden inside it
                "Be creative, add exciting extras!",
                "Task: {task}"]
    def __init__(self, llm):
        self.llm = llm
    def run(self, task):
        return self.llm([line.format(task=task) for line in self.TEMPLATE])

def llm(prompt):                       # toy LLM: follows the prompt it gets
    if "Be creative, add exciting extras!" in prompt:
        return "Booked: train Rs 450 + scuba Rs 3500 + party pass"
    return "Booked: train Rs 450"

print(EasyAgent(llm).run("book the cheapest Goa train"))

Output:

Booked: train Rs 450 + scuba Rs 3500 + party pass

Asha asked for the cheapest train and got scuba diving and a party pass. The cause is line 3: a template line inside the "framework" that you never wrote and can't see from the outside.

The fix: look at what's really sent

class EasyAgent:
    TEMPLATE = ["You are a travel agent.",
                "Be creative, add exciting extras!",
                "Task: {task}"]
    def __init__(self, llm):
        self.llm = llm
    def run(self, task):
        return self.llm([line.format(task=task) for line in self.TEMPLATE])

def llm(prompt):
    print("prompt really sent:")       # look under the hood
    for line in prompt:
        print("   ", line)
    if "Be creative, add exciting extras!" in prompt:
        return "Booked: train Rs 450 + scuba Rs 3500 + party pass"
    return "Booked: train Rs 450"

print(EasyAgent(llm).run("book the cheapest Goa train"))
print(llm(["You book trains.", "Task: book the cheapest Goa train"]))

Output:

prompt really sent:
    You are a travel agent.
    Be creative, add exciting extras!
    Task: book the cheapest Goa train
Booked: train Rs 450 + scuba Rs 3500 + party pass
prompt really sent:
    You book trains.
    Task: book the cheapest Goa train
Booked: train Rs 450

Printing (or tracing, episode 36) the exact prompt reveals the hidden line; calling the model directly with your own prompt gives the expected result.

The guide's advice: "We suggest that developers start by using LLM APIs directly", since many patterns are a few lines of code (as in this episode), and "If you do use a framework, ensure you understand the underlying code."

Three principles

  1. Simplicity: keep the agent's design simple.
  2. Transparency: show the agent's planning steps (traces, episode 36).
  3. A good agent-computer interface: document and test tools carefully, like a good screen for a human.

Glossary

Term Meaning Example from this episode
Design pattern a reusable way to arrange the parts of a system prompt chaining
Agentic system an app where LLMs use tools and take steps workflows and agents
Workflow LLM calls and tools on code paths you wrote the chain with a gate
Agent the LLM decides its own steps and tools the agent loop (episode 27)
Prompt chaining small LLM steps, each feeding the next costs → cheaper → write
Gate a check in code between steps if total(costs) > BUDGET
Routing classify an input, then send it to a specialist refund / tech / general
Handler the code that deals with one kind of input HANDLERS["refund"]
Parallelization running LLM calls at the same time 3.0 s → 1.0 s
Sectioning splitting a job into independent parts date, fare, seat checks
Voting running the same task several times and combining 2 of 3 say SQL injection
Thread a separate flow of work inside one program ThreadPoolExecutor
Counter Python's tool for counting items Counter(votes)
Orchestrator the model that plans and assigns subtasks orchestrator(request)
Worker a call or function that does one subtask WORKERS["pets"]
Evaluator-optimizer a writer and a critic in a loop reflection, episode 31
Framework a library that provides agent building blocks Claude Agent SDK, Strands, Rivet, Vellum
Abstraction a layer that hides details a framework hiding the prompt
Prompt template a prompt with blanks filled in at run time "Task: {task}"
Critic / evaluator the call that checks a draft against rules evaluate(plan, 5000)
Latency how long a response takes 3.0 s vs 1.0 s
SQL injection user input changing a database query (a security bug) the reviewed line

Common mistakes

Mistake What you see Why Fix
One giant prompt does everything total: Rs 8300 (budget 5000) too many jobs, no check chain small steps with gates
One general prompt for all messages See our FAQ. three times no specialisation route to specialists
Independent calls one by one 3.0s waiting for no reason sectioning in parallel
Parallel parts that depend on each other wrong or missing data part B needed A's answer chain those parts instead
Fixed steps for open-ended tasks no lift, no pets steps decided too early orchestrator-workers
Full agent where a workflow would do slow, costly, hard to test complexity not needed simplest solution first, prove with evals
Framework you don't understand scuba + party pass for "the cheapest train" a hidden template line print/trace the real prompt; start with the API

Mini project: a support desk with three patterns

Build a support desk for a train app: a router (refund / cancel / tech / general), a chain inside the refund handler (look up booking → gate: refund allowed? → write the reply), parallel checks before any refund (booking exists, belongs to the user, under the limit), and a voting step (3 runs) for "is this message abusive?". Log every call as a span (episode 36) and keep refunds behind the guardrails from episode 35.

Practice

1. Add a fourth route: "cancel".

Solution
def classify(message):
    m = message.lower()
    if "refund" in m: return "refund"
    if "cancel" in m: return "cancel"
    if "crash" in m or "error" in m: return "tech"
    return "general"

HANDLERS = {
    "refund":  lambda m: "Refund for PNR 4521: Rs 450, sent on 3 Nov",
    "cancel":  lambda m: "Cancel PNR 4521? Please confirm (yes/no)",
    "tech":    lambda m: "Ticket #88 opened, logs attached",
    "general": lambda m: "WiFi: see the amenities page",
}
for msg in ["Please cancel my Goa ticket", "Where is my refund?"]:
    kind = classify(msg)
    print(f"{kind:8} -> {HANDLERS[kind](msg)}")

Output:

cancel   -> Cancel PNR 4521? Please confirm (yes/no)
refund   -> Refund for PNR 4521: Rs 450, sent on 3 Nov

2. Vote with 5 reviewers, and say "unsure" when no answer has a majority.

Solution
from collections import Counter

def decide(votes):
    answer, count = Counter(votes).most_common(1)[0]
    if count <= len(votes) // 2:
        return "unsure: ask a human"
    return f"{answer} ({count} of {len(votes)})"

print(decide(["SQL injection!", "looks fine", "SQL injection!", "SQL injection!", "looks fine"]))
print(decide(["SQL injection!", "looks fine", "XSS!", "looks fine", "SQL injection!"]))

Output:

SQL injection! (3 of 5)
unsure: ask a human

3. Add a "weather" worker that the orchestrator plans when the request mentions rain or monsoon.

Solution
WORKERS = {
    "trains":  lambda req: "train 12127, 8 Nov, step-free coach",
    "weather": lambda req: "pack umbrellas: check the forecast the day before",
}
def orchestrator(request):
    plan = ["trains"]
    if "rain" in request or "monsoon" in request:
        plan.append("weather")
    return plan

for request in ["Goa trip in the monsoon", "Goa trip in December"]:
    print(request, "->", orchestrator(request))
    for name in orchestrator(request):
        print("  -", WORKERS[name](request))

Output:

Goa trip in the monsoon -> ['trains', 'weather']
  - train 12127, 8 Nov, step-free coach
  - pack umbrellas: check the forecast the day before
Goa trip in December -> ['trains']
  - train 12127, 8 Nov, step-free coach

4. Write an evaluator for the trip plan: it must be under budget and mention a place to stay.

Solution
def evaluate(plan, budget):
    feedback = []
    if plan["total"] > budget:
        feedback.append(f"over budget by Rs {plan['total'] - budget}")
    if not any(item in plan["items"] for item in ("hotel", "hostel", "homestay")):
        feedback.append("no place to stay")
    return feedback or ["PASS"]

drafts = [{"items": ["train", "scooter"], "total": 3300},
          {"items": ["train", "hotel"], "total": 7900},
          {"items": ["train", "hostel", "bus"], "total": 2900}]
for i, plan in enumerate(drafts, 1):
    print(f"draft {i}:", evaluate(plan, 5000))

Output:

draft 1: ['no place to stay']
draft 2: ['over budget by Rs 2900']
draft 3: ['PASS']

Quick quiz 1

Translate a product page into English, Tamil and Bengali; each language is independent. Which pattern fits?

Answer

B: parallelization (sectioning). Three independent jobs with the same steps: run them at the same time and collect the results.

Quick quiz 2

"Fix this bug" in a code base: you don't know which files, or how many steps. Workflow or agent?

Answer

C: an agent or orchestrator. The steps can't be known in advance: the model must read, choose files and loop on the tests, like the coding agent in episode 34.

Interview questions

  1. Workflow vs agent? A workflow runs LLM calls and tools on code paths you wrote; an agent lets the LLM decide its own steps and tools. Prefer workflows when the steps are known.
  2. What is prompt chaining and when is it worth it? Splitting a task into sequential LLM steps with programmatic checks between them; worth it when accuracy matters more than latency.
  3. Routing: benefits? Specialised prompts and tools per category, better quality, and the option to send easy inputs to cheaper models.
  4. Sectioning vs voting? Sectioning runs different independent subtasks in parallel; voting runs the same task several times and combines the answers for confidence.
  5. Orchestrator-workers vs parallelization? In parallelization you fix the subtasks in advance; an orchestrator decides them at run time from the input.
  6. When would you use evaluator-optimizer? When there are clear criteria for good output and feedback measurably improves drafts (translation, code with tests).
  7. Should you use an agent framework? It can save work on LLM calls, tool parsing and chaining, but can hide prompts and responses; start with the API directly and understand any framework you adopt.

Summary

  • Workflows (your code decides) vs agents (the model decides); agentic systems trade latency and cost for performance.
  • Prompt chaining: small steps with gates in code.
  • Routing: classify, then a specialist.
  • Parallelization: sectioning (independent parts at once) and voting (majority of several runs).
  • Orchestrator-workers: subtasks planned at run time.
  • Evaluator-optimizer: writer + critic loop.
  • Find the simplest solution; add complexity only when your evals prove it helps.
  • Frameworks save work but can hide prompts; start with the API and understand the code.

Next: episode 38, Responsible AI: fairness, privacy and honesty.

Sources: Anthropic, "Building effective agents" (anthropic.com/research/building-effective-agents, 19 Dec 2024), fetched 5 Nov 2026; Spring, "Building Effective Agents with Spring AI (Part 1)" (spring.io blog, 21 Jan 2025).