These are the notes for AI episode 37. Real teams rarely build one huge prompt or one all-powerful agent. They arrange LLM calls in a few patterns that come up again and again. Anthropic's guide "Building effective agents" (19 December 2024) names five: prompt chaining, routing, parallelization, orchestrator-workers and evaluator-optimizer. In this episode you invent each one by watching the simple way fail on screen, then learn when a framework helps and when it hurts. Every Python example runs as-is in the SyntaxPilot compiler; outputs are real (Python 3.13; timings rounded to 0.1 s). The LLMs are toys (if-rules and fixed lists) standing in for real model calls.
Real-life example: a wedding caterer's kitchen
| In the kitchen | Pattern |
|---|---|
| Chop, cook, taste, serve: one station after another | prompt chaining: steps with checks between |
| The head waiter sends sweets to the halwai, rotis to the tandoor | routing: classify, then a specialist |
| Five cooks roll rotis together; three tasters vote on a new dish | parallelization: sectioning and voting |
| The head chef plans today's work and assigns cooks | orchestrator-workers: subtasks decided at run time |
Workflows vs agents
The guide splits "agentic systems" into two kinds:
| Workflow | Agent | |
|---|---|---|
| Who decides the next step? | your code | the model |
| Definition (Anthropic) | "LLMs and tools are orchestrated through predefined code paths" | LLMs "dynamically direct their own processes and tool usage" |
| Good for | tasks whose steps you know | open-ended tasks whose steps you can't know |
| Trade-offs | predictable, cheaper, easier to test | flexible, but slower, costlier, harder to test |
The guide also notes: "Agentic systems often trade latency and cost for better task performance." So start with workflows.
Problem 1: one giant prompt
def llm(prompt): # toy LLM: one big prompt, one big guess
return ("Goa trip plan: train Rs 1450 x 2, hotel Rs 2500 x 2 nights, "
"scooter Rs 400. Have fun!")
request = "Plan a 2-night Goa trip for 2 people, total under Rs 5000"
print(llm(request))
costs = [1450 * 2, 2500 * 2, 400] # we add it up ourselves...
print("total: Rs", sum(costs))
Output:
Goa trip plan: train Rs 1450 x 2, hotel Rs 2500 x 2 nights, scooter Rs 400. Have fun!
total: Rs 8300
One call had to choose, price, add up, check and write, and nobody checked the total.
Pattern 1: prompt chaining
Split the job into small steps; each LLM call does one thing and feeds the next. Between steps, a gate checks the result in code.
def llm(step, items): # toy LLM: one small job per call
if step == "costs":
return [("train", 2900), ("hotel", 5000), ("scooter", 400)]
if step == "cheaper":
return [("train", 900), ("hostel", 1800), ("bus", 200)]
if step == "write":
return "Goa plan: " + ", ".join(name for name, _ in items)
def total(items): return sum(cost for _, cost in items)
BUDGET = 5000
costs = llm("costs", None) # step 1: list the costs
print("step 1: Rs", total(costs))
if total(costs) > BUDGET: # gate: checked in CODE
costs = llm("cheaper", costs) # step 2: fix it
print("step 2: Rs", total(costs))
print("step 3:", llm("write", costs), "| Rs", total(costs))
Output:
step 1: Rs 8300
step 2: Rs 2900
step 3: Goa plan: train, hostel, bus | Rs 2900
Each step is simpler, so the model gets it right more often, and the gate catches what it misses. The cost: more calls, so it's slower (the guide: chaining trades latency for accuracy).
Problem 2: one prompt for every message
def llm(message): # one general prompt for everything
return "Thanks! We value you. See our FAQ."
for msg in ["Where is my refund for PNR 4521?",
"The app crashes when I pay",
"Do trains to Goa have WiFi?"]:
print(f"{msg:34} -> {llm(msg)}")
Output:
Where is my refund for PNR 4521? -> Thanks! We value you. See our FAQ.
The app crashes when I pay -> Thanks! We value you. See our FAQ.
Do trains to Goa have WiFi? -> Thanks! We value you. See our FAQ.
A refund needs the refund tool; a crash needs a support ticket. One prompt trying to be good at everything ends up good at nothing.
Pattern 2: routing
Step 1, a small and cheap call classifies the input. Step 2 sends it to a specialist with its own prompt and tools.
def classify(message): # step 1: a small, cheap LLM call
m = message.lower()
if "refund" in m: return "refund"
if "crash" in m or "error" in m: return "tech"
return "general"
HANDLERS = { # step 2: a specialist for each kind
"refund": lambda m: "Refund for PNR 4521: Rs 450, sent on 3 Nov",
"tech": lambda m: "Ticket #88 opened, logs attached",
"general": lambda m: "WiFi: see the amenities page",
}
for msg in ["Where is my refund for PNR 4521?",
"The app crashes when I pay",
"Do trains to Goa have WiFi?"]:
kind = classify(msg)
print(f"{kind:8} -> {HANDLERS[kind](msg)}")
Output:
refund -> Refund for PNR 4521: Rs 450, sent on 3 Nov
tech -> Ticket #88 opened, logs attached
general -> WiFi: see the amenities page
This matches the guide's example: general questions, refund requests and technical support go to different downstream processes. You can also route easy questions to a small, cheap model and hard ones to a bigger model.
Problem 3: slow checks, one by one
Before booking, the agent runs three checks (date, fare, seat); each is one LLM call that takes about a second. One after another they take 3 s (the first output line below); with sectioning they take 1 s.
Pattern 3a: parallelization by sectioning
import time
from concurrent.futures import ThreadPoolExecutor
def check(name): # each check = one slow LLM call
time.sleep(1)
return f"{name}: OK"
CHECKS = ["date", "fare", "seat"]
start = time.perf_counter() # one after another
print([check(c) for c in CHECKS], f"{time.perf_counter() - start:.1f}s")
start = time.perf_counter() # all at the same time
with ThreadPoolExecutor() as pool:
print(list(pool.map(check, CHECKS)), f"{time.perf_counter() - start:.1f}s")
Output:
['date: OK', 'fare: OK', 'seat: OK'] 3.0s
['date: OK', 'fare: OK', 'seat: OK'] 1.0s
Sectioning splits a job into independent parts and runs them at the same time. ThreadPoolExecutor runs the calls in threads, so their waiting overlaps. It only works when no part needs another part's answer.
Pattern 3b: parallelization by voting
from collections import Counter
CODE = "sql = 'SELECT * FROM users WHERE name = ' + name"
def reviewer(i, code): # toy LLM run 3 times: answers vary
return ["SQL injection!", "looks fine", "SQL injection!"][i]
votes = [reviewer(i, CODE) for i in range(3)]
print("votes :", votes)
answer, count = Counter(votes).most_common(1)[0]
print("result:", answer, f"({count} of {len(votes)})")
Output:
votes : ['SQL injection!', 'looks fine', 'SQL injection!']
result: SQL injection! (2 of 3)
Voting asks the same question several times and combines the answers (Counter(...).most_common(1) takes the majority). One reviewer alone would have said "looks fine". The guide's example: several prompts reviewing code for vulnerabilities. More calls and cost, fewer misses.
Problem 4: fixed steps for an open-ended request
WORKERS = {
"trains": lambda req: "train 12127, 8 Nov",
"hotel": lambda req: "Sea View Hotel, 2 nights",
}
def fixed_pipeline(request): # the same steps for EVERY request
return [WORKERS[step](request) for step in ["trains", "hotel"]]
request = "Goa trip with Dadi: she uses a wheelchair, and our dog comes"
for line in fixed_pipeline(request):
print("-", line)
print("On arrival: 3 floors, no lift, no pets allowed!")
Output:
- train 12127, 8 Nov
- Sea View Hotel, 2 nights
On arrival: 3 floors, no lift, no pets allowed!
The steps were decided before the request was read, so they can't add what this trip needs.
Pattern 4: orchestrator-workers
An orchestrator (a model) reads the request and plans the subtasks at run time; workers do each one; the results are combined.
WORKERS = {
"trains": lambda req: "train 12127, 8 Nov, step-free coach",
"hotel": lambda req: f"Beach Inn: {req}",
"pets": lambda req: "Beach Inn allows dogs: pet room booked",
}
def orchestrator(request): # toy LLM: plans subtasks at RUN time
plan = [("trains", "")]
if "wheelchair" in request: plan.append(("hotel", "ground floor, ramp"))
if "dog" in request: plan.append(("pets", ""))
return plan
request = "Goa trip with Dadi: she uses a wheelchair, and our dog comes"
plan = orchestrator(request)
print("plan:", [name for name, _ in plan])
for name, detail in plan: # workers do the subtasks
print("-", WORKERS[name](detail))
Output:
plan: ['trains', 'hotel', 'pets']
- train 12127, 8 Nov, step-free coach
- Beach Inn: ground floor, ramp
- Beach Inn allows dogs: pet room booked
The guide's example: coding tools that make changes across many files, where you can't know in advance which files. Difference from parallelization: there, you decide the parts; here, the model decides them.
Pattern 5: evaluator-optimizer
writer: draft ──► evaluator: checks + feedback ──► writer: improves ──► ... ──► done (passes)
One call writes, another evaluates and gives feedback, and they loop until the result passes (you built this as reflection in episode 31). It works best when you can state clearly what "good" means. The guide's example: literary translation, where a critic can point out nuances the first draft missed.
A tiny version with a toy writer (three drafts) and a critic with two clear rules:
DRAFTS = [{"items": ["train", "scooter"], "total": 3300}, # toy writer:
{"items": ["train", "hotel"], "total": 7900}, # one draft
{"items": ["train", "hostel", "bus"], "total": 2900}] # per try
def evaluate(plan, budget): # the critic: clear rules for "good"
feedback = []
if plan["total"] > budget:
feedback.append(f"over budget by Rs {plan['total'] - budget}")
if not {"hotel", "hostel"} & set(plan["items"]):
feedback.append("no place to stay")
return feedback
for attempt, plan in enumerate(DRAFTS, 1): # write -> evaluate -> improve
feedback = evaluate(plan, 5000)
print(f"draft {attempt}: {plan['items']} ->", feedback or "PASS")
if not feedback:
break
Output:
draft 1: ['train', 'scooter'] -> ['no place to stay']
draft 2: ['train', 'hotel'] -> ['over budget by Rs 2900']
draft 3: ['train', 'hostel', 'bus'] -> PASS
In a real system the feedback text goes into the writer's next prompt, so each draft fixes what the critic found.
Which pattern? Start simple
- One good LLM call (with retrieval and examples). Often enough.
- A workflow (chain, route, parallel) when you know the steps.
- An orchestrator or full agent only when the steps can't be known in advance.
The guide: "we recommend finding the simplest solution possible, and only increasing complexity when needed." Prove the need with your evals (episode 36): a more complex design must show a better score.
| Pattern | Use when | Cost |
|---|---|---|
| Prompt chaining | the task splits into fixed steps | more calls, slower |
| Routing | inputs fall into clear categories | one extra (cheap) call |
| Sectioning | parts are independent | same calls, less waiting |
| Voting | you need more confidence | k times the calls |
| Orchestrator-workers | subtasks depend on the input | planning call + workers |
| Evaluator-optimizer | clear criteria for "good" exist | loops of 2 calls |
| Agent | open-ended, unknown number of steps | most expensive, hardest to test |
Frameworks: what they give, what they hide
Frameworks are libraries that give you these patterns ready-made. The current version of the guide names examples such as the Claude Agent SDK, Strands Agents SDK by AWS, Rivet and Vellum.
| They save work | They can hide |
|---|---|
| calling LLMs | the real prompts sent |
| defining and parsing tools | the real responses received |
| chaining calls together | what happens between steps |
Problem: a prompt hidden inside a framework
class EasyAgent: # a toy "framework": one line to use!
TEMPLATE = ["You are a travel agent.", # hidden inside it
"Be creative, add exciting extras!",
"Task: {task}"]
def __init__(self, llm):
self.llm = llm
def run(self, task):
return self.llm([line.format(task=task) for line in self.TEMPLATE])
def llm(prompt): # toy LLM: follows the prompt it gets
if "Be creative, add exciting extras!" in prompt:
return "Booked: train Rs 450 + scuba Rs 3500 + party pass"
return "Booked: train Rs 450"
print(EasyAgent(llm).run("book the cheapest Goa train"))
Output:
Booked: train Rs 450 + scuba Rs 3500 + party pass
Asha asked for the cheapest train and got scuba diving and a party pass. The cause is line 3: a template line inside the "framework" that you never wrote and can't see from the outside.
The fix: look at what's really sent
class EasyAgent:
TEMPLATE = ["You are a travel agent.",
"Be creative, add exciting extras!",
"Task: {task}"]
def __init__(self, llm):
self.llm = llm
def run(self, task):
return self.llm([line.format(task=task) for line in self.TEMPLATE])
def llm(prompt):
print("prompt really sent:") # look under the hood
for line in prompt:
print(" ", line)
if "Be creative, add exciting extras!" in prompt:
return "Booked: train Rs 450 + scuba Rs 3500 + party pass"
return "Booked: train Rs 450"
print(EasyAgent(llm).run("book the cheapest Goa train"))
print(llm(["You book trains.", "Task: book the cheapest Goa train"]))
Output:
prompt really sent:
You are a travel agent.
Be creative, add exciting extras!
Task: book the cheapest Goa train
Booked: train Rs 450 + scuba Rs 3500 + party pass
prompt really sent:
You book trains.
Task: book the cheapest Goa train
Booked: train Rs 450
Printing (or tracing, episode 36) the exact prompt reveals the hidden line; calling the model directly with your own prompt gives the expected result.
The guide's advice: "We suggest that developers start by using LLM APIs directly", since many patterns are a few lines of code (as in this episode), and "If you do use a framework, ensure you understand the underlying code."
Three principles
- Simplicity: keep the agent's design simple.
- Transparency: show the agent's planning steps (traces, episode 36).
- A good agent-computer interface: document and test tools carefully, like a good screen for a human.
Glossary
| Term | Meaning | Example from this episode |
|---|---|---|
| Design pattern | a reusable way to arrange the parts of a system | prompt chaining |
| Agentic system | an app where LLMs use tools and take steps | workflows and agents |
| Workflow | LLM calls and tools on code paths you wrote | the chain with a gate |
| Agent | the LLM decides its own steps and tools | the agent loop (episode 27) |
| Prompt chaining | small LLM steps, each feeding the next | costs → cheaper → write |
| Gate | a check in code between steps | if total(costs) > BUDGET |
| Routing | classify an input, then send it to a specialist | refund / tech / general |
| Handler | the code that deals with one kind of input | HANDLERS["refund"] |
| Parallelization | running LLM calls at the same time | 3.0 s → 1.0 s |
| Sectioning | splitting a job into independent parts | date, fare, seat checks |
| Voting | running the same task several times and combining | 2 of 3 say SQL injection |
| Thread | a separate flow of work inside one program | ThreadPoolExecutor |
Counter |
Python's tool for counting items | Counter(votes) |
| Orchestrator | the model that plans and assigns subtasks | orchestrator(request) |
| Worker | a call or function that does one subtask | WORKERS["pets"] |
| Evaluator-optimizer | a writer and a critic in a loop | reflection, episode 31 |
| Framework | a library that provides agent building blocks | Claude Agent SDK, Strands, Rivet, Vellum |
| Abstraction | a layer that hides details | a framework hiding the prompt |
| Prompt template | a prompt with blanks filled in at run time | "Task: {task}" |
| Critic / evaluator | the call that checks a draft against rules | evaluate(plan, 5000) |
| Latency | how long a response takes | 3.0 s vs 1.0 s |
| SQL injection | user input changing a database query (a security bug) | the reviewed line |
Common mistakes
| Mistake | What you see | Why | Fix |
|---|---|---|---|
| One giant prompt does everything | total: Rs 8300 (budget 5000) |
too many jobs, no check | chain small steps with gates |
| One general prompt for all messages | See our FAQ. three times |
no specialisation | route to specialists |
| Independent calls one by one | 3.0s |
waiting for no reason | sectioning in parallel |
| Parallel parts that depend on each other | wrong or missing data | part B needed A's answer | chain those parts instead |
| Fixed steps for open-ended tasks | no lift, no pets | steps decided too early | orchestrator-workers |
| Full agent where a workflow would do | slow, costly, hard to test | complexity not needed | simplest solution first, prove with evals |
| Framework you don't understand | scuba + party pass for "the cheapest train" | a hidden template line | print/trace the real prompt; start with the API |
Mini project: a support desk with three patterns
Build a support desk for a train app: a router (refund / cancel / tech / general), a chain inside the refund handler (look up booking → gate: refund allowed? → write the reply), parallel checks before any refund (booking exists, belongs to the user, under the limit), and a voting step (3 runs) for "is this message abusive?". Log every call as a span (episode 36) and keep refunds behind the guardrails from episode 35.
Practice
1. Add a fourth route: "cancel".
Solution
def classify(message):
m = message.lower()
if "refund" in m: return "refund"
if "cancel" in m: return "cancel"
if "crash" in m or "error" in m: return "tech"
return "general"
HANDLERS = {
"refund": lambda m: "Refund for PNR 4521: Rs 450, sent on 3 Nov",
"cancel": lambda m: "Cancel PNR 4521? Please confirm (yes/no)",
"tech": lambda m: "Ticket #88 opened, logs attached",
"general": lambda m: "WiFi: see the amenities page",
}
for msg in ["Please cancel my Goa ticket", "Where is my refund?"]:
kind = classify(msg)
print(f"{kind:8} -> {HANDLERS[kind](msg)}")
Output:
cancel -> Cancel PNR 4521? Please confirm (yes/no)
refund -> Refund for PNR 4521: Rs 450, sent on 3 Nov
2. Vote with 5 reviewers, and say "unsure" when no answer has a majority.
Solution
from collections import Counter
def decide(votes):
answer, count = Counter(votes).most_common(1)[0]
if count <= len(votes) // 2:
return "unsure: ask a human"
return f"{answer} ({count} of {len(votes)})"
print(decide(["SQL injection!", "looks fine", "SQL injection!", "SQL injection!", "looks fine"]))
print(decide(["SQL injection!", "looks fine", "XSS!", "looks fine", "SQL injection!"]))
Output:
SQL injection! (3 of 5)
unsure: ask a human
3. Add a "weather" worker that the orchestrator plans when the request mentions rain or monsoon.
Solution
WORKERS = {
"trains": lambda req: "train 12127, 8 Nov, step-free coach",
"weather": lambda req: "pack umbrellas: check the forecast the day before",
}
def orchestrator(request):
plan = ["trains"]
if "rain" in request or "monsoon" in request:
plan.append("weather")
return plan
for request in ["Goa trip in the monsoon", "Goa trip in December"]:
print(request, "->", orchestrator(request))
for name in orchestrator(request):
print(" -", WORKERS[name](request))
Output:
Goa trip in the monsoon -> ['trains', 'weather']
- train 12127, 8 Nov, step-free coach
- pack umbrellas: check the forecast the day before
Goa trip in December -> ['trains']
- train 12127, 8 Nov, step-free coach
4. Write an evaluator for the trip plan: it must be under budget and mention a place to stay.
Solution
def evaluate(plan, budget):
feedback = []
if plan["total"] > budget:
feedback.append(f"over budget by Rs {plan['total'] - budget}")
if not any(item in plan["items"] for item in ("hotel", "hostel", "homestay")):
feedback.append("no place to stay")
return feedback or ["PASS"]
drafts = [{"items": ["train", "scooter"], "total": 3300},
{"items": ["train", "hotel"], "total": 7900},
{"items": ["train", "hostel", "bus"], "total": 2900}]
for i, plan in enumerate(drafts, 1):
print(f"draft {i}:", evaluate(plan, 5000))
Output:
draft 1: ['no place to stay']
draft 2: ['over budget by Rs 2900']
draft 3: ['PASS']
Quick quiz 1
Translate a product page into English, Tamil and Bengali; each language is independent. Which pattern fits?
Answer
B: parallelization (sectioning). Three independent jobs with the same steps: run them at the same time and collect the results.
Quick quiz 2
"Fix this bug" in a code base: you don't know which files, or how many steps. Workflow or agent?
Answer
C: an agent or orchestrator. The steps can't be known in advance: the model must read, choose files and loop on the tests, like the coding agent in episode 34.
Interview questions
- Workflow vs agent? A workflow runs LLM calls and tools on code paths you wrote; an agent lets the LLM decide its own steps and tools. Prefer workflows when the steps are known.
- What is prompt chaining and when is it worth it? Splitting a task into sequential LLM steps with programmatic checks between them; worth it when accuracy matters more than latency.
- Routing: benefits? Specialised prompts and tools per category, better quality, and the option to send easy inputs to cheaper models.
- Sectioning vs voting? Sectioning runs different independent subtasks in parallel; voting runs the same task several times and combines the answers for confidence.
- Orchestrator-workers vs parallelization? In parallelization you fix the subtasks in advance; an orchestrator decides them at run time from the input.
- When would you use evaluator-optimizer? When there are clear criteria for good output and feedback measurably improves drafts (translation, code with tests).
- Should you use an agent framework? It can save work on LLM calls, tool parsing and chaining, but can hide prompts and responses; start with the API directly and understand any framework you adopt.
Summary
- Workflows (your code decides) vs agents (the model decides); agentic systems trade latency and cost for performance.
- Prompt chaining: small steps with gates in code.
- Routing: classify, then a specialist.
- Parallelization: sectioning (independent parts at once) and voting (majority of several runs).
- Orchestrator-workers: subtasks planned at run time.
- Evaluator-optimizer: writer + critic loop.
- Find the simplest solution; add complexity only when your evals prove it helps.
- Frameworks save work but can hide prompts; start with the API and understand the code.
Next: episode 38, Responsible AI: fairness, privacy and honesty.
Sources: Anthropic, "Building effective agents" (anthropic.com/research/building-effective-agents, 19 Dec 2024), fetched 5 Nov 2026; Spring, "Building Effective Agents with Spring AI (Part 1)" (spring.io blog, 21 Jan 2025).