These are the notes for AI episode 23. Until now you typed into a chat box and played with toy models. Here your code talks to a model through an API: you build a request (model, max_tokens, messages), read the response (content blocks, stop_reason, usage), learn the three roles, why the API is stateless, how to keep an API key safe, and what a call costs. Every Python example runs as-is in the SyntaxPilot compiler; outputs are real (Python 3.13). The model in the code is a toy (one word = one token) with the same request and response shape as a real Messages API, so you can practise without an account or a key.
Real-life example: Ramesh's chai stall gets famous
Ramesh gets 500 WhatsApp questions a day ("Is the samosa fresh?", "What does chai cost?"). Copying each one into a chat app and copying the answer back, 500 times a day, is far too slow. The fix: let his code send each question to the model and send the answer back.
Real-life example: a restaurant kitchen
| In a restaurant | Calling an LLM |
|---|---|
| The kitchen window: the only way to reach the cook | API endpoint: a web address your code sends requests to |
| The order slip: table, dish, quantity, in a fixed format | Request: JSON with model, max_tokens, messages |
| The tray and the bill | Response: the answer text plus token usage |
| Your staff card: the kitchen knows who to charge | API key: proves who you are, and you get the bill |
API = application programming interface: a fixed way for one program to talk to another.
The obvious attempt: send a plain string
def llm_api(request): # a toy stand-in for a real LLM API
if not isinstance(request, dict) or "messages" not in request:
return {"type": "error", "error": {"type": "invalid_request_error",
"message": "messages: Field required"}}
return {"type": "message", "content": [{"type": "text", "text": "Hi!"}]}
reply = llm_api("Is the samosa fresh today?") # just send the question
print(reply)
Output:
{'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'messages: Field required'}}
Why it failed: an API is not a chat box. Like the kitchen, it only accepts a request in its fixed format. (This toy API is modelled on a real one: the real API also answers a malformed request with an error instead of an answer.)
The request: an order slip
A real request from Anthropic's documentation for its Messages API:
curl https://api.anthropic.com/v1/messages \
-H "x-api-key: $ANTHROPIC_API_KEY" \
-H "anthropic-version: 2023-06-01" \
-H "content-type: application/json" \
-d '{
"model": "claude-opus-5-5",
"max_tokens": 1024,
"messages": [
{"role": "user", "content": "Hello, Claude"}
]
}'
| Field | What it means |
|---|---|
model |
which model answers, by its exact model id |
max_tokens |
the most tokens the model may write in its answer |
messages |
a list of {"role": ..., "content": ...} |
| headers | x-api-key (your key), anthropic-version, content-type: application/json |
JSON (JavaScript Object Notation) is a text format of keys and values. It looks almost exactly like a Python dictionary, so it is easy to build in code.
The response, field by field
FAQ = {"samosa": "Yes, fresh samosas every 2 hours.",
"chai": "Masala chai is 15 rupees."}
def llm_api(req): # a toy stand-in for a real LLM API
q = req["messages"][-1]["content"]
ans = next((a for k, a in FAQ.items() if k in q.lower()), "No idea.")
out = ans.split()[:req["max_tokens"]] # toy rule: 1 word = 1 token
stop = "end_turn" if out == ans.split() else "max_tokens"
return {"id": "msg_01", "type": "message", "role": "assistant",
"model": req["model"], "stop_reason": stop,
"content": [{"type": "text", "text": " ".join(out)}],
"usage": {"input_tokens": len(q.split()), "output_tokens": len(out)}}
request = {"model": "toy-chai-1", "max_tokens": 100,
"messages": [{"role": "user", "content": "Is the samosa fresh?"}]}
reply = llm_api(request)
for key, value in reply.items():
print(f"{key:12}: {value}")
Output:
id : msg_01
type : message
role : assistant
model : toy-chai-1
stop_reason : end_turn
content : [{'type': 'text', 'text': 'Yes, fresh samosas every 2 hours.'}]
usage : {'input_tokens': 4, 'output_tokens': 6}
| Field | Meaning |
|---|---|
id |
a unique id for this reply |
type |
"message" |
role |
"assistant": the model's side of the chat |
model |
which model answered |
stop_reason |
why it stopped: end_turn = finished naturally |
content |
a list of content blocks; a text block holds the answer |
usage |
input_tokens and output_tokens: the bill |
The real example response from Anthropic's docs:
{
"id": "msg_01XFDUDYJgAACzvnptvVoYEL",
"type": "message",
"role": "assistant",
"content": [{"type": "text", "text": "Hello!"}],
"model": "claude-opus-5-5",
"stop_reason": "end_turn",
"stop_sequence": null,
"usage": {"input_tokens": 12, "output_tokens": 6}
}
Reading the answer
The obvious attempt:
FAQ = {"samosa": "Yes, fresh samosas every 2 hours.",
"chai": "Masala chai is 15 rupees."}
def llm_api(req): # a toy stand-in for a real LLM API
q = req["messages"][-1]["content"]
ans = next((a for k, a in FAQ.items() if k in q.lower()), "No idea.")
out = ans.split()[:req["max_tokens"]] # toy rule: 1 word = 1 token
stop = "end_turn" if out == ans.split() else "max_tokens"
return {"id": "msg_01", "type": "message", "role": "assistant",
"model": req["model"], "stop_reason": stop,
"content": [{"type": "text", "text": " ".join(out)}],
"usage": {"input_tokens": len(q.split()), "output_tokens": len(out)}}
request = {"model": "toy-chai-1", "max_tokens": 100,
"messages": [{"role": "user", "content": "Is the samosa fresh?"}]}
reply = llm_api(request)
for key, value in reply.items():
print(f"{key:12}: {value}")
print(reply["text"])
Output:
id : msg_01
type : message
role : assistant
model : toy-chai-1
stop_reason : end_turn
content : [{'type': 'text', 'text': 'Yes, fresh samosas every 2 hours.'}]
usage : {'input_tokens': 4, 'output_tokens': 6}
Traceback (most recent call last):
File "main.py", line 18, in <module>
print(reply["text"])
~~~~~^^^^^^^^
KeyError: 'text'
Why: there is no top-level text. content is a list (a reply can hold several blocks), and the text is inside the first block:
FAQ = {"samosa": "Yes, fresh samosas every 2 hours.",
"chai": "Masala chai is 15 rupees."}
def llm_api(req): # a toy stand-in for a real LLM API
q = req["messages"][-1]["content"]
ans = next((a for k, a in FAQ.items() if k in q.lower()), "No idea.")
out = ans.split()[:req["max_tokens"]] # toy rule: 1 word = 1 token
stop = "end_turn" if out == ans.split() else "max_tokens"
return {"id": "msg_01", "type": "message", "role": "assistant",
"model": req["model"], "stop_reason": stop,
"content": [{"type": "text", "text": " ".join(out)}],
"usage": {"input_tokens": len(q.split()), "output_tokens": len(out)}}
request = {"model": "toy-chai-1", "max_tokens": 100,
"messages": [{"role": "user", "content": "Is the samosa fresh?"}]}
reply = llm_api(request)
print(reply["content"][0]["text"]) # content is a LIST of blocks
print("stop_reason:", reply["stop_reason"])
Output:
Yes, fresh samosas every 2 hours.
stop_reason: end_turn
max_tokens and stop_reason
FAQ = {"samosa": "Yes, fresh samosas every 2 hours.",
"chai": "Masala chai is 15 rupees."}
def llm_api(req): # a toy stand-in for a real LLM API
q = req["messages"][-1]["content"]
ans = next((a for k, a in FAQ.items() if k in q.lower()), "No idea.")
out = ans.split()[:req["max_tokens"]] # toy rule: 1 word = 1 token
stop = "end_turn" if out == ans.split() else "max_tokens"
return {"id": "msg_01", "type": "message", "role": "assistant",
"model": req["model"], "stop_reason": stop,
"content": [{"type": "text", "text": " ".join(out)}],
"usage": {"input_tokens": len(q.split()), "output_tokens": len(out)}}
request = {"model": "toy-chai-1", "max_tokens": 3,
"messages": [{"role": "user", "content": "Is the samosa fresh?"}]}
reply = llm_api(request)
print(reply["content"][0]["text"])
print("stop_reason:", reply["stop_reason"])
Output:
Yes, fresh samosas
stop_reason: max_tokens
The answer was cut mid-sentence and stop_reason changed to max_tokens. Always check stop_reason: end_turn means the model finished; max_tokens means it hit your limit and the answer is incomplete. Anthropic's docs show a request with "max_tokens": 1 that returns one token and "stop_reason": "max_tokens".
Stateless: the model forgets you
def llm_api(req): # toy: it only knows what is in THIS request
said = " ".join(m["content"] for m in req["messages"])
if "my name?" in req["messages"][-1]["content"]:
if "I am " in said:
return "Your name is " + said.split("I am ")[1].split()[0]
return "Sorry, I don't know your name."
return "Nice to meet you!"
first = {"messages": [{"role": "user", "content": "Hi, I am Ravi."}]}
print("Bot:", llm_api(first))
second = {"messages": [{"role": "user", "content": "What is my name?"}]}
print("Bot:", llm_api(second))
Output:
Bot: Nice to meet you!
Bot: Sorry, I don't know your name.
Why: each call is a new request, and the model only sees what is inside it. Anthropic's docs: "The Messages API is stateless, which means that you always send the full conversational history to the API."
Sending the history
def llm_api(req): # toy: it only knows what is in THIS request
said = " ".join(m["content"] for m in req["messages"])
if "my name?" in req["messages"][-1]["content"]:
if "I am " in said:
return "Your name is " + said.split("I am ")[1].split()[0]
return "Sorry, I don't know your name."
return "Nice to meet you!"
history = [{"role": "user", "content": "Hi, I am Ravi."}]
reply = llm_api({"messages": history})
print("Bot:", reply)
history.append({"role": "assistant", "content": reply})
history.append({"role": "user", "content": "What is my name?"})
print("Bot:", llm_api({"messages": history}))
print("messages sent:", len(history))
Output:
Bot: Nice to meet you!
Bot: Your name is Ravi.
messages sent: 3
You keep the memory yourself: append the model's reply (role assistant) and the next user message, and send the whole list every time. This is how every chat app works underneath. The catch: a long chat means a bigger request (more input tokens) on every turn.
Three roles
| Role | Who | Example |
|---|---|---|
user |
the person asking | "What is my name?" |
assistant |
the model's earlier replies | "Nice to meet you!" |
system |
standing instructions from the developer | "Reply in Hindi." |
In Anthropic's Messages API the system prompt is a separate top-level system field, not the first message.
The system prompt
def llm_api(req): # toy: the system prompt sets the style
q = req["messages"][-1]["content"]
ans = "Masala chai is 15 rupees."
if "Hindi" in req.get("system", ""):
ans = "Masala chai 15 rupaye ki hai."
if "one word" in req.get("system", ""):
ans = "15"
return ans
for system in ["", "Reply in Hindi.", "Reply in one word."]:
req = {"system": system,
"messages": [{"role": "user", "content": "Price of chai?"}]}
print(f"system={system!r:22} -> {llm_api(req)}")
Output:
system='' -> Masala chai is 15 rupees.
system='Reply in Hindi.' -> Masala chai 15 rupaye ki hai.
system='Reply in one word.' -> 15
The user never sees the system prompt, but it shapes every answer: "You are the stall's helper. Be polite. Never promise home delivery." (Prompting basics: episode 20.)
Real-life example: the office ID card
Your office ID card opens the gate and the canteen bills your account. Post a photo of it online and strangers eat on your bill. The fix: block the card, get a new one, keep it private. An API key works the same way: it proves who you are, every call is billed to you, and a leaked key should be revoked and replaced.
Keeping the key safe
import os
key = os.environ.get("ANTHROPIC_API_KEY") # read it, never type it
print("key found:", key is not None)
headers = {"x-api-key": key or "<missing>",
"anthropic-version": "2023-06-01",
"content-type": "application/json"}
url = "https://api.anthropic.com/v1/messages"
print("POST", url)
for name in headers:
print(" ", name)
key = os.environ["ANTHROPIC_API_KEY"]
Output:
key found: False
POST https://api.anthropic.com/v1/messages
x-api-key
anthropic-version
content-type
Traceback (most recent call last):
File "main.py", line 11, in <module>
key = os.environ["ANTHROPIC_API_KEY"]
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
File "<frozen os>", line 717, in __getitem__
KeyError: 'ANTHROPIC_API_KEY'
- Read the key from an environment variable (a setting stored on your computer, outside the code).
os.environ.get(...)returnsNonewhen it is missing;os.environ[...]raisesKeyError, which tells you clearly the key isn't set before you send a request that would fail.
| Hard-coded (wrong) | From the environment (right) |
|---|---|
key = "sk-ant-...secret..." |
key = os.environ["ANTHROPIC_API_KEY"] |
| leaked the day you share or push the code | the secret stays on your machine |
What it costs
reviews = 500 # customer messages per day
input_tokens, output_tokens = 60, 40 # tokens per message (measured)
price_in, price_out = 3, 15 # EXAMPLE prices, $ per million tokens
per_msg = (input_tokens * price_in + output_tokens * price_out) / 1_000_000
print(f"one message : ${per_msg:.5f}")
print(f"per day : ${per_msg * reviews:.2f}")
print(f"per month : ${per_msg * reviews * 30:.2f}")
history_tokens = 60 * 10 # a 10-turn chat resends its history
print("10th turn input tokens:", history_tokens)
Output:
one message : $0.00078
per day : $0.39
per month : $11.70
10th turn input tokens: 600
Cost = tokens × price. Prices are per million tokens and output usually costs more than input. The prices above are made-up examples: check your provider's price page. Because the history is resent, the 10th turn of a chat sends about 10× the input tokens of the first.
The real call: an SDK
An SDK (software development kit) is a library that does the HTTP request, the headers and the JSON for you. Adapted from Anthropic's Python example:
import anthropic
client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY
message = client.messages.create(
model="claude-opus-5-5",
max_tokens=1024,
messages=[{"role": "user", "content": "Hello, Claude"}],
)
print(message.content[0].text)
The same three things: model, max_tokens, messages, and the answer read from the content blocks. Other providers use different field names, but the idea is the same. Settings also differ between models: Anthropic's docs say temperature, top_p and top_k are not supported on its newest models (sending a non-default value returns a 400 error). Always read the docs for the model you use.
Glossary
| Term | Meaning | Example from this episode |
|---|---|---|
| API | a fixed way for programs to talk to each other | the Messages API |
| Endpoint | the web address a request is sent to | https://api.anthropic.com/v1/messages |
| Request | what your code sends | {"model", "max_tokens", "messages"} |
| Response | what comes back | content blocks, stop_reason, usage |
| JSON | a text format of keys and values | {"role": "user"} |
| Model id | the exact name of a model | claude-opus-5-5 |
messages |
the list of turns in the conversation | [{"role": "user", "content": "Hi"}] |
| Role | who is speaking: user, assistant, system |
"role": "assistant" |
| System prompt | standing instructions from the developer | "Reply in one word." |
max_tokens |
the most tokens the model may write | 3 cut the answer |
stop_reason |
why the model stopped | end_turn, max_tokens |
| Content block | one piece of a reply; text is one type | {"type": "text", "text": "Hello!"} |
| Usage | the tokens a call used | input_tokens: 4, output_tokens: 6 |
| Input / output tokens | tokens you sent / tokens it wrote | output usually costs more |
| Stateless | the server keeps no memory between calls | the bot forgot Ravi |
| History | all earlier turns, resent every call | 3 messages in the second call |
| API key | a secret that proves who you are and bills you | ANTHROPIC_API_KEY |
| Environment variable | a setting stored outside the code | os.environ.get("ANTHROPIC_API_KEY") |
| Revoke | cancel a key so it stops working | after a leak |
| HTTP header | extra information sent with a request | x-api-key, content-type |
| SDK | a library that makes API calls for you | pip install anthropic |
Common mistakes
| Code | Exact error | Why | Fix |
|---|---|---|---|
llm_api("Is the samosa fresh?") |
toy API: invalid_request_error, messages: Field required |
a plain string isn't a request | send {"model", "max_tokens", "messages"} |
print(reply["text"]) |
KeyError: 'text' |
the text is inside the content list |
reply["content"][0]["text"] |
os.environ["ANTHROPIC_API_KEY"] with no key set |
KeyError: 'ANTHROPIC_API_KEY' |
the environment variable isn't set | set it on your machine; never paste the key in code |
"max_tokens": 3 |
no error, but stop_reason: max_tokens |
the limit cut the answer | check stop_reason; raise the limit |
| sending only the last message | the bot "forgets" | the API is stateless | send the full history |
Mini project: a WhatsApp helper function
Write ask(history, question) for the toy API from the "The response" section. It should: append the question as a user message, call llm_api, append the reply as an assistant message, print a warning if stop_reason is max_tokens, and keep a running total of input and output tokens. Ask five questions in a row and print the total tokens and the cost at the example prices.
Practice
- Add a third question to the history example ("Tell me my name?"). Does it still remember? How many messages are sent?
- Add
"vada pav"to the toy FAQ and ask about it. - Set
max_tokensto 1. What is thestop_reason? - Work out the cost for 2,000 messages a day.
Solutions
1. Append the reply and a new question, then call again:
history.append({"role": "assistant", "content": "Your name is Ravi."})
history.append({"role": "user", "content": "Tell me my name?"})
print("Bot:", llm_api({"messages": history}))
print("messages sent:", len(history))
Bot: Nice to meet you!
Bot: Your name is Ravi.
Bot: Your name is Ravi.
messages sent: 5
2. Add "vada pav": "Vada pav is 25 rupees." to FAQ and ask "How much is vada pav?":
Vada pav is 25 rupees.
stop_reason: end_turn
3. With "max_tokens": 1:
Yes,
stop_reason: max_tokens
4. With reviews = 2000:
one message : $0.00078
per day : $1.56
per month : $46.80
10th turn input tokens: 600
Quick quiz 1
Ravi gives his name in call 1. In call 2 the bot says "Sorry, I don't know your name." Why? A: max_tokens was too small · B: the API is stateless; call 2 didn't include call 1 · C: the API key was wrong · D: the system prompt was missing
Answer
B. Each request stands alone. Send the full history every time.
Quick quiz 2
The text is "The shop opens at" and stop_reason is max_tokens. Is the answer complete? A: yes, the model finished · B: no, it hit the token limit · C: no, the key expired · D: yes, end_turn means done
Answer
B. max_tokens means the limit cut the answer off; end_turn would mean it finished.
Interview questions
- What goes into an LLM API request? The model id, a maximum number of output tokens, and the messages (role + content), plus headers for authentication and the content type; optionally a system prompt.
- What does "stateless" mean for a chat API, and what follows from it? The server keeps no conversation memory, so the client resends the full history each call; input tokens and cost grow with conversation length.
- Why check
stop_reason? To know whether the answer is complete (end_turn) or was cut off (max_tokens) or ended for another reason, and handle each case. - How do you store API keys safely? In environment variables or a secrets manager, never in source code; rotate or revoke leaked keys.
- How do you estimate the cost of an LLM feature? Measure input and output tokens per call (from
usage), multiply by the per-million-token prices, multiply by expected volume; account for resent history. - What is an SDK and why use it? A client library that handles HTTP, headers, JSON, and errors, so you work with objects like
message.content[0].text.
Summary
- Your code calls a model through an API endpoint with a request:
model,max_tokens,messages. - The response has content blocks (the text), a
stop_reasonandusage(tokens). - Roles: system (your rules), user, assistant.
- The API is stateless: resend the history every time.
- Keep the API key in an environment variable, never in code.
- Cost = tokens × price, and long chats cost more per turn.
- Next episode: RAG, giving AI your own documents.