SyntaxPilot
Courses/AI: Zero to Agents/Lesson 23

AI: Zero to Agents · Lesson 23 of 40 · 12 min

Calling an LLM from code

queued

These are the notes for AI episode 23. Until now you typed into a chat box and played with toy models. Here your code talks to a model through an API: you build a request (model, max_tokens, messages), read the response (content blocks, stop_reason, usage), learn the three roles, why the API is stateless, how to keep an API key safe, and what a call costs. Every Python example runs as-is in the SyntaxPilot compiler; outputs are real (Python 3.13). The model in the code is a toy (one word = one token) with the same request and response shape as a real Messages API, so you can practise without an account or a key.

Real-life example: Ramesh's chai stall gets famous

Ramesh gets 500 WhatsApp questions a day ("Is the samosa fresh?", "What does chai cost?"). Copying each one into a chat app and copying the answer back, 500 times a day, is far too slow. The fix: let his code send each question to the model and send the answer back.

Real-life example: a restaurant kitchen

In a restaurant Calling an LLM
The kitchen window: the only way to reach the cook API endpoint: a web address your code sends requests to
The order slip: table, dish, quantity, in a fixed format Request: JSON with model, max_tokens, messages
The tray and the bill Response: the answer text plus token usage
Your staff card: the kitchen knows who to charge API key: proves who you are, and you get the bill

API = application programming interface: a fixed way for one program to talk to another.

The obvious attempt: send a plain string

def llm_api(request):                # a toy stand-in for a real LLM API
    if not isinstance(request, dict) or "messages" not in request:
        return {"type": "error", "error": {"type": "invalid_request_error",
                "message": "messages: Field required"}}
    return {"type": "message", "content": [{"type": "text", "text": "Hi!"}]}

reply = llm_api("Is the samosa fresh today?")    # just send the question
print(reply)

Output:

{'type': 'error', 'error': {'type': 'invalid_request_error', 'message': 'messages: Field required'}}

Why it failed: an API is not a chat box. Like the kitchen, it only accepts a request in its fixed format. (This toy API is modelled on a real one: the real API also answers a malformed request with an error instead of an answer.)

The request: an order slip

A real request from Anthropic's documentation for its Messages API:

curl https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d '{
    "model": "claude-opus-5-5",
    "max_tokens": 1024,
    "messages": [
      {"role": "user", "content": "Hello, Claude"}
    ]
  }'
Field What it means
model which model answers, by its exact model id
max_tokens the most tokens the model may write in its answer
messages a list of {"role": ..., "content": ...}
headers x-api-key (your key), anthropic-version, content-type: application/json

JSON (JavaScript Object Notation) is a text format of keys and values. It looks almost exactly like a Python dictionary, so it is easy to build in code.

The response, field by field

FAQ = {"samosa": "Yes, fresh samosas every 2 hours.",
       "chai": "Masala chai is 15 rupees."}
def llm_api(req):                # a toy stand-in for a real LLM API
    q = req["messages"][-1]["content"]
    ans = next((a for k, a in FAQ.items() if k in q.lower()), "No idea.")
    out = ans.split()[:req["max_tokens"]]    # toy rule: 1 word = 1 token
    stop = "end_turn" if out == ans.split() else "max_tokens"
    return {"id": "msg_01", "type": "message", "role": "assistant",
        "model": req["model"], "stop_reason": stop,
        "content": [{"type": "text", "text": " ".join(out)}],
        "usage": {"input_tokens": len(q.split()), "output_tokens": len(out)}}

request = {"model": "toy-chai-1", "max_tokens": 100,
           "messages": [{"role": "user", "content": "Is the samosa fresh?"}]}
reply = llm_api(request)
for key, value in reply.items():
    print(f"{key:12}: {value}")

Output:

id          : msg_01
type        : message
role        : assistant
model       : toy-chai-1
stop_reason : end_turn
content     : [{'type': 'text', 'text': 'Yes, fresh samosas every 2 hours.'}]
usage       : {'input_tokens': 4, 'output_tokens': 6}
Field Meaning
id a unique id for this reply
type "message"
role "assistant": the model's side of the chat
model which model answered
stop_reason why it stopped: end_turn = finished naturally
content a list of content blocks; a text block holds the answer
usage input_tokens and output_tokens: the bill

The real example response from Anthropic's docs:

{
  "id": "msg_01XFDUDYJgAACzvnptvVoYEL",
  "type": "message",
  "role": "assistant",
  "content": [{"type": "text", "text": "Hello!"}],
  "model": "claude-opus-5-5",
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {"input_tokens": 12, "output_tokens": 6}
}

Reading the answer

The obvious attempt:

FAQ = {"samosa": "Yes, fresh samosas every 2 hours.",
       "chai": "Masala chai is 15 rupees."}
def llm_api(req):                # a toy stand-in for a real LLM API
    q = req["messages"][-1]["content"]
    ans = next((a for k, a in FAQ.items() if k in q.lower()), "No idea.")
    out = ans.split()[:req["max_tokens"]]    # toy rule: 1 word = 1 token
    stop = "end_turn" if out == ans.split() else "max_tokens"
    return {"id": "msg_01", "type": "message", "role": "assistant",
        "model": req["model"], "stop_reason": stop,
        "content": [{"type": "text", "text": " ".join(out)}],
        "usage": {"input_tokens": len(q.split()), "output_tokens": len(out)}}

request = {"model": "toy-chai-1", "max_tokens": 100,
           "messages": [{"role": "user", "content": "Is the samosa fresh?"}]}
reply = llm_api(request)
for key, value in reply.items():
    print(f"{key:12}: {value}")
print(reply["text"])

Output:

id          : msg_01
type        : message
role        : assistant
model       : toy-chai-1
stop_reason : end_turn
content     : [{'type': 'text', 'text': 'Yes, fresh samosas every 2 hours.'}]
usage       : {'input_tokens': 4, 'output_tokens': 6}
Traceback (most recent call last):
  File "main.py", line 18, in <module>
    print(reply["text"])
          ~~~~~^^^^^^^^
KeyError: 'text'

Why: there is no top-level text. content is a list (a reply can hold several blocks), and the text is inside the first block:

FAQ = {"samosa": "Yes, fresh samosas every 2 hours.",
       "chai": "Masala chai is 15 rupees."}
def llm_api(req):                # a toy stand-in for a real LLM API
    q = req["messages"][-1]["content"]
    ans = next((a for k, a in FAQ.items() if k in q.lower()), "No idea.")
    out = ans.split()[:req["max_tokens"]]    # toy rule: 1 word = 1 token
    stop = "end_turn" if out == ans.split() else "max_tokens"
    return {"id": "msg_01", "type": "message", "role": "assistant",
        "model": req["model"], "stop_reason": stop,
        "content": [{"type": "text", "text": " ".join(out)}],
        "usage": {"input_tokens": len(q.split()), "output_tokens": len(out)}}

request = {"model": "toy-chai-1", "max_tokens": 100,
           "messages": [{"role": "user", "content": "Is the samosa fresh?"}]}
reply = llm_api(request)
print(reply["content"][0]["text"])       # content is a LIST of blocks
print("stop_reason:", reply["stop_reason"])

Output:

Yes, fresh samosas every 2 hours.
stop_reason: end_turn

max_tokens and stop_reason

FAQ = {"samosa": "Yes, fresh samosas every 2 hours.",
       "chai": "Masala chai is 15 rupees."}
def llm_api(req):                # a toy stand-in for a real LLM API
    q = req["messages"][-1]["content"]
    ans = next((a for k, a in FAQ.items() if k in q.lower()), "No idea.")
    out = ans.split()[:req["max_tokens"]]    # toy rule: 1 word = 1 token
    stop = "end_turn" if out == ans.split() else "max_tokens"
    return {"id": "msg_01", "type": "message", "role": "assistant",
        "model": req["model"], "stop_reason": stop,
        "content": [{"type": "text", "text": " ".join(out)}],
        "usage": {"input_tokens": len(q.split()), "output_tokens": len(out)}}

request = {"model": "toy-chai-1", "max_tokens": 3,
           "messages": [{"role": "user", "content": "Is the samosa fresh?"}]}
reply = llm_api(request)
print(reply["content"][0]["text"])
print("stop_reason:", reply["stop_reason"])

Output:

Yes, fresh samosas
stop_reason: max_tokens

The answer was cut mid-sentence and stop_reason changed to max_tokens. Always check stop_reason: end_turn means the model finished; max_tokens means it hit your limit and the answer is incomplete. Anthropic's docs show a request with "max_tokens": 1 that returns one token and "stop_reason": "max_tokens".

Stateless: the model forgets you

def llm_api(req):              # toy: it only knows what is in THIS request
    said = " ".join(m["content"] for m in req["messages"])
    if "my name?" in req["messages"][-1]["content"]:
        if "I am " in said:
            return "Your name is " + said.split("I am ")[1].split()[0]
        return "Sorry, I don't know your name."
    return "Nice to meet you!"

first = {"messages": [{"role": "user", "content": "Hi, I am Ravi."}]}
print("Bot:", llm_api(first))
second = {"messages": [{"role": "user", "content": "What is my name?"}]}
print("Bot:", llm_api(second))

Output:

Bot: Nice to meet you!
Bot: Sorry, I don't know your name.

Why: each call is a new request, and the model only sees what is inside it. Anthropic's docs: "The Messages API is stateless, which means that you always send the full conversational history to the API."

Sending the history

def llm_api(req):              # toy: it only knows what is in THIS request
    said = " ".join(m["content"] for m in req["messages"])
    if "my name?" in req["messages"][-1]["content"]:
        if "I am " in said:
            return "Your name is " + said.split("I am ")[1].split()[0]
        return "Sorry, I don't know your name."
    return "Nice to meet you!"

history = [{"role": "user", "content": "Hi, I am Ravi."}]
reply = llm_api({"messages": history})
print("Bot:", reply)
history.append({"role": "assistant", "content": reply})
history.append({"role": "user", "content": "What is my name?"})
print("Bot:", llm_api({"messages": history}))
print("messages sent:", len(history))

Output:

Bot: Nice to meet you!
Bot: Your name is Ravi.
messages sent: 3

You keep the memory yourself: append the model's reply (role assistant) and the next user message, and send the whole list every time. This is how every chat app works underneath. The catch: a long chat means a bigger request (more input tokens) on every turn.

Three roles

Role Who Example
user the person asking "What is my name?"
assistant the model's earlier replies "Nice to meet you!"
system standing instructions from the developer "Reply in Hindi."

In Anthropic's Messages API the system prompt is a separate top-level system field, not the first message.

The system prompt

def llm_api(req):               # toy: the system prompt sets the style
    q = req["messages"][-1]["content"]
    ans = "Masala chai is 15 rupees."
    if "Hindi" in req.get("system", ""):
        ans = "Masala chai 15 rupaye ki hai."
    if "one word" in req.get("system", ""):
        ans = "15"
    return ans

for system in ["", "Reply in Hindi.", "Reply in one word."]:
    req = {"system": system,
           "messages": [{"role": "user", "content": "Price of chai?"}]}
    print(f"system={system!r:22} -> {llm_api(req)}")

Output:

system=''                     -> Masala chai is 15 rupees.
system='Reply in Hindi.'      -> Masala chai 15 rupaye ki hai.
system='Reply in one word.'   -> 15

The user never sees the system prompt, but it shapes every answer: "You are the stall's helper. Be polite. Never promise home delivery." (Prompting basics: episode 20.)

Real-life example: the office ID card

Your office ID card opens the gate and the canteen bills your account. Post a photo of it online and strangers eat on your bill. The fix: block the card, get a new one, keep it private. An API key works the same way: it proves who you are, every call is billed to you, and a leaked key should be revoked and replaced.

Keeping the key safe

import os
key = os.environ.get("ANTHROPIC_API_KEY")       # read it, never type it
print("key found:", key is not None)
headers = {"x-api-key": key or "<missing>",
           "anthropic-version": "2023-06-01",
           "content-type": "application/json"}
url = "https://api.anthropic.com/v1/messages"
print("POST", url)
for name in headers:
    print(" ", name)
key = os.environ["ANTHROPIC_API_KEY"]

Output:

key found: False
POST https://api.anthropic.com/v1/messages
  x-api-key
  anthropic-version
  content-type
Traceback (most recent call last):
  File "main.py", line 11, in <module>
    key = os.environ["ANTHROPIC_API_KEY"]
          ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "<frozen os>", line 717, in __getitem__
KeyError: 'ANTHROPIC_API_KEY'
  • Read the key from an environment variable (a setting stored on your computer, outside the code).
  • os.environ.get(...) returns None when it is missing; os.environ[...] raises KeyError, which tells you clearly the key isn't set before you send a request that would fail.
Hard-coded (wrong) From the environment (right)
key = "sk-ant-...secret..." key = os.environ["ANTHROPIC_API_KEY"]
leaked the day you share or push the code the secret stays on your machine

What it costs

reviews = 500                       # customer messages per day
input_tokens, output_tokens = 60, 40    # tokens per message (measured)
price_in, price_out = 3, 15         # EXAMPLE prices, $ per million tokens

per_msg = (input_tokens * price_in + output_tokens * price_out) / 1_000_000
print(f"one message : ${per_msg:.5f}")
print(f"per day     : ${per_msg * reviews:.2f}")
print(f"per month   : ${per_msg * reviews * 30:.2f}")
history_tokens = 60 * 10            # a 10-turn chat resends its history
print("10th turn input tokens:", history_tokens)

Output:

one message : $0.00078
per day     : $0.39
per month   : $11.70
10th turn input tokens: 600

Cost = tokens × price. Prices are per million tokens and output usually costs more than input. The prices above are made-up examples: check your provider's price page. Because the history is resent, the 10th turn of a chat sends about 10× the input tokens of the first.

The real call: an SDK

An SDK (software development kit) is a library that does the HTTP request, the headers and the JSON for you. Adapted from Anthropic's Python example:

import anthropic

client = anthropic.Anthropic()          # reads ANTHROPIC_API_KEY
message = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello, Claude"}],
)
print(message.content[0].text)

The same three things: model, max_tokens, messages, and the answer read from the content blocks. Other providers use different field names, but the idea is the same. Settings also differ between models: Anthropic's docs say temperature, top_p and top_k are not supported on its newest models (sending a non-default value returns a 400 error). Always read the docs for the model you use.

Glossary

Term Meaning Example from this episode
API a fixed way for programs to talk to each other the Messages API
Endpoint the web address a request is sent to https://api.anthropic.com/v1/messages
Request what your code sends {"model", "max_tokens", "messages"}
Response what comes back content blocks, stop_reason, usage
JSON a text format of keys and values {"role": "user"}
Model id the exact name of a model claude-opus-5-5
messages the list of turns in the conversation [{"role": "user", "content": "Hi"}]
Role who is speaking: user, assistant, system "role": "assistant"
System prompt standing instructions from the developer "Reply in one word."
max_tokens the most tokens the model may write 3 cut the answer
stop_reason why the model stopped end_turn, max_tokens
Content block one piece of a reply; text is one type {"type": "text", "text": "Hello!"}
Usage the tokens a call used input_tokens: 4, output_tokens: 6
Input / output tokens tokens you sent / tokens it wrote output usually costs more
Stateless the server keeps no memory between calls the bot forgot Ravi
History all earlier turns, resent every call 3 messages in the second call
API key a secret that proves who you are and bills you ANTHROPIC_API_KEY
Environment variable a setting stored outside the code os.environ.get("ANTHROPIC_API_KEY")
Revoke cancel a key so it stops working after a leak
HTTP header extra information sent with a request x-api-key, content-type
SDK a library that makes API calls for you pip install anthropic

Common mistakes

Code Exact error Why Fix
llm_api("Is the samosa fresh?") toy API: invalid_request_error, messages: Field required a plain string isn't a request send {"model", "max_tokens", "messages"}
print(reply["text"]) KeyError: 'text' the text is inside the content list reply["content"][0]["text"]
os.environ["ANTHROPIC_API_KEY"] with no key set KeyError: 'ANTHROPIC_API_KEY' the environment variable isn't set set it on your machine; never paste the key in code
"max_tokens": 3 no error, but stop_reason: max_tokens the limit cut the answer check stop_reason; raise the limit
sending only the last message the bot "forgets" the API is stateless send the full history

Mini project: a WhatsApp helper function

Write ask(history, question) for the toy API from the "The response" section. It should: append the question as a user message, call llm_api, append the reply as an assistant message, print a warning if stop_reason is max_tokens, and keep a running total of input and output tokens. Ask five questions in a row and print the total tokens and the cost at the example prices.

Practice

  1. Add a third question to the history example ("Tell me my name?"). Does it still remember? How many messages are sent?
  2. Add "vada pav" to the toy FAQ and ask about it.
  3. Set max_tokens to 1. What is the stop_reason?
  4. Work out the cost for 2,000 messages a day.
Solutions

1. Append the reply and a new question, then call again:

history.append({"role": "assistant", "content": "Your name is Ravi."})
history.append({"role": "user", "content": "Tell me my name?"})
print("Bot:", llm_api({"messages": history}))
print("messages sent:", len(history))
Bot: Nice to meet you!
Bot: Your name is Ravi.
Bot: Your name is Ravi.
messages sent: 5

2. Add "vada pav": "Vada pav is 25 rupees." to FAQ and ask "How much is vada pav?":

Vada pav is 25 rupees.
stop_reason: end_turn

3. With "max_tokens": 1:

Yes,
stop_reason: max_tokens

4. With reviews = 2000:

one message : $0.00078
per day     : $1.56
per month   : $46.80
10th turn input tokens: 600

Quick quiz 1

Ravi gives his name in call 1. In call 2 the bot says "Sorry, I don't know your name." Why? A: max_tokens was too small · B: the API is stateless; call 2 didn't include call 1 · C: the API key was wrong · D: the system prompt was missing

Answer

B. Each request stands alone. Send the full history every time.

Quick quiz 2

The text is "The shop opens at" and stop_reason is max_tokens. Is the answer complete? A: yes, the model finished · B: no, it hit the token limit · C: no, the key expired · D: yes, end_turn means done

Answer

B. max_tokens means the limit cut the answer off; end_turn would mean it finished.

Interview questions

  1. What goes into an LLM API request? The model id, a maximum number of output tokens, and the messages (role + content), plus headers for authentication and the content type; optionally a system prompt.
  2. What does "stateless" mean for a chat API, and what follows from it? The server keeps no conversation memory, so the client resends the full history each call; input tokens and cost grow with conversation length.
  3. Why check stop_reason? To know whether the answer is complete (end_turn) or was cut off (max_tokens) or ended for another reason, and handle each case.
  4. How do you store API keys safely? In environment variables or a secrets manager, never in source code; rotate or revoke leaked keys.
  5. How do you estimate the cost of an LLM feature? Measure input and output tokens per call (from usage), multiply by the per-million-token prices, multiply by expected volume; account for resent history.
  6. What is an SDK and why use it? A client library that handles HTTP, headers, JSON, and errors, so you work with objects like message.content[0].text.

Summary

  • Your code calls a model through an API endpoint with a request: model, max_tokens, messages.
  • The response has content blocks (the text), a stop_reason and usage (tokens).
  • Roles: system (your rules), user, assistant.
  • The API is stateless: resend the history every time.
  • Keep the API key in an environment variable, never in code.
  • Cost = tokens × price, and long chats cost more per turn.
  • Next episode: RAG, giving AI your own documents.