These are the notes for AI episode 27. Last time you invented an AI agent: a model, plus tools, plus a loop, working towards a goal. Today we open the loop. You'll see why a one-shot answer fails on multi-step tasks, build the think → act → observe loop, discover that the model is stateless so the loop must resend the message history, watch that history grow, add thoughts to get a ReAct trace, read stop_reason (tool_use vs end_turn), and turn tool errors into observations so the agent can fix its own mistakes. Every Python example runs as-is in the SyntaxPilot compiler; outputs and errors are real (Python 3.13). The models are toys (if-rules) with the same shape as a real model API.
Real-life example: Neha cooks biryani for the first time
| In the kitchen | In an agent |
|---|---|
| Goal: biryani for six friends | the user's goal |
| Throw everything in at once, no tasting | a one-shot answer: no feedback |
| "Needs salt" → add a little → taste | think → act → observe |
| Taste again, adjust again | repeat: each result is new information |
| Stop when it tastes right, or call Mummy | stopping conditions and a human hand-off |
Problem 1: one shot from memory
The question: 3 chai and 2 samosa, with 5% GST: how much? A model with no tools answers in one go, from memory.
def model(question): # toy LLM, no tools: one guess
return "3 chai + 2 samosa with GST: about Rs 70." # old prices!
print(model("Cost of 3 chai and 2 samosa, with 5% GST?"))
PRICES = {"chai": 15, "samosa": 20} # today's real menu
print("Real answer:", (3 * PRICES["chai"] + 2 * PRICES["samosa"]) * 1.05)
Output:
3 chai + 2 samosa with GST: about Rs 70.
Real answer: 89.25
The model says about Rs 70; the real answer is Rs 89.25. It answered from old knowledge, in one step. The task actually needs several steps (chai price → samosa price → add GST), and each step depends on the previous result.
The agent loop
+-------+ +------+ +---------+
-->| THINK | --> | ACT | --> | OBSERVE |---+
+-------+ +------+ +---------+ |
what next? call a tool read result |
^-------------------------------------+
repeat until the goal is met
This pattern is called ReAct (Reasoning + Acting), from Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models (arXiv 2210.03629, first submitted 6 Oct 2022). The model interleaves reasoning traces (thoughts) with actions (tool calls). The paper reports that reasoning helps the model plan, track and update its actions, while actions let it gather outside information; using a simple Wikipedia API, ReAct reduced hallucination and error propagation on question answering (HotpotQA) and fact checking (FEVER) compared with chain-of-thought alone, and on the ALFWorld and WebShop benchmarks it beat imitation and reinforcement-learning methods by 34% and 10% absolute success rate with only one or two examples.
Problem 2: the loop forgets
Two tools (price, add_gst) and a toy model that decides the next step from the tool results it is sent. We send it only the question each time:
PRICES = {"chai": 15, "samosa": 20}
def price(item): return PRICES[item]
def add_gst(amount): return round(amount * 1.05, 2)
TOOLS = {"price": price, "add_gst": add_gst}
def use(tool, x): return {"stop_reason": "tool_use", "tool": tool, "input": x}
def model(messages): # toy LLM: reads what you SEND it
seen = [m["content"] for m in messages if m["role"] == "tool"]
if len(seen) == 0: return use("price", "chai")
if len(seen) == 1: return use("price", "samosa")
if len(seen) == 2: return use("add_gst", 3 * seen[0] + 2 * seen[1])
return {"stop_reason": "end_turn", "text": f"Total with GST: Rs {seen[2]}"}
question = {"role": "user", "content": "3 chai + 2 samosa, with GST?"}
for step in range(1, 6): # at most 5 steps
reply = model([question]) # sends ONLY the question
if reply["stop_reason"] == "end_turn":
print(reply["text"]); break
result = TOOLS[reply["tool"]](reply["input"])
print(f"step {step}: {reply['tool']}({reply['input']}) -> {result}")
else:
print("Stopped: 5 steps and no answer.")
Output:
step 1: price(chai) -> 15
step 2: price(chai) -> 15
step 3: price(chai) -> 15
step 4: price(chai) -> 15
step 5: price(chai) -> 15
Stopped: 5 steps and no answer.
price(chai) five times, then no answer. The model API is stateless (episode 23): it remembers nothing between calls. If you send only the question, every call looks like the first call.
The fix: keep a message history
After every step, append what the model asked for and what the tool returned, and send the whole list every time.
PRICES = {"chai": 15, "samosa": 20}
def price(item): return PRICES[item]
def add_gst(amount): return round(amount * 1.05, 2)
TOOLS = {"price": price, "add_gst": add_gst}
def use(tool, x): return {"stop_reason": "tool_use", "tool": tool, "input": x}
def model(messages): # toy LLM: reads what you SEND it
seen = [m["content"] for m in messages if m["role"] == "tool"]
if len(seen) == 0: return use("price", "chai")
if len(seen) == 1: return use("price", "samosa")
if len(seen) == 2: return use("add_gst", 3 * seen[0] + 2 * seen[1])
return {"stop_reason": "end_turn", "text": f"Total with GST: Rs {seen[2]}"}
messages = [{"role": "user", "content": "3 chai + 2 samosa, with GST?"}]
for step in range(1, 6):
print(f"call {step}: model reads {len(messages)} messages")
reply = model(messages) # sends the WHOLE history
if reply["stop_reason"] == "end_turn":
print(reply["text"]); break
result = TOOLS[reply["tool"]](reply["input"])
print(f" {reply['tool']}({reply['input']}) -> {result}")
messages.append({"role": "assistant", **reply}) # what it asked
messages.append({"role": "tool", "content": result}) # what happened
Output:
call 1: model reads 1 messages
price(chai) -> 15
call 2: model reads 3 messages
price(samosa) -> 20
call 3: model reads 5 messages
add_gst(85) -> 89.25
call 4: model reads 7 messages
Total with GST: Rs 89.25
The messages list is the agent's state: its short-term memory of this task. Notice the counts: 1, 3, 5, 7 messages. Every step resends everything, so the context (and the token bill, and the wait) grows with every step.
| Step | Messages sent | Why |
|---|---|---|
| 1 | 1 | the question |
| 2 | 3 | + tool call + chai result |
| 3 | 5 | + tool call + samosa result |
| 4 | 7 | + tool call + GST result |
The real API shape: stop_reason
| In the kitchen | In a real model API (Anthropic) |
|---|---|
| "Get me the salt" (asks, then waits) | reply has stop_reason: "tool_use" and a tool_use block |
| The helper brings it: "here, salt" | your code runs the tool and sends a tool_result |
| "Done, serve it" | stop_reason: "end_turn": the final answer, leave the loop |
Anthropic's tool-use docs: for client tools, "Claude responds with stop_reason: "tool_use" and one or more tool_use blocks. Your code runs the operation and sends back a tool_result." Our toy model returns the same stop_reason values.
The same loop in the real reply shape (a toy model, but the message shapes follow the API: content blocks, a tool_use id, and tool_result blocks sent back in the next user message):
PRICES = {"chai": 15}
def model(messages): # toy with the REAL reply shape
if len(messages) == 1:
return {"stop_reason": "tool_use", "content": [
{"type": "tool_use", "id": "toolu_1", "name": "price",
"input": {"item": "chai"}}]}
return {"stop_reason": "end_turn",
"content": [{"type": "text", "text": "Chai is Rs 15."}]}
messages = [{"role": "user", "content": "How much is chai?"}]
while True:
reply = model(messages)
messages.append({"role": "assistant", "content": reply["content"]})
print("stop_reason:", reply["stop_reason"])
if reply["stop_reason"] != "tool_use":
print("Answer:", reply["content"][0]["text"]); break
results = [{"type": "tool_result", "tool_use_id": b["id"],
"content": str(PRICES[b["input"]["item"]])}
for b in reply["content"] if b["type"] == "tool_use"]
print("tool_result:", results)
messages.append({"role": "user", "content": results})
print([m["role"] for m in messages])
Output:
stop_reason: tool_use
tool_result: [{'type': 'tool_result', 'tool_use_id': 'toolu_1', 'content': '15'}]
stop_reason: end_turn
Answer: Chai is Rs 15.
['user', 'assistant', 'user', 'assistant']
Loop while stop_reason is tool_use; swap the toy for a real model call and the skeleton stays the same.
Thoughts: the ReAct trace
When an agent does something strange, you need to know why. So the model writes a short thought before each action.
PRICES = {"chai": 15, "samosa": 20}
def price(item): return PRICES[item]
def add_gst(amount): return round(amount * 1.05, 2)
TOOLS = {"price": price, "add_gst": add_gst}
def model(messages): # toy LLM: a THOUGHT before every action
seen = [m["content"] for m in messages if m["role"] == "tool"]
if len(seen) == 0: return ("I need today's chai price.", "price", "chai")
if len(seen) == 1: return ("Chai is 15. Now samosa.", "price", "samosa")
if len(seen) == 2:
bill = 3 * seen[0] + 2 * seen[1]
return (f"3x15 + 2x20 = {bill}. Add GST.", "add_gst", bill)
return ("I have the total.", "answer", f"Rs {seen[2]}")
messages = [{"role": "user", "content": "3 chai + 2 samosa, with GST?"}]
for step in range(1, 6):
thought, tool, x = model(messages)
print("Thought: ", thought)
if tool == "answer":
print("Answer: ", x); break
result = TOOLS[tool](x)
print("Action: ", f"{tool}({x!r})")
print("Observation: ", result)
messages += [{"role": "assistant", "content": thought},
{"role": "tool", "content": result}]
Output:
Thought: I need today's chai price.
Action: price('chai')
Observation: 15
Thought: Chai is 15. Now samosa.
Action: price('samosa')
Observation: 20
Thought: 3x15 + 2x20 = 85. Add GST.
Action: add_gst(85)
Observation: 89.25
Thought: I have the total.
Answer: Rs 89.25
The full record of thoughts, actions and observations is the trajectory (or trace). With a real model the thoughts are written by the model, not fixed strings. Anthropic's Building effective agents lists "Prioritize transparency by explicitly showing the agent's planning steps" as a core principle: when something goes wrong, read the trace.
Problem 3: a tool error kills the loop
The toy model makes a typo: samosas.
PRICES = {"chai": 15, "samosa": 20}
def price(item): return PRICES[item]
TOOLS = {"price": price}
def model(messages): # toy LLM: makes a typo first
seen = [m["content"] for m in messages if m["role"] == "tool"]
if not seen: return {"tool": "price", "input": "samosas"}
if str(seen[-1]).startswith("error"):
return {"tool": "price", "input": "samosa"} # fix and retry
return {"answer": f"A samosa costs Rs {seen[-1]}."}
messages = [{"role": "user", "content": "How much is a samosa?"}]
for step in range(1, 6):
reply = model(messages)
if "answer" in reply:
print("Answer:", reply["answer"]); break
result = TOOLS[reply["tool"]](reply["input"])
print(f"{reply['tool']}({reply['input']!r}) -> {result}")
messages += [{"role": "assistant", **reply},
{"role": "tool", "content": result}]
Output:
Traceback (most recent call last):
File "main.py", line 17, in <module>
result = TOOLS[reply["tool"]](reply["input"])
File "main.py", line 2, in price
def price(item): return PRICES[item]
~~~~~~^^^^^^
KeyError: 'samosas'
One small typo, and the whole program crashes; the customer gets nothing.
The fix: errors are observations
Catch the error and send it back as the tool result, with a hint of what's allowed. The model reads it like any other observation and retries.
PRICES = {"chai": 15, "samosa": 20}
def price(item): return PRICES[item]
TOOLS = {"price": price}
def model(messages): # toy LLM: makes a typo first
seen = [m["content"] for m in messages if m["role"] == "tool"]
if not seen: return {"tool": "price", "input": "samosas"}
if str(seen[-1]).startswith("error"):
return {"tool": "price", "input": "samosa"} # fix and retry
return {"answer": f"A samosa costs Rs {seen[-1]}."}
messages = [{"role": "user", "content": "How much is a samosa?"}]
for step in range(1, 6):
reply = model(messages)
if "answer" in reply:
print("Answer:", reply["answer"]); break
try:
result = TOOLS[reply["tool"]](reply["input"])
except KeyError as e: # the tool failed:
result = f"error: unknown item {e}. Menu: {list(PRICES)}"
print(f"{reply['tool']}({reply['input']!r}) -> {result}")
messages += [{"role": "assistant", **reply},
{"role": "tool", "content": result}]
Output:
price('samosas') -> error: unknown item 'samosas'. Menu: ['chai', 'samosa']
price('samosa') -> 20
Answer: A samosa costs Rs 20.
A one-shot answer can't recover from a mistake; a loop that observes can. Clear error messages ("unknown item 'samosas'. Menu: ['chai', 'samosa']") help the model fix itself.
Every loop needs an exit
| Exit | When |
|---|---|
| Final answer | stop_reason is end_turn |
| Max steps | never loop for ever (Anthropic: "stopping conditions (such as a maximum number of iterations)") |
| Repeat check | same action, same result again → the agent is stuck: stop and report |
| Human checkpoint | before paying, booking, sending or deleting |
| Budget | tokens, time and money grow every step |
Glossary
| Term | Meaning | Example from this episode |
|---|---|---|
| One-shot answer | The model answers in a single step, from memory | "about Rs 70" |
| Agent loop | Think → act → observe, repeated until done | the for step in range(...) loop |
| Thought (reasoning trace) | The model's short note on what to do next | "Chai is 15. Now samosa." |
| Action | A tool call the model asks for | price('samosa') |
| Observation | The result the agent sees after an action | 20 |
| ReAct | Reasoning + Acting: thoughts and actions interleaved (Yao et al., 2022) | the Thought/Action/Observation trace |
| Trajectory / trace | The full record of an agent's steps | the printed trace |
| Stateless | The model remembers nothing between calls | model([question]) gets stuck |
| Message history / state | The list of everything so far, resent every call | messages |
| Context | Everything the model reads in one call | 1, 3, 5, 7 messages |
| stop_reason | Why the model stopped writing | "tool_use" or "end_turn" |
| tool_use | The model is asking for a tool: keep looping | use("price", "chai") |
| end_turn | The model finished: leave the loop | "Total with GST: Rs 89.25" |
| tool_result | The message carrying a tool's output back to the model | {"role": "tool", "content": 15} |
| Tool error as observation | Return an error as text instead of crashing | "error: unknown item 'samosas'..." |
| Stopping condition | A rule that ends the loop | max steps, repeat check |
| GST | Goods and Services Tax (India), here 5% | add_gst(85) → 89.25 |
Common mistakes
| Mistake | What you see | Why | Fix |
|---|---|---|---|
| Sending only the question | price(chai) ×5 → Stopped: 5 steps and no answer. |
the model is stateless | append every step; send all messages |
No try around the tool |
KeyError: 'samosas' |
one tool error kills the loop | catch it; return the error as text |
while True: with no exit |
loops for ever, bill grows | impossible goals never finish | end_turn, max steps, repeat check |
| Ignoring the trace | can't tell why it misbehaved | no visibility | log thought, action, observation |
Mini project: a trace logger
Wrap the loop so every step is saved as a dictionary {"step", "thought", "action", "observation"} in a list, and print the whole trace as a table at the end. Add a repeat check: stop if the same action is asked twice in a row.
Practice
1. Add a discount(amount, pct) tool and find the price of 4 chai + 1 samosa with 10% off, then GST.
Solution
PRICES = {"chai": 15, "samosa": 20}
def price(item): return PRICES[item]
def discount(amount, pct): return amount * (100 - pct) / 100
def add_gst(amount): return round(amount * 1.05, 2)
seen = []
for thought, tool, args in [("Chai price?", price, ("chai",)),
("Samosa price?", price, ("samosa",))]:
seen.append(tool(*args)); print("Thought:", thought, "| Observation:", seen[-1])
bill = 4 * seen[0] + 1 * seen[1]
after = discount(bill, 10); print("Thought: 10% off", bill, "| Observation:", after)
print("Answer: Rs", add_gst(after))
Output:
Thought: Chai price? | Observation: 15
Thought: Samosa price? | Observation: 20
Thought: 10% off 80 | Observation: 72.0
Answer: Rs 75.6
2. Stop the loop if the model repeats the same action twice in a row (the video's "Catch a stuck agent" demo).
Solution
def model(messages): # a stuck toy model
return {"stop_reason": "tool_use", "tool": "price", "input": "chai"}
messages, last = [{"role": "user", "content": "3 chai?"}], None
for step in range(1, 6):
reply = model(messages)
action = (reply["tool"], reply["input"])
if action == last:
print(f"step {step}: repeated {action}, stopping"); break
print(f"step {step}: {action}")
last = action
messages += [{"role": "assistant", **reply}, {"role": "tool", "content": 15}]
Output:
step 1: ('price', 'chai')
step 2: repeated ('price', 'chai'), stopping
3. A trace shows price(chai) three times in a row. What's the most likely bug?
Answer
The loop isn't sending the history (tool calls and results), so the stateless model sees the same question every time.
Quick quiz 1
reply["stop_reason"] is "tool_use". What does the loop do next?
Answer
B: run the tool, append the result to the messages, and call the model again. end_turn means stop.
Quick quiz 2
Why is it stuck: price(chai) -> 15 three times?
Answer
B: the history isn't being sent.
Interview questions
- Describe the agent loop. The model reads the goal and history, decides the next action (or a final answer), your code runs the tool, the result is appended as an observation, and it repeats until a stopping condition.
- What is ReAct? A prompting pattern (Yao et al., 2022) that interleaves reasoning traces with actions, so reasoning guides actions and observations ground the reasoning.
- Why must you resend the history? Model APIs are stateless; without the earlier calls and results the model can't know what it has already done.
- What does
stop_reason: "tool_use"mean? The model is requesting one or more tools; run them and sendtool_results back.end_turnmeans it has finished. - How should an agent handle a tool error? Return the error as text in the tool result, with a hint, so the model can correct itself; don't crash the loop.
- Why do long agent runs get expensive? Each step resends the growing history, so tokens per call grow step by step.
Summary
- One-shot answers from memory fail on multi-step tasks.
- The agent loop: think → act → observe → repeat.
- The model is stateless: keep and resend a message history, which grows every step.
- ReAct adds a thought before each action; the trace makes agents debuggable.
tool_use= keep looping;end_turn= final answer.- Turn tool errors into observations; add max steps, repeat checks and human checkpoints.
Next: episode 28, tools and function calling: how a real model learns which tools exist and how to describe them.
Sources: Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models", arXiv 2210.03629 (6 Oct 2022); Anthropic docs, "Tool use with Claude" (platform.claude.com/docs/en/agents-and-tools/tool-use/overview); Anthropic, "Building effective agents", 19 Dec 2024.