M04 — Memory & Conversation
Every run so far started from a blank slate. Real support is a conversation: "Hi, I'm Sam, where's my order?" then "Actually — cancel it." For the second message to make sense, Acme must remember the first. This is the module where the three SDKs disagree the most — and where you learn the one truth underneath all of them: the model remembers nothing. You do.
Learning Objectives
- State the core truth: LLM APIs are statelessEach API call is independent. The model has no memory of previous calls — any "memory" comes from you resending the earlier turns as part of the new request. — memory is context you resend
- Build multi-turn memory the universal way: keep a history list and resend it every turn, in all three SDKs
- Use each SDK's convenience layer: a manager class (Anthropic), a chat session (Gemini), and server-stored state via
previous_response_id(OpenAI) - Weigh the tradeoff: you-own-the-history (portable, but you resend tokens) vs provider-stored state (less to send, but tied to that provider)
- Anticipate the context-window problem that memory creates — and why "compaction" exists
Why Memory?
BEFORE: Imagine a support rep with perfect knowledge but a five-second memory. You say "Hi, I'm Sam, where's order AC-1042?" They answer beautifully. You say "Great — cancel it." They blink: "Cancel what? Who are you?"
PAIN: Every follow-up collapses. The customer has to repeat their name, their order number, and the whole context in every single message. No relationship, no flow — just a series of disconnected first-contacts. Unusable for anything real.
MAPPING: That five-second-memory rep is a raw model call. It genuinely remembers nothing between requests. The fix isn't a smarter model — it's handing the rep a transcript of the conversation so far before each new message. That transcript is memory. This module is about who keeps the transcript, and how.
The Core Truth: The Model Is Stateless
A chat model API is stateless: each call is independent and the model retains nothing from previous calls. When ChatGPT or Claude.ai "remembers" your conversation, it's because the application resends the entire prior transcript with every new message. There is no hidden server-side brain holding your chat — unless you explicitly opt into a stored-state feature (which OpenAI's Responses API offers). So "giving an agent memory" means one of two things: (A) you keep the transcript and resend it, or (B) you ask the provider to keep it and reference it by id. That's the entire subject of this module.
Here's the same two-message conversation, once without memory (each call blank) and once with (the history resent). Watch the second turn succeed only on the right:
✗ No memory (each call blank)
✓ With memory (history resent)
Same model, same two messages — the only difference is that the right side resent turns 1–2 along with turn 3, so the model could see "Sam" and "AC-1042." The model didn't get smarter; it got context. Memory is not a model capability. It's a bookkeeping decision you make in your code.
Approach A — Manual History (the universal baseline)
The approach that works in all three SDKs: keep a running list of turns, append each user message and each model reply, and pass the whole list every time. You own the transcript. Each tab is a complete file with a tiny chat() helper — call it twice and the second turn remembers the first.
# memory_anthropic.py
from anthropic import Anthropic
client = Anthropic()
SYSTEM = "You are Acme Support."
history = [] # <-- the memory. You own it.
def chat(user_msg: str) -> str:
history.append({"role": "user", "content": user_msg})
resp = client.messages.create(
model="claude-sonnet-5", max_tokens=512, system=SYSTEM, messages=history,
)
reply = "".join(b.text for b in resp.content if b.type == "text")
history.append({"role": "assistant", "content": reply}) # remember the reply too
return reply
print(chat("Hi, I'm Sam. Where's order AC-1042?"))
print(chat("Great — actually, can you cancel it?")) # knows "Sam" + "AC-1042"
// memory_anthropic.mjs
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic();
const SYSTEM = "You are Acme Support.";
const history = []; // the memory — you own it
async function chat(userMsg) {
history.push({ role: "user", content: userMsg });
const resp = await client.messages.create({
model: "claude-sonnet-5", max_tokens: 512, system: SYSTEM, messages: history,
});
const reply = resp.content.filter(b => b.type === "text").map(b => b.text).join("");
history.push({ role: "assistant", content: reply });
return reply;
}
console.log(await chat("Hi, I'm Sam. Where's order AC-1042?"));
console.log(await chat("Great — actually, can you cancel it?"));
# memory_gemini.py (manual history)
from google import genai
from google.genai import types
client = genai.Client()
config = types.GenerateContentConfig(system_instruction="You are Acme Support.")
history = [] # list of types.Content — you own it
def chat(user_msg: str) -> str:
history.append(types.Content(role="user", parts=[types.Part(text=user_msg)]))
resp = client.models.generate_content(
model="gemini-2.5-flash", contents=history, config=config,
)
history.append(resp.candidates[0].content) # remember the model's turn
return resp.text
print(chat("Hi, I'm Sam. Where's order AC-1042?"))
print(chat("Great — actually, can you cancel it?"))
// memory_gemini.mjs (manual history)
import { GoogleGenAI } from "@google/genai";
const ai = new GoogleGenAI({});
const config = { systemInstruction: "You are Acme Support." };
const history = []; // list of Content — you own it
async function chat(userMsg) {
history.push({ role: "user", parts: [{ text: userMsg }] });
const resp = await ai.models.generateContent({
model: "gemini-2.5-flash", contents: history, config,
});
history.push(resp.candidates[0].content);
return resp.text;
}
console.log(await chat("Hi, I'm Sam. Where's order AC-1042?"));
console.log(await chat("Great — actually, can you cancel it?"));
# memory_openai.py (Responses API, manual input list)
from openai import OpenAI
client = OpenAI()
SYSTEM = "You are Acme Support."
history = [] # the memory — you own it
def chat(user_msg: str) -> str:
history.append({"role": "user", "content": user_msg})
resp = client.responses.create(model="gpt-5.5", instructions=SYSTEM, input=history)
history.extend(resp.output) # append the model's output items
return resp.output_text
print(chat("Hi, I'm Sam. Where's order AC-1042?"))
print(chat("Great — actually, can you cancel it?"))
// memory_openai.mjs (Responses API, manual input list)
import OpenAI from "openai";
const client = new OpenAI();
const SYSTEM = "You are Acme Support.";
let history = []; // the memory — you own it
async function chat(userMsg) {
history.push({ role: "user", content: userMsg });
const resp = await client.responses.create({ model: "gpt-5.5", instructions: SYSTEM, input: history });
history = history.concat(resp.output); // append the model's output items
return resp.output_text;
}
console.log(await chat("Hi, I'm Sam. Where's order AC-1042?"));
console.log(await chat("Great — actually, can you cancel it?"));
Every version is the same three moves: append the user turn, send the whole history, append the reply. The list is the memory. This is exactly what the M01 and M03 tool loops did between iterations — a multi-turn conversation is just that loop stretched across separate user messages. If you understand this, you understand memory. Everything else in this module is a convenience wrapper around it.
Run It
Turn 1: Hi Sam! Order AC-1042 shipped via UPS and arrives Tuesday. Turn 2: Of course, Sam — I'll cancel order AC-1042 for you right away.
history.append/push lines that store the reply and re-run: turn 2 falls apart, exactly like the left side of the animation.
Approach B — Each SDK's Shortcut
Manual history is universal but repetitive. Each SDK offers a nicer path — and here they genuinely diverge. This is the heart of the module: same goal, three different philosophies.
# Anthropic: no built-in session — you wrap the manual pattern in a class.
# (This is deliberate: Claude's base API keeps you in control of the transcript.)
from anthropic import Anthropic
class Conversation:
def __init__(self, system: str):
self.client = Anthropic()
self.system = system
self.history = []
def send(self, msg: str) -> str:
self.history.append({"role": "user", "content": msg})
r = self.client.messages.create(
model="claude-sonnet-5", max_tokens=512,
system=self.system, messages=self.history,
)
reply = "".join(b.text for b in r.content if b.type == "text")
self.history.append({"role": "assistant", "content": reply})
return reply
convo = Conversation("You are Acme Support.")
print(convo.send("Hi, I'm Sam. Where's order AC-1042?"))
print(convo.send("Great — cancel it."))
# Want server-side sessions? That's the Claude Agent SDK / Managed Agents (M07).
// Anthropic: no built-in session — wrap the manual pattern in a class.
import Anthropic from "@anthropic-ai/sdk";
class Conversation {
constructor(system) {
this.client = new Anthropic();
this.system = system;
this.history = [];
}
async send(msg) {
this.history.push({ role: "user", content: msg });
const r = await this.client.messages.create({
model: "claude-sonnet-5", max_tokens: 512,
system: this.system, messages: this.history,
});
const reply = r.content.filter(b => b.type === "text").map(b => b.text).join("");
this.history.push({ role: "assistant", content: reply });
return reply;
}
}
const convo = new Conversation("You are Acme Support.");
console.log(await convo.send("Hi, I'm Sam. Where's order AC-1042?"));
console.log(await convo.send("Great — cancel it."));
# Google: a built-in chat SESSION keeps the history for you.
from google import genai
from google.genai import types
client = genai.Client()
chat = client.chats.create(
model="gemini-2.5-flash",
config=types.GenerateContentConfig(system_instruction="You are Acme Support."),
)
print(chat.send_message("Hi, I'm Sam. Where's order AC-1042?").text)
print(chat.send_message("Great — cancel it.").text) # history handled automatically
# Inspect it anytime:
for turn in chat.get_history():
print(turn.role, turn.parts[0].text[:40])
// Google: a built-in chat SESSION keeps the history for you.
import { GoogleGenAI } from "@google/genai";
const ai = new GoogleGenAI({});
const chat = ai.chats.create({
model: "gemini-2.5-flash",
config: { systemInstruction: "You are Acme Support." },
});
console.log((await chat.sendMessage({ message: "Hi, I'm Sam. Where's order AC-1042?" })).text);
console.log((await chat.sendMessage({ message: "Great — cancel it." })).text);
// chat.getHistory() returns the stored turns.
# OpenAI: the SERVER stores the state — reference it by id, don't resend history.
from openai import OpenAI
client = OpenAI()
r1 = client.responses.create(
model="gpt-5.5",
instructions="You are Acme Support.",
input="Hi, I'm Sam. Where's order AC-1042?",
)
print(r1.output_text)
r2 = client.responses.create(
model="gpt-5.5",
input="Great — cancel it.",
previous_response_id=r1.id, # <-- server recalls the prior turn; no resend
)
print(r2.output_text)
# Responses are stored by default. Pass store=False to opt out (then thread manually).
// OpenAI: the SERVER stores the state — reference it by id.
import OpenAI from "openai";
const client = new OpenAI();
const r1 = await client.responses.create({
model: "gpt-5.5",
instructions: "You are Acme Support.",
input: "Hi, I'm Sam. Where's order AC-1042?",
});
console.log(r1.output_text);
const r2 = await client.responses.create({
model: "gpt-5.5",
input: "Great — cancel it.",
previous_response_id: r1.id, // server recalls the prior turn; no resend
});
console.log(r2.output_text);
- Anthropic — you own it, explicitly. The base Messages API has no session object; you keep the history (often in a small class). For a server-managed session you step up to the Claude Agent SDK / Managed Agents (M07).
- Gemini — a client-side session helper.
client.chats.create()returns achatobject that stores the turns for you and resends them under the hood. Convenient, but the history still lives in your process. - OpenAI — server-side state. With the Responses API, each response is stored on OpenAI's servers;
previous_response_idchains a new turn onto it without resending the transcript. Genuinely different: the state lives on their side.
Which Should I Use?
| You keep the history | Provider keeps the state | |
|---|---|---|
| Examples | Manual list, Anthropic class, Gemini chats | OpenAI previous_response_id |
| Portability | High — move to any provider, log, replay | Lower — state tied to that provider |
| What you resend | The whole transcript every turn (grows) | Just the new message + an id |
| Control & audit | Full — you can inspect/edit every turn | Partial — the transcript lives server-side |
| Best when | You want portability, logging, or custom memory | You want the least plumbing on one provider |
Every approach that resends the transcript pays for it: turn 20 sends turns 1–19 again, as input tokens, every time. Long conversations get slow and expensive, and eventually overflow the model's context windowThe maximum number of tokens a model can consider at once. When a conversation exceeds it, older turns must be dropped or summarized.. The fixes — trimming old turns, or summarizing them (compaction) — are their own topic; each provider offers help (Anthropic compaction, Gemini/OpenAI context tools). For now, just know: memory isn't free, and unbounded history is a real production cost. We revisit this in M07.
Three Ways, One Idea
| Concept | Anthropic | Google Gemini | OpenAI |
|---|---|---|---|
| Manual history field | messages=[…] | contents=[…] | input=[…] |
| Append the reply | {"role":"assistant",…} | resp.candidates[0].content | resp.output items |
| Built-in session? | No (roll your own / Agent SDK) | Yes — client.chats.create() | Yes — server-side, via id |
| Session send | convo.send(msg) (your class) | chat.send_message(msg) | responses.create(previous_response_id=…) |
| Where state lives | Your process | Your process (the chat object) | OpenAI's servers |
This reflects each company's stance on where an agent's state should live. Anthropic keeps the base API deliberately stateless — explicit, portable, auditable — and offers server sessions only at the Agent-SDK tier. Gemini adds a friendly client-side chats helper but still holds the transcript in your process. OpenAI leaned into server-stored state with the Responses API, trading portability for less plumbing. Same underlying truth (the model is stateless); three different answers to "who keeps the transcript." Knowing all three means you can pick per project instead of per habit.
Knowledge Check
Q1: How does a chat model "remember" earlier turns of a conversation?
Q2: In the manual approach, what do you append to the history each turn?
Q3: Which SDK lets the server hold the conversation state, referenced by id?
chats.create()previous_response_idchats holds state in your process; Anthropic's base API keeps you fully in control.Q4: What's the main downside of "you keep the history and resend it"?
Q5: Why might you prefer "you keep the history" over server-stored state?
Module Summary
Key Takeaways
- The model is stateless. It remembers nothing between calls — memory is context you resend or a stored state you reference.
- Manual history works everywhere: append the user turn, send the whole list, append the reply. The list is the memory.
- Each SDK has a shortcut — and they differ: a class you write (Anthropic), a
chatssession (Gemini), server-stored state viaprevious_response_id(OpenAI). - Pick by tradeoff: you-own-it is portable and auditable; provider-stored is less plumbing but tied to that provider.
- Memory isn't free. Resent transcripts grow input cost and eventually overflow the context window — the reason compaction exists.
Next: M05 — RAG / Retrieval Grounding
Acme now remembers the conversation — but it still doesn't know your return policy, your shipping FAQ, or any of the help docs it was never trained on. In M05 you'll ground Acme in a knowledge base: embed the docs, retrieve the relevant passages for each question, and feed them in — so the agent answers from your content, not its guesses. Three SDKs, three embedding APIs, one retrieval pattern.