M01: The LLM Mental Model

Understand what a Large Language Model really is, how Claude processes your text, and why the right mental model changes everything you'll build in this course.

Learning Objectives

• Explain what a Large Language Model is and how it generates text one token at a time
• Describe the difference between how Claude reads input (all at once) and writes output (one token at a time)
• Use temperature, top-p, and top-k controls and predict how they change output
• Make your first Claude API call using Python and Node.js
• Adopt the "thinker, not calculator" mental model for working with LLMs

What Is a Large Language Model?

What this actually looks like: When you type "The capital of France is", the model doesn't look up "France" in a table. Instead, it computes a probability for every possible next token. Here's a simplified version of what that prediction looks like internally:

Input: "The capital of France is"

Next token predictions:
  " Paris"    → 0.92  (92% probability)
  " the"      → 0.03  (3%)
  " located"  → 0.02  (2%)
  " a"        → 0.01  (1%)
  " Lyon"     → 0.005 (0.5%)
  ... thousands more tokens with tiny probabilities

How Claude Processes Text

What this actually looks like: Here's a real API response showing the timing asymmetry. Notice how the input (your prompt) is processed almost instantly, but the output tokens trickle out one by one:

Request: 850 tokens of input
Response: 120 tokens of output

Timeline:
  0ms    → Request sent
  180ms  → First output token arrives (all 850 input tokens processed)
  180ms  → "The"
  210ms  → " best"
  240ms  → " approach"
  ...      (each token ~30ms apart)
  3780ms → Final token generated

Input processing:  180ms  (850 tokens, all at once)
Output generation: 3600ms (120 tokens, one at a time)

How Inference Actually Works

Sampling — What “Logits” Become a Token

The transformer doesn’t pick a token; it outputs logits — a raw score per token in the vocabulary (~200K entries for Claude). To turn logits into a chosen token, three steps run:

1. Softmax with temperature. Logits — divide by temperature T — then softmax to a probability distribution. T = 0 collapses to one-hot (always pick the argmax); T = 1 keeps the model’s native distribution; T > 1 flattens it (more surprise).
2. Top-k / Top-p filter. Optionally truncate the distribution to either the top-k highest-probability tokens, or the smallest set whose cumulative probability exceeds p (nucleus sampling). This blocks the long tail of nonsense tokens.
3. Sample. Draw one token from the (renormalized) filtered distribution. This single token becomes the “next token,” gets appended, and the loop continues.

Important consequence: the model’s output is non-deterministic even at temperature=1 because of the sampling step. Set temperature=0 if you need reproducibility — that picks the argmax with no sampling. The next H2 on Temperature/Top-p/Top-k goes deeper on when to use each.

Latency Anatomy — Where Your Seconds Go

Real numbers, on Claude Sonnet 4.6 (typical 2026 production load, single-region):

Streaming vs Batch — Same Inference, Different Delivery

One request, two ways to receive the answer:

• Non-streaming — you wait for the full decode, then receive the entire response in one chunk. Simple; the agent code in M05–M07 uses this shape.
• Streaming — the server sends each decoded token (or small group) as Server-Sent Events. Same total time, but the user sees the first tokens immediately. Essential for chat UIs and any agent loop where you want to surface progress before the response is complete.

from anthropic import Anthropic
import time

client = Anthropic()

# Streaming: each token (or small chunk) arrives as it’s decoded.
t0 = time.perf_counter()
first_token_t = None
total_tokens = 0

with client.messages.stream(
    model="claude-sonnet-4-6",
    max_tokens=512,
    messages=[{"role": "user", "content": "Explain inference in one paragraph."}],
) as stream:
    for chunk in stream.text_stream:
        if first_token_t is None:
            first_token_t = time.perf_counter() - t0
            print(f"[TTFT: {first_token_t*1000:.0f} ms]")
        print(chunk, end="", flush=True)
        total_tokens += 1   # text chunks, not perfect token counts — close enough for ops

elapsed = time.perf_counter() - t0
print(f"\n[total: {elapsed:.2f}s, decode rate ~ {total_tokens/max(elapsed-first_token_t, 0.001):.0f} chunks/s]")

import Anthropic from "@anthropic-ai/sdk";

const client = new Anthropic();
const t0 = performance.now();
let firstTokenMs: number | null = null;
let chunks = 0;

const stream = client.messages.stream({
  model: "claude-sonnet-4-6",
  max_tokens: 512,
  messages: [{ role: "user", content: "Explain inference in one paragraph." }],
});

for await (const event of stream) {
  if (event.type === "content_block_delta" && event.delta.type === "text_delta") {
    if (firstTokenMs === null) {
      firstTokenMs = performance.now() - t0;
      console.log(`[TTFT: ${firstTokenMs.toFixed(0)} ms]`);
    }
    process.stdout.write(event.delta.text);
    chunks++;
  }
}

const elapsed = performance.now() - t0;
console.log(`\n[total: ${(elapsed / 1000).toFixed(2)}s, decode rate ~ ${(chunks / Math.max((elapsed - (firstTokenMs ?? 0)) / 1000, 0.001)).toFixed(0)} chunks/s]`);

Temperature, Top-p & Top-k

What this actually looks like: Here's the same set of next-token probabilities at three different temperature settings. Watch how the distribution shifts:

Prompt: "The best way to learn programming is"

Temperature 0.0 (greedy — always pick the top word):
  " to"       → 99.8%  ← always chosen
  " by"       →  0.1%
  " through"  →  0.1%

Temperature 0.5 (moderate — top words dominate but others have a chance):
  " to"       → 58%
  " by"       → 22%
  " through"  → 12%
  " with"     →  5%
  " from"     →  3%

Temperature 1.0 (creative — spread across many options):
  " to"       → 30%
  " by"       → 22%
  " through"  → 15%
  " with"     → 10%
  " from"     →  6%
  " when"     →  4%
  ... more words now have a real chance

The "Calculator vs. Thinker" Mental Model

What this actually looks like: Here's a real-world example of why the "thinker" model matters. Same prompt, two runs, both at temperature 0.0:

Prompt: "What is the population of Tokyo?"

Run 1: "Tokyo has a population of approximately 13.96 million
        people in the city proper as of 2023."

Run 2: "The population of Tokyo is about 14 million in the
        city proper, or roughly 37 million in the greater
        metropolitan area."

Both are reasonable. Neither is "wrong." But a calculator
would give you the exact same answer every time.
An agent that routes decisions based on this output
needs to handle BOTH variations gracefully.

This mental model is the single most important idea in this course. In plain English: a calculator always gives you 2 + 2 = 4. A thinker gives you their best reasoning, which is usually excellent but occasionally off. When you build an agent, you're building around a thinker — so you design for "usually right" rather than "always right."

How does this work internally? When Claude generates a response, it's making thousands of probabilistic choices (one per token). Each choice has some chance of going a different direction. Even at temperature 0.0, server-side implementation details like floating-point rounding and batch scheduling can cause tiny variations. The result is that outputs are highly consistent but not identical — and when the model is uncertain (borderline cases, ambiguous questions, math), those small variations can compound into meaningfully different answers.

How is this different from traditional software? In conventional programming, if a function returns the wrong result, it's a bug — you fix the code and it works. With LLMs, variation isn't a bug; it's a fundamental property of how the system works. This means your job as an agent builder shifts from "make it correct" to "make it reliably useful despite occasional imperfection." That's a completely different engineering discipline, and it's what the rest of this course teaches.

Here's how this mental model affects every decision you'll make:

• Prompts (M03): You're giving instructions to a thinker, not writing code for a machine
• Tool Use (M05): You give the thinker tools to compensate for what prediction can't do (real-time data, calculations, database lookups)
• Guardrails (M16–M17): You build checks because thinkers can make mistakes
• Evaluation (M18): You measure quality probabilistically, not as pass/fail

Code Walkthrough: Your First Claude API Call

Let's make your first call to the Claude API. You'll need an API keyA secret string that authenticates your application with Anthropic's servers. Never share it, never commit it to code. Always store it as an environment variable. from console.anthropic.com.

Setup: API Key as Environment Variable

# Set your API key as an environment variable
export ANTHROPIC_API_KEY="your-key-here"

Making the Call

# pip install anthropic>=0.30.0
import anthropic

# The client reads ANTHROPIC_API_KEY from your environment
client = anthropic.Anthropic()

try:
    message = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=1024,
        system="You are a helpful assistant who explains things clearly.",
        messages=[
            {"role": "user", "content": "What is a large language model? Explain in 2 sentences."}
        ]
    )
    # The response content is a list of content blocks
    print(message.content[0].text)
    print(f"\nTokens used: {message.usage.input_tokens} in, {message.usage.output_tokens} out")
except anthropic.AuthenticationError:
    print("Invalid API key. Check your ANTHROPIC_API_KEY environment variable.")
except anthropic.APIError as e:
    print(f"API error: {e.message}")

// npm install @anthropic-ai/sdk@^0.30.0
import Anthropic from '@anthropic-ai/sdk';

// The client reads ANTHROPIC_API_KEY from your environment
const client = new Anthropic();

try {
  const message = await client.messages.create({
    model: 'claude-sonnet-4-6',
    max_tokens: 1024,
    system: 'You are a helpful assistant who explains things clearly.',
    messages: [
      { role: 'user', content: 'What is a large language model? Explain in 2 sentences.' }
    ]
  });

  // The response content is a list of content blocks
  console.log(message.content[0].text);
  console.log(`\nTokens used: ${message.usage.input_tokens} in, ${message.usage.output_tokens} out`);
} catch (error) {
  if (error instanceof Anthropic.APIError) {
    console.error(`API error: ${error.status} - ${error.message}`);
  } else {
    throw error;
  }
}

curl https://api.anthropic.com/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-sonnet-4-6",
    "max_tokens": 1024,
    "system": "You are a helpful assistant who explains things clearly.",
    "messages": [
      {"role": "user", "content": "What is a large language model? Explain in 2 sentences."}
    ]
  }'

Experimenting with Temperature

import anthropic

client = anthropic.Anthropic()

prompt = "Write a one-sentence description of the moon."

for temp in [0.0, 0.5, 1.0]:
    try:
        message = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=100,
            temperature=temp,
            messages=[{"role": "user", "content": prompt}]
        )
        print(f"Temperature {temp}: {message.content[0].text}")
    except anthropic.APIError as e:
        print(f"Error at temperature {temp}: {e.message}")

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();
const prompt = 'Write a one-sentence description of the moon.';

for (const temp of [0.0, 0.5, 1.0]) {
  try {
    const message = await client.messages.create({
      model: 'claude-sonnet-4-6',
      max_tokens: 100,
      temperature: temp,
      messages: [{ role: 'user', content: prompt }]
    });
    console.log(`Temperature ${temp}: ${message.content[0].text}`);
  } catch (error) {
    if (error instanceof Anthropic.APIError) {
      console.error(`Error at temp ${temp}: ${error.message}`);
    } else {
      throw error;
    }
  }
}

Hands-On Exercise: Hello Claude

Environment Setup

Copy and paste this entire block into your terminal to create a project folder and install the SDK:

mkdir hello-claude && cd hello-claude
python -m venv venv
# macOS/Linux:
source venv/bin/activate
# Windows:
# venv\Scripts\activate
pip install "anthropic>=0.30.0"
export ANTHROPIC_API_KEY="your-key-here"   # Windows: set ANTHROPIC_API_KEY=your-key-here

mkdir hello-claude && cd hello-claude
npm init -y
npm install @anthropic-ai/sdk@^0.30.0
export ANTHROPIC_API_KEY="your-key-here"   # Windows: set ANTHROPIC_API_KEY=your-key-here

Step 1: Make Your First API Call

This step verifies that your API key works and that you can communicate with Claude. It's the "hello world" of agent development — if this works, everything else in the course will build on it.

Create a new file called hello.py (or hello.mjs for Node.js):

import anthropic

client = anthropic.Anthropic()

try:
    message = client.messages.create(
        model="claude-sonnet-4-6",
        max_tokens=1024,
        system="You are a helpful assistant who explains things clearly.",
        messages=[
            {"role": "user", "content": "What is a large language model? Explain in 2 sentences."}
        ]
    )
    print(message.content[0].text)
    print(f"\nTokens used: {message.usage.input_tokens} in, {message.usage.output_tokens} out")
except anthropic.AuthenticationError:
    print("Invalid API key. Check your ANTHROPIC_API_KEY environment variable.")
except anthropic.APIError as e:
    print(f"API error: {e.message}")

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();

try {
  const message = await client.messages.create({
    model: 'claude-sonnet-4-6',
    max_tokens: 1024,
    system: 'You are a helpful assistant who explains things clearly.',
    messages: [
      { role: 'user', content: 'What is a large language model? Explain in 2 sentences.' }
    ]
  });
  console.log(message.content[0].text);
  console.log(`\nTokens used: ${message.usage.input_tokens} in, ${message.usage.output_tokens} out`);
} catch (error) {
  if (error instanceof Anthropic.APIError) {
    console.error(`API error: ${error.status} - ${error.message}`);
  } else {
    throw error;
  }
}

Run it: python hello.py (or node hello.mjs)

Step 2: Experiment with System Prompts

System prompts shape Claude's personality and behavior. This step shows you how much control a single string gives you over the output. You'll use the same user message but swap the system prompt to see wildly different responses.

Create a new file called system_prompts.py (or system_prompts.mjs):

import anthropic

client = anthropic.Anthropic()

prompts = [
    "You are a pirate. Respond in pirate speak.",
    "You are a formal academic. Use precise, scholarly language.",
    "Respond only in haiku format (5-7-5 syllables).",
]

for system_prompt in prompts:
    try:
        message = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=200,
            system=system_prompt,
            messages=[{"role": "user", "content": "What is the moon?"}]
        )
        print(f"System: {system_prompt}")
        print(f"Response: {message.content[0].text}\n")
    except anthropic.APIError as e:
        print(f"Error: {e.message}")

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();

const prompts = [
  'You are a pirate. Respond in pirate speak.',
  'You are a formal academic. Use precise, scholarly language.',
  'Respond only in haiku format (5-7-5 syllables).',
];

for (const systemPrompt of prompts) {
  try {
    const message = await client.messages.create({
      model: 'claude-sonnet-4-6',
      max_tokens: 200,
      system: systemPrompt,
      messages: [{ role: 'user', content: 'What is the moon?' }]
    });
    console.log(`System: ${systemPrompt}`);
    console.log(`Response: ${message.content[0].text}\n`);
  } catch (error) {
    console.error(`Error: ${error.message}`);
  }
}

Run it: python system_prompts.py (or node system_prompts.mjs)

Step 3: Temperature Experiment

This step makes the "thinker, not calculator" concept visceral. You'll run the exact same prompt multiple times at different temperatures and compare how consistent the outputs are. This is the experiment that makes non-determinism click.

Create a new file called temperature.py (or temperature.mjs):

import anthropic

client = anthropic.Anthropic()
prompt = "Write a one-sentence description of the moon."

for temp in [0.0, 1.0]:
    print(f"\n--- Temperature {temp} ---")
    for i in range(3):
        try:
            message = client.messages.create(
                model="claude-sonnet-4-6",
                max_tokens=100,
                temperature=temp,
                messages=[{"role": "user", "content": prompt}]
            )
            print(f"  Run {i+1}: {message.content[0].text}")
        except anthropic.APIError as e:
            print(f"  Error: {e.message}")

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();
const prompt = 'Write a one-sentence description of the moon.';

for (const temp of [0.0, 1.0]) {
  console.log(`\n--- Temperature ${temp} ---`);
  for (let i = 0; i < 3; i++) {
    try {
      const message = await client.messages.create({
        model: 'claude-sonnet-4-6',
        max_tokens: 100,
        temperature: temp,
        messages: [{ role: 'user', content: prompt }]
      });
      console.log(`  Run ${i + 1}: ${message.content[0].text}`);
    } catch (error) {
      console.error(`  Error: ${error.message}`);
    }
  }
}

Run it: python temperature.py (or node temperature.mjs)

Step 4: Observe Token Usage

Token counts drive API costs and context limits — two concepts you'll use throughout the entire course. This step trains you to notice how different prompts and settings affect token consumption, so it becomes instinctive. This uses the message.usage object from Step 1.

Create a new file called token_usage.py (or token_usage.mjs):

import anthropic

client = anthropic.Anthropic()

tests = [
    ("Short prompt", "Hi!", 50),
    ("Medium prompt", "Explain what a large language model is in detail.", 200),
    ("Long prompt with constraint", "Write a 3-paragraph essay about the history of computing.", 1024),
]

for label, prompt, max_tok in tests:
    try:
        message = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=max_tok,
            messages=[{"role": "user", "content": prompt}]
        )
        u = message.usage
        print(f"{label}:")
        print(f"  Input tokens:  {u.input_tokens}")
        print(f"  Output tokens: {u.output_tokens}")
        print(f"  Total tokens:  {u.input_tokens + u.output_tokens}\n")
    except anthropic.APIError as e:
        print(f"Error: {e.message}")

import Anthropic from '@anthropic-ai/sdk';

const client = new Anthropic();

const tests = [
  ['Short prompt', 'Hi!', 50],
  ['Medium prompt', 'Explain what a large language model is in detail.', 200],
  ['Long prompt with constraint', 'Write a 3-paragraph essay about the history of computing.', 1024],
];

for (const [label, prompt, maxTok] of tests) {
  try {
    const message = await client.messages.create({
      model: 'claude-sonnet-4-6',
      max_tokens: maxTok,
      messages: [{ role: 'user', content: prompt }]
    });
    const u = message.usage;
    console.log(`${label}:`);
    console.log(`  Input tokens:  ${u.input_tokens}`);
    console.log(`  Output tokens: ${u.output_tokens}`);
    console.log(`  Total tokens:  ${u.input_tokens + u.output_tokens}\n`);
  } catch (error) {
    console.error(`Error: ${error.message}`);
  }
}

Run it: python token_usage.py (or node token_usage.mjs)

Step 5 (Stretch Goal): Build a CLI Chat

import anthropic

client = anthropic.Anthropic()
conversation = []

print("Chat with Claude! (type 'quit' to exit)\n")

while True:
    user_input = input("You: ").strip()
    if user_input.lower() in ("quit", "exit"):
        break
    if not user_input:
        continue

    conversation.append({"role": "user", "content": user_input})

    try:
        response = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=512,
            system="You are a friendly, helpful assistant.",
            messages=conversation
        )
        assistant_msg = response.content[0].text
        conversation.append({"role": "assistant", "content": assistant_msg})
        print(f"\nClaude: {assistant_msg}\n")
    except anthropic.APIError as e:
        print(f"\nError: {e.message}\n")
        # Remove the failed user message so conversation stays consistent
        conversation.pop()

import Anthropic from '@anthropic-ai/sdk';
import * as readline from 'readline';

const client = new Anthropic();
const conversation = [];

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout
});

console.log("Chat with Claude! (type 'quit' to exit)\n");

function ask() {
  rl.question('You: ', async (userInput) => {
    userInput = userInput.trim();
    if (['quit', 'exit'].includes(userInput.toLowerCase())) {
      rl.close();
      return;
    }
    if (!userInput) { ask(); return; }

    conversation.push({ role: 'user', content: userInput });

    try {
      const response = await client.messages.create({
        model: 'claude-sonnet-4-6',
        max_tokens: 512,
        system: 'You are a friendly, helpful assistant.',
        messages: conversation
      });
      const assistantMsg = response.content[0].text;
      conversation.push({ role: 'assistant', content: assistantMsg });
      console.log(`\nClaude: ${assistantMsg}\n`);
    } catch (error) {
      console.error(`\nError: ${error.message}\n`);
      conversation.pop();
    }
    ask();
  });
}
ask();

Verify Everything Works

Run all four scripts in sequence to confirm your setup is complete:

# Python
python hello.py && python system_prompts.py && python temperature.py && python token_usage.py

# Node.js
node hello.mjs && node system_prompts.mjs && node temperature.mjs && node token_usage.mjs

Knowledge Check

Test your understanding of the concepts from this module. Select the best answer for each question.

message = client.messages.create(
    model="claude-sonnet-4-6",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}]
)
print(message.______[0].text)

Module Summary