M01: The LLM Mental Model

Understand what a Large Language Model really is, how the model processes your text, and why the right mental model changes everything you'll build in this course.

Learning Objectives

• Explain what a Large Language Model is and how it generates text one token at a time
• Describe the difference between how the model reads input (all at once) and writes output (one token at a time)
• Use temperature, top-p, and top-k controls and predict how they change output
• Make your first the model API call using Python and Node.js
• Identify the four major LLM specializations — generative, embedding, reranker, multimodal — and when each is used in an agent pipeline
• Adopt the "thinker, not calculator" mental model for working with LLMs

What Is a Large Language Model?

What this actually looks like: When you type "The capital of France is", the model doesn't look up "France" in a table. Instead, it computes a probability for every possible next token. Here's a simplified version of what that prediction looks like internally:

Input: "The capital of France is"

Next token predictions:
  " Paris"    → 0.92  (92% probability)
  " the"      → 0.03  (3%)
  " located"  → 0.02  (2%)
  " a"        → 0.01  (1%)
  " Lyon"     → 0.005 (0.5%)
  ... thousands more tokens with tiny probabilities

The Model Zoo: Not All AI Models Do the Same Job

Mistral 7B — the model powering the rest of this course — is a generative, text-only model. The other types are purpose-built models you run alongside it: the RAG track (M09–M11) adds embedding and reranker models, and images go to a vision model like llava. For now, let’s zoom in on exactly how the model processes text.

How the model Processes Text

What this actually looks like: Here's a real API response showing the timing asymmetry. Notice how the input (your prompt) is processed almost instantly, but the output tokens trickle out one by one:

Request: 850 tokens of input
Response: 120 tokens of output

Timeline:
  0ms    → Request sent
  180ms  → First output token arrives (all 850 input tokens processed)
  180ms  → "The"
  210ms  → " best"
  240ms  → " approach"
  ...      (each token ~30ms apart)
  3780ms → Final token generated

Input processing:  180ms  (850 tokens, all at once)
Output generation: 3600ms (120 tokens, one at a time)

How Inference Actually Works

Sampling — How a List of Scores Becomes One Word

Start with the question every beginner actually has: why does the model give a different answer when I ask the exact same thing twice? The answer lives in the last step of decode, and it’s simpler than it sounds. Let’s shrink the problem down to something you can hold in your head.

Imagine you ask the model for the next word in a sentence, and instead of 32,000 choices it only has five. The model does not pick a word. It hands you a scorecard — one raw, uncalibrated number per word:

Those raw scores are called logitsThe raw, unnormalized scores a neural network outputs — one per token in the vocabulary. They can be any real number (negative or positive) and do not add up to 1. Softmax converts them into probabilities.. The real model produces one for every word it knows — tens of thousands of them (~32K for a model like Mistral) — but five is enough to see the whole mechanism. Logits aren’t probabilities yet: they can be negative, and they don’t add up to 100%. Three steps turn this scorecard into one chosen word.

Step 1 — Softmax: turn scores into percentages. SoftmaxA function that squashes a list of real-valued scores into positive numbers that sum to 1, while preserving their order. Bigger score in → bigger probability out. It exponentiates each score then divides by the total. is just a formula that squashes any list of scores into percentages that add up to 100%, while keeping their order (biggest stays biggest). Run our scorecard through it and we get a real probability distribution — a likelihood for each word:

Step 2 — Temperature: the “boldness dial.” Before softmax runs, we divide every score by a number T called temperatureA scalar that divides the logits before softmax. Low temperature sharpens the distribution toward the top token (more predictable); high temperature flattens it (more variety). Named by analogy to thermodynamics, where higher temperature means more random motion.. Think of it as a dial for how bold the model is allowed to be:

• T = 0 → always take the single highest score. “cat,” every single time. Predictable and repeatable, but it can feel robotic.
• T = 1 → use the model’s honest percentages above (cat 63%, dog 23%…).
• T = 2 → flatten the gaps so the underdogs get a real shot. The same scorecard now reads cat 42%, dog 25%, bird 15%, rock 9%, the 7%. More variety, more surprise — and more risk of “rock.”

Step 3 — Top-k / Top-p: the bouncer at the door. Even after softmax, we usually don’t want the model to ever blurt out a clearly-wrong word like “rock.” So we trim the tail before drawing:

• Top-kKeep only the k highest-probability tokens, discard the rest, then renormalize. A hard cap on how many candidates survive. — keep only the k highest words. With k = 2, only cat and dog survive; everything else is turned away.
• Top-pAlso called nucleus sampling. Sort tokens by probability and keep the smallest set whose cumulative probability reaches p, then renormalize. The number of survivors flexes with how confident the model is. (nucleus) — keep just enough words to cover p% of the probability. With p = 0.90, we add words until we’ve covered 90%: cat (63%) + dog (86%) + bird (95% ≥ 90%) — so cat, dog, bird stay and rock, the are cut.

Then: roll the dice. From whichever words survived the bouncer, the model draws one at random — weighted by those percentages, so “cat” comes up far more often than “bird,” but “bird” can still win sometimes. That chosen word becomes the next token, gets appended to the sequence, and the whole decode loop repeats for the word after it.

Latency Anatomy — Where Your Seconds Go

Real numbers, on the model Sonnet 4.6 (typical 2026 production load, single-region):

Streaming vs Batch — Same Inference, Different Delivery

One request, two ways to receive the answer:

• Non-streaming — you wait for the full decode, then receive the entire response in one chunk. Simple; the agent code in M05–M07 uses this shape.
• Streaming — the server sends each decoded token (or small group) as Server-Sent Events. Same total time, but the user sees the first tokens immediately. Essential for chat UIs and any agent loop where you want to surface progress before the response is complete.

from openai import OpenAI
import time

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

# Streaming: each token (or small chunk) arrives as it’s decoded.
t0 = time.perf_counter()
first_token_t = None
total_tokens = 0

stream = client.chat.completions.create(
    model="mistral",
    messages=[{"role": "user", "content": "Explain inference in one paragraph."}],
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if not delta:                      # role-only / empty deltas arrive first
        continue
    if first_token_t is None:
        first_token_t = time.perf_counter() - t0
        print(f"[TTFT: {first_token_t*1000:.0f} ms]")
    print(delta, end="", flush=True)
    total_tokens += 1   # text chunks, not perfect token counts — close enough for ops

elapsed = time.perf_counter() - t0
print(f"\n[total: {elapsed:.2f}s, decode rate ~ {total_tokens/max(elapsed-first_token_t, 0.001):.0f} chunks/s]")

import OpenAI from "openai";

const client = new OpenAI({ baseURL: 'http://localhost:11434/v1', apiKey: 'ollama' });
const t0 = performance.now();
let firstTokenMs: number | null = null;
let chunks = 0;

const stream = await client.chat.completions.create({
  model: "mistral",
  messages: [{ role: "user", content: "Explain inference in one paragraph." }],
  stream: true,
});

for await (const chunk of stream) {
  const delta = chunk.choices[0]?.delta?.content;
  if (!delta) continue;              // role-only / empty deltas arrive first
  if (firstTokenMs === null) {
    firstTokenMs = performance.now() - t0;
    console.log(`[TTFT: ${firstTokenMs.toFixed(0)} ms]`);
  }
  process.stdout.write(delta);
  chunks++;
}

const elapsed = performance.now() - t0;
console.log(`\n[total: ${(elapsed / 1000).toFixed(2)}s, decode rate ~ ${(chunks / Math.max((elapsed - (firstTokenMs ?? 0)) / 1000, 0.001)).toFixed(0)} chunks/s]`);

Temperature, Top-p & Top-k

What this actually looks like: Here's the same set of next-token probabilities at three different temperature settings. Watch how the distribution shifts:

Prompt: "The best way to learn programming is"

Temperature 0.0 (greedy — always pick the top word):
  " to"       → 99.8%  ← always chosen
  " by"       →  0.1%
  " through"  →  0.1%

Temperature 0.5 (moderate — top words dominate but others have a chance):
  " to"       → 58%
  " by"       → 22%
  " through"  → 12%
  " with"     →  5%
  " from"     →  3%

Temperature 1.0 (creative — spread across many options):
  " to"       → 30%
  " by"       → 22%
  " through"  → 15%
  " with"     → 10%
  " from"     →  6%
  " when"     →  4%
  ... more words now have a real chance

The "Calculator vs. Thinker" Mental Model

What this actually looks like: Here's a real-world example of why the "thinker" model matters. Same prompt, two runs, both at temperature 0.0:

Prompt: "What is the population of Tokyo?"

Run 1: "Tokyo has a population of approximately 13.96 million
        people in the city proper as of 2023."

Run 2: "The population of Tokyo is about 14 million in the
        city proper, or roughly 37 million in the greater
        metropolitan area."

Both are reasonable. Neither is "wrong." But a calculator
would give you the exact same answer every time.
An agent that routes decisions based on this output
needs to handle BOTH variations gracefully.

This mental model is the single most important idea in this course. In plain English: a calculator always gives you 2 + 2 = 4. A thinker gives you their best reasoning, which is usually excellent but occasionally off. When you build an agent, you're building around a thinker — so you design for "usually right" rather than "always right."

How does this work internally? When the model generates a response, it's making thousands of probabilistic choices (one per token). Each choice has some chance of going a different direction. Even at temperature 0.0, server-side implementation details like floating-point rounding and batch scheduling can cause tiny variations. The result is that outputs are highly consistent but not identical — and when the model is uncertain (borderline cases, ambiguous questions, math), those small variations can compound into meaningfully different answers.

How is this different from traditional software? In conventional programming, if a function returns the wrong result, it's a bug — you fix the code and it works. With LLMs, variation isn't a bug; it's a fundamental property of how the system works. This means your job as an agent builder shifts from "make it correct" to "make it reliably useful despite occasional imperfection." That's a completely different engineering discipline, and it's what the rest of this course teaches.

Here's how this mental model affects every decision you'll make:

• Prompts (M03): You're giving instructions to a thinker, not writing code for a machine
• Tool Use (M05): You give the thinker tools to compensate for what prediction can't do (real-time data, calculations, database lookups)
• Guardrails (M16–M17): You build checks because thinkers can make mistakes
• Evaluation (M18): You measure quality probabilistically, not as pass/fail

Code Walkthrough: Your First the model API Call

Let's make your first call to the model API. This track uses Ollama running locally — no API key needed. Ollama exposes an OpenAI-compatible endpointA URL where your code sends requests. Ollama's Chat Completions endpoint is http://localhost:11434/v1 — the same interface as the OpenAI API, just running on your machine. at http://localhost:11434/v1.

Setup: Start Ollama & Pull a Model

# Start Ollama (runs as a background daemon)
ollama serve

# Pull Mistral (first time only — ~4GB download)
ollama pull mistral

# Verify it's available
ollama list   # should show "mistral" in the list

Making the Call

# pip install openai>=0.30.0
from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

try:
    message = client.chat.completions.create(
        model="mistral",
        messages=[
            {"role": "system", "content": "You are a helpful assistant who explains things clearly."},
            {"role": "user", "content": "What is a large language model? Explain in 2 sentences."}
        ]
    )
    print(message.choices[0].message.content)
    print(f"\nTokens used: {message.usage.prompt_tokens} in, {message.usage.completion_tokens} out")
except Exception as e:
    print(f"API error: {e}")

// npm install openai
import OpenAI from 'openai';

const client = new OpenAI({ baseURL: 'http://localhost:11434/v1', apiKey: 'ollama' });

try {
  const message = await client.chat.completions.create({
    model: 'mistral',
    messages: [
      { role: 'system', content: 'You are a helpful assistant who explains things clearly.' },
      { role: 'user', content: 'What is a large language model? Explain in 2 sentences.' }
    ]
  });

  console.log(message.choices[0].message.content);
  console.log(`\nTokens used: ${message.usage.prompt_tokens} in, ${message.usage.completion_tokens} out`);
} catch (error) {
  if (error?.status) {
    console.error(`API error: ${error.status} - ${error.message}`);
  } else {
    throw error;
  }
}

curl http://localhost:11434/v1/chat/completions \
  -H "content-type: application/json" \
  -d '{
    "model": "mistral",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant who explains things clearly."},
      {"role": "user", "content": "What is a large language model? Explain in 2 sentences."}
    ]
  }'

Experimenting with Temperature

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

prompt = "Write a one-sentence description of the moon."

for temp in [0.0, 0.5, 1.0]:
    try:
        message = client.chat.completions.create(
            model="mistral",
            temperature=temp,
            messages=[{"role": "user", "content": prompt}]
        )
        print(f"Temperature {temp}: {message.choices[0].message.content}")
    except Exception as e:
        print(f"Error at temperature {temp}: {e}")

import OpenAI from 'openai';

const client = new OpenAI({ baseURL: 'http://localhost:11434/v1', apiKey: 'ollama' });
const prompt = 'Write a one-sentence description of the moon.';

for (const temp of [0.0, 0.5, 1.0]) {
  try {
    const message = await client.chat.completions.create({
      model: 'mistral',
      temperature: temp,
      messages: [{ role: 'user', content: prompt }]
    });
    console.log(`Temperature ${temp}: ${message.choices[0].message.content}`);
  } catch (error) {
    if (error?.status) {
      console.error(`Error at temp ${temp}: ${error.message}`);
    } else {
      throw error;
    }
  }
}

Hands-On Exercise: Hello the model

Environment Setup

Copy and paste this entire block into your terminal to create a project folder and install the SDK:

mkdir hello-model && cd hello-model
python -m venv venv
# macOS/Linux:
source venv/bin/activate
# Windows:
# venv\Scripts\activate
pip install openai
# No API key needed — Ollama runs locally!
# Make sure Ollama is running: ollama serve
# Make sure mistral is downloaded: ollama pull mistral

mkdir hello-model && cd hello-model
npm init -y
npm install openai
# No API key needed — Ollama runs locally!
# Make sure Ollama is running: ollama serve
# Make sure mistral is downloaded: ollama pull mistral

Step 1: Make Your First API Call

This step verifies that Ollama is running and you can communicate with the model. It's the "hello world" of agent development — if this works, everything else in the course will build on it.

Create a new file called hello.py (or hello.mjs for Node.js):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

try:
    message = client.chat.completions.create(
        model="mistral",
        messages=[
            {"role": "system", "content": "You are a helpful assistant who explains things clearly."},
            {"role": "user", "content": "What is a large language model? Explain in 2 sentences."}
        ]
    )
    print(message.choices[0].message.content)
    print(f"\nTokens used: {message.usage.prompt_tokens} in, {message.usage.completion_tokens} out")
except Exception as e:
    print(f"API error: {e}")
    print("Is Ollama running? Try: ollama serve")

import OpenAI from 'openai';

const client = new OpenAI({ baseURL: 'http://localhost:11434/v1', apiKey: 'ollama' });

try {
  const message = await client.chat.completions.create({
    model: 'mistral',
    messages: [
      { role: 'system', content: 'You are a helpful assistant who explains things clearly.' },
      { role: 'user', content: 'What is a large language model? Explain in 2 sentences.' }
    ]
  });
  console.log(message.choices[0].message.content);
  console.log(`\nTokens used: ${message.usage.prompt_tokens} in, ${message.usage.completion_tokens} out`);
} catch (error) {
  console.error(`API error: ${error.message}`);
  console.error('Is Ollama running? Try: ollama serve');
}

Run it: python hello.py (or node hello.mjs)

Step 2: Experiment with System Prompts

System prompts shape the model's personality and behavior. This step shows you how much control a single string gives you over the output. You'll use the same user message but swap the system prompt to see wildly different responses.

Create a new file called system_prompts.py (or system_prompts.mjs):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

prompts = [
    "You are a pirate. Respond in pirate speak.",
    "You are a formal academic. Use precise, scholarly language.",
    "Respond only in haiku format (5-7-5 syllables).",
]

for system_prompt in prompts:
    try:
        message = client.chat.completions.create(
            model="mistral",
            messages=[{"role": "user", "content": "What is the moon?"}]
        )
        print(f"System: {system_prompt}")
        print(f"Response: {message.choices[0].message.content}\n")
    except Exception as e:
        print(f"Error: {e}")

import OpenAI from 'openai';

const client = new OpenAI({ baseURL: 'http://localhost:11434/v1', apiKey: 'ollama' });

const prompts = [
  'You are a pirate. Respond in pirate speak.',
  'You are a formal academic. Use precise, scholarly language.',
  'Respond only in haiku format (5-7-5 syllables).',
];

for (const systemPrompt of prompts) {
  try {
    const message = await client.chat.completions.create({
      model: 'mistral',
      messages: [{ role: 'user', content: 'What is the moon?' }]
    });
    console.log(`System: ${systemPrompt}`);
    console.log(`Response: ${message.choices[0].message.content}\n`);
  } catch (error) {
    console.error(`Error: ${error.message}`);
  }
}

Run it: python system_prompts.py (or node system_prompts.mjs)

Step 3: Temperature Experiment

This step makes the "thinker, not calculator" concept visceral. You'll run the exact same prompt multiple times at different temperatures and compare how consistent the outputs are. This is the experiment that makes non-determinism click.

Create a new file called temperature.py (or temperature.mjs):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
prompt = "Write a one-sentence description of the moon."

for temp in [0.0, 1.0]:
    print(f"\n--- Temperature {temp} ---")
    for i in range(3):
        try:
            message = client.chat.completions.create(
                model="mistral",
                temperature=temp,
                messages=[{"role": "user", "content": prompt}]
            )
            print(f"  Run {i+1}: {message.choices[0].message.content}")
        except Exception as e:
            print(f"  Error: {e}")

import OpenAI from 'openai';

const client = new OpenAI({ baseURL: 'http://localhost:11434/v1', apiKey: 'ollama' });
const prompt = 'Write a one-sentence description of the moon.';

for (const temp of [0.0, 1.0]) {
  console.log(`\n--- Temperature ${temp} ---`);
  for (let i = 0; i < 3; i++) {
    try {
      const message = await client.chat.completions.create({
        model: 'mistral',
        temperature: temp,
        messages: [{ role: 'user', content: prompt }]
      });
      console.log(`  Run ${i + 1}: ${message.choices[0].message.content}`);
    } catch (error) {
      console.error(`  Error: ${error.message}`);
    }
  }
}

Run it: python temperature.py (or node temperature.mjs)

Step 4: Observe Token Usage

Token counts drive API costs and context limits — two concepts you'll use throughout the entire course. This step trains you to notice how different prompts and settings affect token consumption, so it becomes instinctive. This uses the message.usage object from Step 1.

Create a new file called token_usage.py (or token_usage.mjs):

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")

tests = [
    ("Short prompt", "Hi!", 50),
    ("Medium prompt", "Explain what a large language model is in detail.", 200),
    ("Long prompt with constraint", "Write a 3-paragraph essay about the history of computing.", 1024),
]

for label, prompt, max_tok in tests:
    try:
        message = client.chat.completions.create(
            model="mistral",
            messages=[{"role": "user", "content": prompt}]
        )
        u = message.usage
        print(f"{label}:")
        print(f"  Input tokens:  {u.prompt_tokens}")
        print(f"  Output tokens: {u.completion_tokens}")
        print(f"  Total tokens:  {u.prompt_tokens + u.completion_tokens}\n")
    except Exception as e:
        print(f"Error: {e}")

import OpenAI from 'openai';

const client = new OpenAI({ baseURL: 'http://localhost:11434/v1', apiKey: 'ollama' });

const tests = [
  ['Short prompt', 'Hi!', 50],
  ['Medium prompt', 'Explain what a large language model is in detail.', 200],
  ['Long prompt with constraint', 'Write a 3-paragraph essay about the history of computing.', 1024],
];

for (const [label, prompt, maxTok] of tests) {
  try {
    const message = await client.chat.completions.create({
      model: 'mistral',
      max_tokens: maxTok,
      messages: [{ role: 'user', content: prompt }]
    });
    const u = message.usage;
    console.log(`${label}:`);
    console.log(`  Input tokens:  ${u.prompt_tokens}`);
    console.log(`  Output tokens: ${u.completion_tokens}`);
    console.log(`  Total tokens:  ${u.prompt_tokens + u.completion_tokens}\n`);
  } catch (error) {
    console.error(`Error: ${error.message}`);
  }
}

Run it: python token_usage.py (or node token_usage.mjs)

Step 5 (Stretch Goal): Build a CLI Chat

from openai import OpenAI

client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
conversation = []

print("Chat with the model! (type 'quit' to exit)\n")

while True:
    user_input = input("You: ").strip()
    if user_input.lower() in ("quit", "exit"):
        break
    if not user_input:
        continue

    conversation.append({"role": "user", "content": user_input})

    try:
        response = client.chat.completions.create(
            model="mistral",
            messages=[{"role": "system", "content": "You are a friendly, helpful assistant."}] + conversation
        )
        assistant_msg = response.choices[0].message.content
        conversation.append({"role": "assistant", "content": assistant_msg})
        print(f"\nClaude: {assistant_msg}\n")
    except Exception as e:
        print(f"\nError: {e}\n")
        # Remove the failed user message so conversation stays consistent
        conversation.pop()

import OpenAI from 'openai';
import * as readline from 'readline';

const client = new OpenAI({ baseURL: 'http://localhost:11434/v1', apiKey: 'ollama' });
const conversation = [];

const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout
});

console.log("Chat with the model! (type 'quit' to exit)\n");

function ask() {
  rl.question('You: ', async (userInput) => {
    userInput = userInput.trim();
    if (['quit', 'exit'].includes(userInput.toLowerCase())) {
      rl.close();
      return;
    }
    if (!userInput) { ask(); return; }

    conversation.push({ role: 'user', content: userInput });

    try {
      const response = await client.chat.completions.create({
        model: 'mistral',
        messages: [{ role: 'system', content: 'You are a friendly, helpful assistant.' }, ...conversation],
      });
      const assistantMsg = response.choices[0].message.content;
      conversation.push({ role: 'assistant', content: assistantMsg });
      console.log(`\nClaude: ${assistantMsg}\n`);
    } catch (error) {
      console.error(`\nError: ${error.message}\n`);
      conversation.pop();
    }
    ask();
  });
}
ask();

Bonus Step 6: Experience the Multimodal Mode

The Model Zoo section introduced four model types. You’ve been using the generative mode the whole lab. This bonus step lets you experience the multimodal mode: pass an image URL alongside a text question and the model describes what it sees. No additional API key required.

The script (model_zoo_lab.py / model_zoo_lab.mjs) is in the starter folder alongside the other labs. Fill in the two TODO functions, then run it:

from openai import OpenAI
from dotenv import load_dotenv

load_dotenv()
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
MODEL = "mistral"   # swap to "llama3" or "phi3" for other models

# --- GENERATIVE (text → text) ---
def generative_call(question: str) -> str:
    response = client.chat.completions.create(
        model=MODEL,
        messages=[{"role": "user", "content": question}],
    )
    return response.choices[0].message.content

# --- MULTIMODAL (image + text → text) — needs a vision model: `ollama pull llava` ---
def multimodal_call(image_url: str, question: str) -> str:
    response = client.chat.completions.create(
        model="llava",   # Mistral is text-only; images need a multimodal model
        messages=[{
            "role": "user",
            "content": [
                {"type": "text", "text": question},
                # OpenAI-compatible image block (Ollama accepts a URL or a base64 data: URI)
                {"type": "image_url", "image_url": {"url": image_url}},
            ],
        }],
    )
    return response.choices[0].message.content

IMAGE_URL = "https://upload.wikimedia.org/wikipedia/commons/thumb/3/3f/Bikesgray.jpg/320px-Bikesgray.jpg"

print(generative_call("What is the Eiffel Tower? One sentence."))
print(multimodal_call(IMAGE_URL, "Describe this image in one sentence."))

import OpenAI from 'openai';
import 'dotenv/config';

const client = new OpenAI({ baseURL: 'http://localhost:11434/v1', apiKey: 'ollama' });
const MODEL = 'mistral';

async function generativeCall(question) {
  const r = await client.chat.completions.create({
    model: MODEL,    messages: [{ role: 'user', content: question }],
  });
  return r.choices[0].message.content;
}

// needs a vision model: `ollama pull llava`
async function multimodalCall(imageUrl, question) {
  const r = await client.chat.completions.create({
    model: 'llava',    // Mistral is text-only; images need a multimodal model
    messages: [{
      role: 'user',
      content: [
        { type: 'text', text: question },
        // OpenAI-compatible image block (Ollama accepts a URL or a base64 data: URI)
        { type: 'image_url', image_url: { url: imageUrl } },
      ],
    }],
  });
  return r.choices[0].message.content;
}

const IMAGE_URL = 'https://upload.wikimedia.org/wikipedia/commons/thumb/3/3f/Bikesgray.jpg/320px-Bikesgray.jpg';

console.log(await generativeCall('What is the Eiffel Tower? One sentence.'));
console.log(await multimodalCall(IMAGE_URL, 'Describe this image in one sentence.'));

Why not call an embedding or reranker here? Those models need a separate provider key (Voyage, Cohere, or OpenAI) and only make sense once you have a document corpus to search. You’ll build the full pipeline in M09 (embedding + vector index) and M10 (reranking). For now, the four-type taxonomy is your design compass.

Verify Everything Works

Run all scripts in sequence to confirm your setup is complete:

# Python
python hello.py && python system_prompts.py && python temperature.py && python token_usage.py

# Bonus: multimodal (requires internet access for the image URL)
python model_zoo_lab.py

# Node.js
node hello.mjs && node system_prompts.mjs && node temperature.mjs && node token_usage.mjs
node model_zoo_lab.mjs

🏢 Want to Run This Without an API Key?

You're already here — this track uses Ollama + Mistral-7B for all code examples. All the concepts apply to any LLM. The comparison below shows what the same call looks like in the main course (Claude/Anthropic SDK) vs. this track (OpenAI SDK + Ollama), so you can follow along with either version.

# Main course: Claude via Anthropic SDK
from anthropic import Anthropic
client = Anthropic()
response = client.messages.create(
    model="claude-sonnet-4-6", max_tokens=512,
    messages=[{"role": "user", "content": "Hello!"}]
)
text = response.content[0].text

# This track: Mistral-7B via Ollama (ollama pull mistral)
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
response = client.chat.completions.create(
    model="mistral",
    messages=[{"role": "user", "content": "Hello!"}]
)
text = response.choices[0].message.content

Three differences: client initializer, model name, and response path (content[0].text vs choices[0].message.content). Every other concept — system prompts, temperature, multi-turn history, tool use — maps directly between the two.

Module M28 covers the full swap: Ollama setup, hardware requirements, LiteLLM for provider-agnostic agents, and an honest comparison of what Claude-specific features have no open-source equivalent. You can take M28 anytime — it’s a standalone appendix module.

Knowledge Check

Test your understanding of the concepts from this module. Select the best answer for each question.

message = client.chat.completions.create(
    model="mistral",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(message.______[0].text)

Module Summary