M03B: Context Engineering — Curating What the Model Sees

M03 taught you how to write the message. This module teaches you how to curate everything the model sees on each turn — system prompt, tools, history, retrieved docs, tool results — under a finite budget. Get this right and every later module (RAG, memory, multi-agent, caching) snaps into place.

Learning Objectives

• Distinguish prompt engineering (writing the message) from context engineering (curating what the model sees) and explain why both matter
• Inventory the six layers Claude reads on every API call: system, tools, history, retrieved docs, tool results, current turn
• Apply the four context-engineering levers — add, compress, retrieve, offload — to a context budget problem
• Order static and dynamic content correctly so prompt caching can hit, and explain the lost-in-the-middle effect
• Diagnose context rot in a long-running agent transcript and fix it without losing critical case facts

Prompt Engineering vs. Context Engineering

In M03 you learned prompt engineering — the craft of writing a single message that guides Claude toward the answer you want. Roles, few-shot examples, structured output instructions, chain-of-thought. That is the visible iceberg.

Underneath it is the bigger discipline: context engineering. Every API call assembles a stack of content for the model — the system prompt you wrote, the tool definitions you registered, the conversation history that has piled up, any retrieved documents, the results of past tool calls, and finally the user's current message. The model attends to all of it. Most learners only think about the last line they typed. The other 95% sets the constraints, the priors, and (when neglected) the failure modes.

Here's the working definition we'll use throughout the course:

Why split them out? Because the levers are different. Prompt engineering asks "what words go in this message?" Context engineering asks "what gets included at all, in what order, fresh or cached, full or summarized?" The first is a writing problem. The second is a budgeting problem. By M11 you'll be juggling both on every agent you build — this module makes the budgeting half explicit.

Concept 1: What Does the Model Actually See?

Before you can engineer context, you have to know what's in it. On every Anthropic API call, Claude receives a layered stack of content. Most of it isn't in the messages array you passed in — it's assembled by the SDK from various parameters. Let's enumerate the six layers.

The Six Layers, From Top to Bottom

Every Anthropic API call assembles these six layers into one giant promptNote: in the SDK you don't see "the prompt" as one string — you pass system, tools, messages separately. But internally the model receives a single sequence of tokens assembled from all of them. the model reads in order. Each one costs tokens, each one influences the answer:

1. System prompt — your instructions, persona, output rules. Set once per conversation. Static across turns.
2. Tool definitions — the JSON Schema for every tool you've registered. Even if Claude doesn't use any, it reads them all to decide. Static across turns.
3. Conversation history — every prior user/assistant message, in order. Grows with every turn.
4. Retrieved documents — chunks injected by your RAG pipeline (M09). Different on every turn.
5. Tool results — the JSON output of past tool calls, embedded as messages. Accumulates with every tool use.
6. Current user turn — the message you just appended. Smallest, most-attended layer.

Here's what that stack looks like for a UCC research agent on turn 8 of a session:

Concept 2: The Four Levers

Now that you can see the stack, the question becomes: what do you do when it gets too big, too noisy, or too stale? Every context-engineering decision — every one — comes down to one of four moves. The rest of Track 3 is just specialized applications of these four.

Here are the four levers, with what each one does to a single fact:

When to Reach for Each Lever

A simple decision heuristic, refined throughout Track 3:

• Add when the fact is small, accessed every turn, and exact wording matters — e.g. policy rules, the user's name, current date.
• Compress when the fact's gist matters but specifics are nice-to-have — e.g. "we already discussed the auth flow and concluded X" replacing 12 turns of debate.
• Retrieve when the fact is one of many candidates and only relevant ones matter — e.g. one document out of 10,000 in a knowledge base.
• Offload when the fact requires its own reasoning chain that would otherwise pollute your main context — e.g. "research the entire filing history of this debtor" handed to a subagent.

Concept 3: Static vs. Dynamic Context

Inside your context stack, some layers don't change between turns — the system prompt, the tool definitions, sometimes a fixed reference doc. Other layers change constantly — the conversation history grows, retrieved chunks differ per query. This split has a name and a consequence.

The Ordering Rule

The rule is simple: static blocks first, dynamic blocks last. The cache reuses a prefix as long as the prefix is byte-identical to a previous call. As soon as the prefix diverges, every byte after that point is "new" — even if it would have been identical further down. So the order has to be: static system, static tools, static reference doc, then dynamic history, then dynamic current turn.

Concept 4: Position Effects — Lost in the Middle

Even if every byte you put in context is high-quality, where you put it matters. Large language models attend more strongly to content at the start and end of the context window than to content in the middle. This isn't a bug to fix — it's a property of how attention layers work, replicated across nearly every model family. It's been documented under the name "lost in the middleFrom the 2023 Liu et al. paper "Lost in the Middle: How Language Models Use Long Contexts." Researchers placed the same answer at different positions in a long context and measured retrieval accuracy. Accuracy was high at the start (~75%), high at the end (~70%), and dropped to ~45% in the middle.."

Here's what that looks like when researchers test the same fact at different positions:

What This Means For Your Code

Three concrete rules drop out of the position-effects research:

1. Put the most critical instruction at the start of the system prompt, not buried at line 47.
2. Put the most-relevant retrieved chunk at the end of the retrieved-docs block (closest to the user's current question). Re-ranking pipelines exist exactly for this reason — they reorder chunks so the best one is last.
3. Repeat critical case facts — once in the system prompt (start position) and once just before the user message (end position). This redundancy costs ~50 tokens and meaningfully improves recall.

Concept 5: Context Rot

Long-running agents accumulate junk. Tool results from searches that went nowhere. Plans that got abandoned three turns later. User instructions that were superseded ("actually, ignore that last bit"). Errors that were resolved. The model dutifully attends to all of it — including the parts that no longer apply. The result: by turn 30, your agent retries already-failed searches, contradicts its own corrections, and answers questions you never asked.

Here's a real symptom you'll see in production: the agent works great for the first 10 turns, then starts visibly degrading. Token usage may still be well under the limit. The fix isn't a bigger window — it's a smaller, cleaner one. You compact, summarize, or evict the rotted content.

Code Walkthrough: The ContextBudget Class

Time to make this concrete. We're going to build a small class that takes the six layers of your context, accounts for tokens in each, and applies one of the four levers when the total exceeds a budget. You'll use this directly in the lab and reuse the pattern in M08 (history) and M11 (memory).

Here's the design: ContextBudget is initialized with a target budget and a strategy. You feed it the six layers via constructor. It exposes account() for the per-layer breakdown, fits() for the budget check, and build_messages() for the assembled message array ready for the Anthropic SDK. We'll annotate the implementation in four chunks.

Chunk 1: Per-layer Token Accounting

WHAT: The constructor stores the six layers separately and counts tokens in each. WHY: You can't apply a lever intelligently without knowing which layer is the heaviest. GOTCHA: Don't use len(text) // 4 as a token count — that approximation is off by 20%+ for tool definitions and JSON-heavy content. Use anthropic.Anthropic().count_tokens() for production code; we'll use a fast heuristic plus a real API call wrapper for the lab.

from dataclasses import dataclass, field
from typing import Optional

# Fast heuristic: ~4 chars per token. Good enough for budgeting decisions.
# For production cost tracking, use anthropic.count_tokens().
def estimate_tokens(text: str) -> int:
    return max(1, len(text) // 4)

@dataclass
class ContextBudget:
    """Accounts for and curates the six layers of context."""
    system_prompt: str = ""
    tool_definitions: list = field(default_factory=list)  # list of dicts
    history: list = field(default_factory=list)            # list of {"role", "content"}
    retrieved_docs: list = field(default_factory=list)     # list of strings
    tool_results: list = field(default_factory=list)       # list of strings
    current_user_message: str = ""
    target_budget: int = 8000  # tokens

    def account(self) -> dict:
        """Return per-layer token counts."""
        import json
        return {
            "system": estimate_tokens(self.system_prompt),
            "tools": estimate_tokens(json.dumps(self.tool_definitions)),
            "history": sum(estimate_tokens(m["content"]) for m in self.history),
            "retrieved": sum(estimate_tokens(d) for d in self.retrieved_docs),
            "tool_results": sum(estimate_tokens(r) for r in self.tool_results),
            "current": estimate_tokens(self.current_user_message),
        }

    def total(self) -> int:
        return sum(self.account().values())

    def fits(self) -> bool:
        return self.total() <= self.target_budget

// Fast heuristic: ~4 chars per token. Good enough for budgeting decisions.
// For production cost tracking, use anthropic.countTokens().
function estimateTokens(text) {
  return Math.max(1, Math.floor((text || '').length / 4));
}

export class ContextBudget {
  constructor({
    systemPrompt = '',
    toolDefinitions = [],
    history = [],            // [{ role, content }]
    retrievedDocs = [],      // [string]
    toolResults = [],        // [string]
    currentUserMessage = '',
    targetBudget = 8000,
  } = {}) {
    Object.assign(this, {
      systemPrompt, toolDefinitions, history,
      retrievedDocs, toolResults, currentUserMessage, targetBudget,
    });
  }

  account() {
    return {
      system: estimateTokens(this.systemPrompt),
      tools: estimateTokens(JSON.stringify(this.toolDefinitions)),
      history: this.history.reduce((s, m) => s + estimateTokens(m.content), 0),
      retrieved: this.retrievedDocs.reduce((s, d) => s + estimateTokens(d), 0),
      toolResults: this.toolResults.reduce((s, r) => s + estimateTokens(r), 0),
      current: estimateTokens(this.currentUserMessage),
    };
  }

  total() {
    return Object.values(this.account()).reduce((a, b) => a + b, 0);
  }

  fits() {
    return this.total() <= this.targetBudget;
  }
}

Chunk 2: The Compress Lever

WHAT: When the budget is busted, replace old turns with a Claude-generated summary, keeping the most recent k turns verbatim. WHY: History is usually the heaviest layer in long sessions, and old turns have lower relevance per token than recent ones. GOTCHA: Use a separate Claude call to write the summary, with the explicit instruction "preserve all named entities, IDs, and numeric values." Without that instruction, summarization eats critical specifics first — this is the cert-tip flag for Domain 5.1.

import anthropic

SUMMARY_INSTRUCTION = """Summarize the following conversation in 3-5 sentences.
CRITICAL: preserve every named entity (people, companies, IDs), exact numeric
values (dollar amounts, dates, counts), and any explicit user instructions.
Drop only redundant or resolved tool errors and abandoned plans.

Conversation:
{transcript}

Summary:"""

def compress_history(history: list, keep_recent: int = 6) -> list:
    """Summarize older turns; keep the last `keep_recent` turns verbatim."""
    if len(history) <= keep_recent:
        return history  # nothing to compress

    older = history[:-keep_recent]
    recent = history[-keep_recent:]

    transcript = "\n".join(f"{m['role']}: {m['content']}" for m in older)
    client = anthropic.Anthropic()
    try:
        result = client.messages.create(
            model="claude-haiku-4-5-20251001",  # cheap model for summarization
            max_tokens=400,
            messages=[{"role": "user",
                       "content": SUMMARY_INSTRUCTION.format(transcript=transcript)}],
        )
        summary_text = result.content[0].text
    except anthropic.APIError as e:
        # If summarization fails, fall back to truncation rather than crashing.
        print(f"Summarization failed: {e.message}; falling back to truncation.")
        return recent

    summary_msg = {"role": "user",
                   "content": f"[Summary of {len(older)} earlier turns] {summary_text}"}
    return [summary_msg] + recent

import Anthropic from '@anthropic-ai/sdk';

const SUMMARY_INSTRUCTION = `Summarize the following conversation in 3-5 sentences.
CRITICAL: preserve every named entity (people, companies, IDs), exact numeric
values (dollar amounts, dates, counts), and any explicit user instructions.
Drop only redundant or resolved tool errors and abandoned plans.

Conversation:
{transcript}

Summary:`;

export async function compressHistory(history, keepRecent = 6) {
  if (history.length <= keepRecent) return history;

  const older = history.slice(0, -keepRecent);
  const recent = history.slice(-keepRecent);
  const transcript = older.map(m => `${m.role}: ${m.content}`).join('\n');

  const client = new Anthropic();
  let summaryText;
  try {
    const result = await client.messages.create({
      model: 'claude-haiku-4-5-20251001',
      max_tokens: 400,
      messages: [{ role: 'user',
                   content: SUMMARY_INSTRUCTION.replace('{transcript}', transcript) }],
    });
    summaryText = result.content[0].text;
  } catch (error) {
    console.error(`Summarization failed: ${error.message}; falling back to truncation.`);
    return recent;
  }

  const summaryMsg = {
    role: 'user',
    content: `[Summary of ${older.length} earlier turns] ${summaryText}`,
  };
  return [summaryMsg, ...recent];
}

Chunk 3: Strategy Dispatch

WHAT: One method picks the right lever based on the budget overflow. WHY: You don't want strategy selection scattered across your agent code — centralize it. GOTCHA: The order of fallback matters — we compress history before evicting retrieved docs, because retrieval was already curated by the RAG pipeline whereas history is "everything that ever happened." If that still overflows, we'd reach for retrieve/offload (left as stretch goals).

    def apply_compress(self, keep_recent: int = 6) -> "ContextBudget":
        """Compress history; return self for chaining."""
        self.history = compress_history(self.history, keep_recent=keep_recent)
        return self

    def apply_strategy(self, strategy: str = "auto") -> "ContextBudget":
        """Dispatch to a specific lever, or auto-pick based on overflow.
        Strategies: 'compress', 'truncate_retrieved', 'auto', 'noop'.
        """
        if self.fits() or strategy == "noop":
            return self

        if strategy == "compress" or strategy == "auto":
            self.apply_compress()

        if not self.fits() and strategy in ("truncate_retrieved", "auto"):
            # Last resort: keep only the top 2 retrieved chunks.
            self.retrieved_docs = self.retrieved_docs[:2]

        return self

// Add these methods to ContextBudget class

  async applyCompress(keepRecent = 6) {
    this.history = await compressHistory(this.history, keepRecent);
    return this;
  }

  async applyStrategy(strategy = 'auto') {
    if (this.fits() || strategy === 'noop') return this;

    if (strategy === 'compress' || strategy === 'auto') {
      await this.applyCompress();
    }

    if (!this.fits() && (strategy === 'truncateRetrieved' || strategy === 'auto')) {
      this.retrievedDocs = this.retrievedDocs.slice(0, 2);
    }
    return this;
  }

Chunk 4: Static-First Assembly

WHAT: build_messages() assembles the final SDK payload with static blocks first, dynamic last — the cache-friendly order from Concept 3. WHY: This is where Concept 3's ordering rule turns into actual money saved. GOTCHA: Retrieved docs are technically dynamic (they change per query), but we put them before history because in many RAG pipelines the same chunks recur for similar queries within a session — partial cache hits are still wins.

    def build_messages(self) -> tuple[str, list, list]:
        """Return (system, tools, messages) ready for client.messages.create().
        Order: STATIC (system, tools) - SEMI-STATIC (retrieved) - DYNAMIC (history, current).
        """
        # Retrieved docs prepended to first message (closer to the question = better recall).
        retrieved_block = ""
        if self.retrieved_docs:
            retrieved_block = "\n\n<reference_docs>\n" + \
                "\n---\n".join(self.retrieved_docs) + "\n</reference_docs>\n\n"

        messages = list(self.history)  # copy to avoid mutating caller's list
        # Append current user turn with retrieved docs at the END (high-recall position).
        messages.append({
            "role": "user",
            "content": retrieved_block + self.current_user_message,
        })
        return self.system_prompt, self.tool_definitions, messages

  buildMessages() {
    let retrievedBlock = '';
    if (this.retrievedDocs.length) {
      retrievedBlock = '\n\n<reference_docs>\n' +
        this.retrievedDocs.join('\n---\n') + '\n</reference_docs>\n\n';
    }

    const messages = [
      ...this.history,
      { role: 'user', content: retrievedBlock + this.currentUserMessage },
    ];
    return {
      system: this.systemPrompt,
      tools: this.toolDefinitions,
      messages,
    };
  }

Hands-On Lab: The Poisoned Transcript

Environment Setup

Single copy-pasteable block. Pick Python or Node.js, not both:

# Python
mkdir m03b-context-lab && cd m03b-context-lab
python -m venv venv && source venv/bin/activate    # Windows: venv\Scripts\activate
pip install anthropic
export ANTHROPIC_API_KEY=sk-ant-...                # Windows: set ANTHROPIC_API_KEY=sk-ant-...

# Node.js (alternative)
mkdir m03b-context-lab && cd m03b-context-lab
npm init -y && npm install @anthropic-ai/sdk
export ANTHROPIC_API_KEY=sk-ant-...

Step 1: Drop in the Poisoned Transcript Fixture

This is your raw material — a UCC research session that started fine and rotted by turn 30. The user is asking about debtor "Acme Corp" filings; over the session they corrected scope twice, hit two rate limits, and abandoned one tool path. The current question is at the end. Save this as poisoned_transcript.json:

{
  "system_prompt": "You are a UCC filings research assistant. Help users find and analyze public records filings. Always cite filing IDs. If a search fails, retry once with a normalized query.",
  "tool_definitions": [
    {"name": "search_filings", "description": "Search UCC filings by debtor name", "input_schema": {"type": "object", "properties": {"debtor": {"type": "string"}}, "required": ["debtor"]}},
    {"name": "get_filing", "description": "Retrieve a single filing by ID", "input_schema": {"type": "object", "properties": {"filing_id": {"type": "string"}}, "required": ["filing_id"]}},
    {"name": "calc_collateral_total", "description": "Sum collateral values across filing IDs", "input_schema": {"type": "object", "properties": {"filing_ids": {"type": "array", "items": {"type": "string"}}}, "required": ["filing_ids"]}}
  ],
  "history": [
    {"role": "user", "content": "Research debtor Acme Corp - I need an overview of their UCC filings"},
    {"role": "assistant", "content": "I'll search for filings under 'Acme Corp'."},
    {"role": "user", "content": "[tool_result] search_filings(debtor='Acme Corp') returned 12 filings spanning 2017-2023, IDs F-2017-0044, F-2018-04421, F-2018-04422, F-2019-1101, F-2019-2287, F-2020-0883, F-2020-1455, F-2021-0192, F-2021-3340, F-2022-0028, F-2022-1199, F-2023-0501"},
    {"role": "assistant", "content": "Found 12 filings from 2017 to 2023. Want a year-by-year breakdown?"},
    {"role": "user", "content": "Yes, but actually focus only on 2019 and later"},
    {"role": "assistant", "content": "Got it. Restricting to 2019+: 8 filings. Pulling details."},
    {"role": "user", "content": "[tool_result] get_filing('F-2018-04421') returned full filing data for 2018 (collateral: $8.1M, secured party: First Bank)"},
    {"role": "assistant", "content": "I see I pulled a 2018 filing by mistake. Discarding."},
    {"role": "user", "content": "[tool_result] get_filing('F-2018-04422') returned ERROR: rate_limit_exceeded after 3 retries"},
    {"role": "assistant", "content": "Rate limit on another 2018 filing - skipping, that's outside our scope anyway."},
    {"role": "user", "content": "[tool_result] search_filings(debtor='ACME CORP') returned 12 filings (same as before, normalization quirk)"},
    {"role": "assistant", "content": "Duplicate result from a normalized retry. Ignoring."},
    {"role": "user", "content": "I said 2019+! Why are you still pulling 2018 filings?"},
    {"role": "assistant", "content": "Apologies - that was a confused retry. Going forward, only 2019 and later: F-2019-1101, F-2019-2287, F-2020-0883, F-2020-1455, F-2021-0192, F-2021-3340, F-2022-0028, F-2022-1199, F-2023-0501. That's 9 filings."},
    {"role": "user", "content": "[tool_result] get_filing('F-2019-1101') returned collateral: $4.2M, secured party: Capital Trust, year: 2019"},
    {"role": "assistant", "content": "F-2019-1101: $4.2M collateral, Capital Trust as secured party."},
    {"role": "user", "content": "[tool_result] get_filing('F-2019-2287') returned collateral: $7.8M, secured party: First Bank, year: 2019"},
    {"role": "assistant", "content": "F-2019-2287: $7.8M, First Bank."},
    {"role": "user", "content": "[tool_result] get_filing('F-2020-0883') returned collateral: $11.5M, secured party: Capital Trust, year: 2020"},
    {"role": "assistant", "content": "F-2020-0883: $11.5M, Capital Trust."},
    {"role": "user", "content": "[tool_result] get_filing('F-2020-1455') returned collateral: $3.4M, secured party: GE Capital, year: 2020"},
    {"role": "assistant", "content": "F-2020-1455: $3.4M, GE Capital."},
    {"role": "user", "content": "[tool_result] get_filing('F-2021-0192') returned collateral: $9.0M, secured party: First Bank, year: 2021"},
    {"role": "assistant", "content": "F-2021-0192: $9.0M, First Bank."},
    {"role": "user", "content": "[tool_result] get_filing('F-2021-3340') returned collateral: $5.5M, secured party: Capital Trust, year: 2021"},
    {"role": "assistant", "content": "F-2021-3340: $5.5M, Capital Trust."},
    {"role": "user", "content": "[tool_result] get_filing('F-2022-0028') returned collateral: $13.2M, secured party: First Bank, year: 2022"},
    {"role": "assistant", "content": "F-2022-0028: $13.2M, First Bank."},
    {"role": "user", "content": "[tool_result] get_filing('F-2022-1199') returned collateral: $6.7M, secured party: GE Capital, year: 2022"},
    {"role": "assistant", "content": "F-2022-1199: $6.7M, GE Capital."},
    {"role": "user", "content": "[tool_result] get_filing('F-2023-0501') returned collateral: $8.9M, secured party: Capital Trust, year: 2023"},
    {"role": "assistant", "content": "F-2023-0501: $8.9M, Capital Trust."}
  ],
  "current_user_message": "What's the total collateral value across all 2019+ filings, and which secured party shows up most often?"
}

What just landed in your folder: a fixture with one system prompt, three tool definitions, 30 history turns (mix of user, assistant, and tool-result messages), and a current question. The poisoning is subtle but real: two rate-limit results, two 2018 filings the user explicitly excluded, one duplicate normalized search. Your diagnostic will surface these.

Step 2: Build the Diagnostic + Compress Runner

What & why: This script loads the fixture, instantiates the ContextBudget class from the walkthrough, prints per-layer token breakdown before any fix, applies the compress lever, then prints the breakdown after. It's the smallest possible loop that exercises the full module.

First, save the ContextBudget class from the Code Walkthrough as context_budget.py (Python) or contextBudget.mjs (Node). Then create the runner:

"""diagnose.py - Reads poisoned_transcript.json, runs ContextBudget, prints before/after."""
import json
from context_budget import ContextBudget

def print_breakdown(label: str, account: dict, total: int, target: int):
    print(f"\n--- {label} ---")
    for layer, tokens in account.items():
        bar = "#" * max(1, tokens // 100)
        print(f"  {layer:<14} {tokens:>5} tok  {bar}")
    pct = round(100 * total / target)
    status = "OVER BUDGET" if total > target else "ok"
    print(f"  TOTAL          {total:>5} tok  ({pct}% of {target} budget) [{status}]")

def detect_rot(history: list) -> list:
    """Quick heuristic rot scan - look for explicit signals in tool-result messages."""
    signals = []
    for i, msg in enumerate(history):
        c = msg.get("content", "")
        if "rate_limit" in c.lower() or "ERROR" in c:
            signals.append(f"  turn {i}: failed tool call (likely retried)")
        if "duplicate" in c.lower() or "same as before" in c.lower():
            signals.append(f"  turn {i}: duplicate result")
    return signals

def main():
    with open("poisoned_transcript.json") as f:
        fixture = json.load(f)

    # Build the budget object from the fixture.
    budget = ContextBudget(
        system_prompt=fixture["system_prompt"],
        tool_definitions=fixture["tool_definitions"],
        history=fixture["history"],
        current_user_message=fixture["current_user_message"],
        target_budget=4000,  # deliberately tight to force compression
    )

    print("=" * 60)
    print("POISONED TRANSCRIPT DIAGNOSTIC")
    print("=" * 60)
    print_breakdown("BEFORE", budget.account(), budget.total(), budget.target_budget)

    rot_signals = detect_rot(budget.history)
    if rot_signals:
        print("\nRot signals detected:")
        for s in rot_signals:
            print(s)

    # Apply the compress lever.
    print("\nApplying compress lever (keep_recent=6)...")
    budget.apply_compress(keep_recent=6)

    print_breakdown("AFTER COMPRESSION", budget.account(), budget.total(), budget.target_budget)

    # Build the final payload that would be sent to Claude.
    system, tools, messages = budget.build_messages()
    print(f"\nFinal payload: {len(messages)} messages, "
          f"{budget.total()} input tokens, fits budget = {budget.fits()}")

if __name__ == "__main__":
    main()

// diagnose.mjs - Reads poisoned_transcript.json, runs ContextBudget, prints before/after.
import { readFileSync } from 'fs';
import { ContextBudget } from './contextBudget.mjs';

function printBreakdown(label, account, total, target) {
  console.log(`\n--- ${label} ---`);
  for (const [layer, tokens] of Object.entries(account)) {
    const bar = '#'.repeat(Math.max(1, Math.floor(tokens / 100)));
    console.log(`  ${layer.padEnd(14)} ${String(tokens).padStart(5)} tok  ${bar}`);
  }
  const pct = Math.round((100 * total) / target);
  const status = total > target ? 'OVER BUDGET' : 'ok';
  console.log(`  TOTAL          ${String(total).padStart(5)} tok  (${pct}% of ${target} budget) [${status}]`);
}

function detectRot(history) {
  const signals = [];
  history.forEach((msg, i) => {
    const c = msg.content || '';
    if (c.toLowerCase().includes('rate_limit') || c.includes('ERROR')) {
      signals.push(`  turn ${i}: failed tool call (likely retried)`);
    }
    if (c.toLowerCase().includes('duplicate') || c.toLowerCase().includes('same as before')) {
      signals.push(`  turn ${i}: duplicate result`);
    }
  });
  return signals;
}

async function main() {
  const fixture = JSON.parse(readFileSync('poisoned_transcript.json', 'utf-8'));

  const budget = new ContextBudget({
    systemPrompt: fixture.system_prompt,
    toolDefinitions: fixture.tool_definitions,
    history: fixture.history,
    currentUserMessage: fixture.current_user_message,
    targetBudget: 4000,
  });

  console.log('='.repeat(60));
  console.log('POISONED TRANSCRIPT DIAGNOSTIC');
  console.log('='.repeat(60));
  printBreakdown('BEFORE', budget.account(), budget.total(), budget.targetBudget);

  const rotSignals = detectRot(budget.history);
  if (rotSignals.length) {
    console.log('\nRot signals detected:');
    rotSignals.forEach(s => console.log(s));
  }

  console.log('\nApplying compress lever (keepRecent=6)...');
  await budget.applyCompress(6);

  printBreakdown('AFTER COMPRESSION', budget.account(), budget.total(), budget.targetBudget);

  const { messages } = budget.buildMessages();
  console.log(`\nFinal payload: ${messages.length} messages, ${budget.total()} input tokens, fits budget = ${budget.fits()}`);
}

main().catch(err => { console.error(err); process.exit(1); });

Run it:

python diagnose.py     # or: node diagnose.mjs

Step 3: Compare Answer Quality Before vs. After

What & why: Token reduction is a means, not the end. The end is answer quality. This step sends the same final question to Claude twice — once with the rotted context, once with the compressed context — and prints both answers so you can compare. Append this to diagnose.py (or create a new file). Important: replace the existing if __name__ == "__main__": block from Step 2 with the new one at the bottom of this snippet so main() only runs once. For the Node.js version, replace the trailing main().catch(...) line from Step 2 with the new top-level await calls at the bottom.

# Append to diagnose.py (or save as compare.py and import as needed)
import anthropic
import time

def ask_claude(budget: ContextBudget) -> tuple[str, dict]:
    """Send the budget's payload to Claude and return (text, usage)."""
    client = anthropic.Anthropic()
    system, tools, messages = budget.build_messages()
    t0 = time.time()
    try:
        result = client.messages.create(
            model="claude-sonnet-4-6",
            max_tokens=400,
            system=system,
            messages=messages,
        )
    except anthropic.APIError as e:
        return f"[error: {e.message}]", {"input_tokens": 0, "output_tokens": 0, "latency_s": 0}
    elapsed = round(time.time() - t0, 2)
    return result.content[0].text, {
        "input_tokens": result.usage.input_tokens,
        "output_tokens": result.usage.output_tokens,
        "latency_s": elapsed,
    }

def compare_run():
    with open("poisoned_transcript.json") as f:
        fixture = json.load(f)

    # Run 1: rotted context (no fix applied)
    rotted = ContextBudget(
        system_prompt=fixture["system_prompt"],
        tool_definitions=fixture["tool_definitions"],
        history=fixture["history"],
        current_user_message=fixture["current_user_message"],
        target_budget=4000,
    )
    print("\n>>> Run 1: ROTTED context (no fix)")
    answer_a, usage_a = ask_claude(rotted)
    print(f"Tokens: {usage_a['input_tokens']} in, {usage_a['output_tokens']} out  ({usage_a['latency_s']}s)")
    print(f"Answer:\n{answer_a}\n")

    # Run 2: compressed context (compress lever applied)
    fixed = ContextBudget(
        system_prompt=fixture["system_prompt"],
        tool_definitions=fixture["tool_definitions"],
        history=fixture["history"],
        current_user_message=fixture["current_user_message"],
        target_budget=4000,
    ).apply_compress(keep_recent=6)
    print(">>> Run 2: COMPRESSED context")
    answer_b, usage_b = ask_claude(fixed)
    print(f"Tokens: {usage_b['input_tokens']} in, {usage_b['output_tokens']} out  ({usage_b['latency_s']}s)")
    print(f"Answer:\n{answer_b}\n")

    # Delta
    print("=" * 60)
    print(f"Token delta:    {usage_a['input_tokens'] - usage_b['input_tokens']} fewer input tokens")
    print(f"Latency delta:  {usage_a['latency_s'] - usage_b['latency_s']:.2f}s faster")

if __name__ == "__main__":
    main()        # original diagnostic
    compare_run() # new comparison

// Append to diagnose.mjs
import Anthropic from '@anthropic-ai/sdk';

async function askClaude(budget) {
  const client = new Anthropic();
  const { system, messages } = budget.buildMessages();
  const t0 = Date.now();
  let result;
  try {
    result = await client.messages.create({
      model: 'claude-sonnet-4-6',
      max_tokens: 400,
      system,
      messages,
    });
  } catch (error) {
    return [`[error: ${error.message}]`, { input_tokens: 0, output_tokens: 0, latency_s: 0 }];
  }
  const elapsed = ((Date.now() - t0) / 1000).toFixed(2);
  return [result.content[0].text, {
    input_tokens: result.usage.input_tokens,
    output_tokens: result.usage.output_tokens,
    latency_s: parseFloat(elapsed),
  }];
}

async function compareRun() {
  const fixture = JSON.parse(readFileSync('poisoned_transcript.json', 'utf-8'));
  const baseArgs = {
    systemPrompt: fixture.system_prompt,
    toolDefinitions: fixture.tool_definitions,
    history: fixture.history,
    currentUserMessage: fixture.current_user_message,
    targetBudget: 4000,
  };

  const rotted = new ContextBudget(baseArgs);
  console.log('\n>>> Run 1: ROTTED context (no fix)');
  const [answerA, usageA] = await askClaude(rotted);
  console.log(`Tokens: ${usageA.input_tokens} in, ${usageA.output_tokens} out  (${usageA.latency_s}s)`);
  console.log(`Answer:\n${answerA}\n`);

  const fixed = new ContextBudget(baseArgs);
  await fixed.applyCompress(6);
  console.log('>>> Run 2: COMPRESSED context');
  const [answerB, usageB] = await askClaude(fixed);
  console.log(`Tokens: ${usageB.input_tokens} in, ${usageB.output_tokens} out  (${usageB.latency_s}s)`);
  console.log(`Answer:\n${answerB}\n`);

  console.log('='.repeat(60));
  console.log(`Token delta:    ${usageA.input_tokens - usageB.input_tokens} fewer input tokens`);
  console.log(`Latency delta:  ${(usageA.latency_s - usageB.latency_s).toFixed(2)}s faster`);
}

await main();
await compareRun();

Run it:

python diagnose.py     # or: node diagnose.mjs

Verify Everything Works

One command runs the full diagnostic + comparison end-to-end:

python diagnose.py    # or: node diagnose.mjs

Stretch Goals (Optional)

• Implement the truncate-retrieved lever: when the budget overflows even after compression, drop retrieved docs to the top 2 by relevance score.
• Static-first ordering test: send the same payload twice with cache_control markers, measure cache hit rate from usage.cache_read_input_tokens.
• Position-effects test: place the critical case fact ("user wants 2019+ only") at three positions — start of system prompt, middle of history, just before the user message — and compare answer accuracy.
• Auto-detect summary failures: parse the summary message after compression and assert that all 9 filing IDs from the original history still appear. If not, fall back to a less aggressive keep_recent.

Knowledge Check

Six questions on context inventory, the four levers, ordering, position effects, and rot.

Module Summary