The 5 ideas of context engineering

Tap any concept to jump to its 5-card cluster.

• 1Prompt vs. ContextWriting the message vs. curating the stack
• 2The Six LayersWhat the model actually sees each call
• 3The Four LeversCrop, Compress, Summarize, Select
• 4Lost in the MiddlePosition decides what gets recalled
• 5Context RotHow long sessions poison themselves

Prompting is the tip; context is the iceberg

Prompt engineering is writing one good message: roles, examples, format instructions. That's the visible part. Underneath sits the bigger discipline — context engineering: deciding what content occupies the model's window on each turn, in what order, and how it's compressed or fetched.

Every call assembles a whole stack — system prompt, tools, history, retrieved docs, tool results, your message — and the model attends to all of it. Most learners think only about the last line they typed. The other 95% sets the priors, the constraints, and the failure modes.

The evidence binder vs. the closing argument

BEFORE: A lawyer hands the jury a thick evidence binder — contracts, photos, prior testimony — then delivers a closing argument. The jury hears the closing last, but they've been reading that binder for hours.

PAIN: Rookie lawyers obsess over the closing and ignore the binder. But the jury's verdict rests mostly on the binder. A brilliant closing on top of a sloppy binder still loses the case.

MAPPING: The closing argument is your prompt. The binder is your context. The model is the jury — it reads everything. If retrieved chunks are irrelevant, tool defs are bloated, or history is full of resolved errors, no clever prompting fixes the verdict.

Two different questions

1. Prompt engineering asks"What words go in this one message?" A writing problem — roles, examples, format.
2. Context engineering asks"What gets included at all, in what order, full or summarized?" A budgeting problem.
3. The window is finiteEverything competes for the same token budget — and the response needs room too.
4. Local models feel it firstMistral 7B's 32K window fills before turn 20 in a typical research agent. Budgeting isn't optional.

Which discipline is this?

IF the answer is worded badly
   but the right facts were present:
  → it's a PROMPT problem
     (fix the message wording)

IF the right facts were missing,
   buried, stale, or crowded out:
  → it's a CONTEXT problem
     (fix what's in the window)

# most "bad model" days are
# actually context problems.

On a 32K Mistral window you hit the budget fast — so the context half is where local-model agents live or die.

Misconceptions

Every call is six layers, not one message

Before you can engineer context, you have to see what's in it. On every call the model receives a layered stack: system prompt, tool definitions, retrieved docs, conversation history, tool results, and finally your current message.

Most of it isn't in the messages array you passed — it's assembled from parameters. On turn 6 of a Mistral order-tracking agent, the user types 72 tokens but the model receives ~7,300. Ninety-nine percent of what shapes the answer is content you assembled.

The desk before you start writing

BEFORE: Imagine a researcher sitting down to answer one question. On the desk already: the house style guide, a rack of tools, a stack of pulled reference pages, the meeting notes so far, printouts of earlier lookups — and only then, the sticky note with today's question.

PAIN: If you only think about the sticky note, you forget the desk is already covered. A cluttered desk with the wrong reference pages produces a confident, wrong answer — and you never notice, because you only wrote the sticky note.

MAPPING: The six layers are that desk. Style guide = system prompt; tool rack = tool definitions; reference pages = retrieved docs; meeting notes = history; lookup printouts = tool results; sticky note = current message. Engineer the desk, not just the note.

The stack, top to bottom

1. System + tools (static)Persona, rules, and every tool's JSON schema — sent every turn even if no tool is called.
2. Retrieved docs (per-turn)Chunks your RAG pipeline injects for this specific query.
3. History + tool results (growing)Every prior turn and past tool output — the main driver of overflow in long sessions.
4. Current message (smallest)Most-attended end position, but shaped by everything stacked above it.

Pseudocode

FUNCTION assemble_context(turn):
  stack = []
  stack.add(system_prompt)      # static
  stack.add(tool_definitions)   # static, always sent
  stack.add(retrieved_docs)     # per-query
  stack.add(history)            # grows each turn
  stack.add(tool_results)       # accumulates
  stack.add(current_message)    # smallest

  used = sum(count_tokens(x) FOR x IN stack)
  # used ~ 7,300 tok — user typed only 72
  RETURN stack, used

Misconceptions

Four moves, that's all you get

When the stack gets too big, too noisy, or too stale, every fix reduces to one of four moves: Crop (remove a whole layer), Compress (shorten what stays), Summarize (collapse old history into a dense gist), and Select (retrieve only the relevant docs on demand).

That's the entire toolbox. Every RAG technique, every memory architecture, every multi-agent pattern later in the course is one of these four wearing a different costume. When an agent degrades, the question is simply: which lever do I pull?

Packing a suitcase that's already full

BEFORE: You're packing for a two-week trip and the suitcase won't close. For any item you have four options, not one.

PAIN: Most people default to cramming more in. The zipper breaks, and something critical — your laptop charger — gets left behind because something had to give at the last second.

MAPPING: The four real options are: throw it out (Crop), use packing cubes so it takes half the space (Compress), ship it ahead as a note-to-self (Summarize), or buy it on arrival only when needed (Select). Those are exactly Crop, Compress, Summarize, Select — the only four moves for curating context.

The four levers

Crop is zero-risk when a layer clearly doesn't apply. Summarize is lossy — always instruct it to preserve exact IDs and numbers.

Which lever fits the problem?

IF a whole layer doesn't apply
   this turn (no tools, no docs):
  → CROP it out

IF a layer is huge but only a
   few fields matter (big JSON):
  → COMPRESS to those fields

IF history is long and old turns
   are low-relevance:
  → SUMMARIZE old turns

IF the knowledge base is bigger
   than ~3 docs:
  → SELECT via retrieval

Misconceptions

Where you put a fact decides if it's recalled

Even if every byte in your context is high quality, position matters. Models attend far more strongly to content at the start and end of the window than to content in the middle. This is the documented "lost in the middle" effect.

Recall runs roughly ~74% at the start, ~72% at the end, but drops to ~43% dead center. And it's worse on smaller models — Mistral 7B and Gemma 2 9B suffer more sharply than a 70B model, because smaller attention heads spread thinner across the window.

The 20-bullet meeting agenda

BEFORE: A meeting runs through a 20-bullet agenda. A week later, ask a participant what was decided.

PAIN: They remember bullet 1 (the opening goal), bullet 20 (the action items), and maybe the one heated argument. Bullets 8–14? Gone — even though they heard every word. The most important constraint got put at bullet 11, and now nobody implements it.

MAPPING: The model reads context the same way. Position 1 and the very end are recalled with high fidelity; the middle of a long context, even when literally present, can be effectively invisible to the model's reasoning — especially on smaller local models.

Recall by position (Mistral 7B)

1. Critical instruction → startPut it at line 1 of the system prompt, never buried at line 40.
2. Best chunk → endPlace the most-relevant retrieved doc closest to the user's question.
3. Repeat key factsOnce at the top, once just before the message — 50 tokens buys 20–30% recall.

Where does this fact go?

IF it's a must-hold constraint:
  → put it at the START of the
     system prompt AND repeat it
     just before the user message

IF it's the key retrieved chunk:
  → place it LAST in the doc block
     (re-rank so it lands there)

IF you're on Gemma 2 9B:
  → treat MIDDLE recall as ~0;
     keep the KB at the very end

Misconceptions

Long sessions poison themselves

Long-running agents accumulate junk: tool results from searches that went nowhere, abandoned plans, superseded instructions ("actually, ignore that"), resolved errors. The model dutifully attends to all of it — including the parts that no longer apply.

The result: by turn 20 the agent retries already-failed lookups, contradicts its own corrections, and answers questions you never asked. Critically, context rot is a signal-to-noise problem, not a budget problem — you can hit it at 55% window utilization with plenty of tokens to spare.

The email forwarded fifteen times

BEFORE: An email thread gets forwarded fifteen times, everyone leaving the full quoted history. The actual decision is buried under "Re: Re: Re: FW: FW:".

PAIN: A new participant joins, reads top-down, and acts on stale info from forward #3 — the old price, the abandoned plan. Every reply makes the thread harder to read, not easier. It becomes net-negative information density.

MAPPING: The same thing happens in your model's window. Each turn appends; nothing self-cleans. By turn 20 on Mistral you're reading a thread that's mostly noise, and a critical constraint from turn 8 is buried under tool-result garbage from turns 9–19.

A compaction pass

1. Detect low signal-to-noiseWatch for stale errors, superseded turns, duplicate results piling up.
2. Summarize old turnsFold them into one clean case-fact message; keep recent turns verbatim.
3. Put the question next to the factsNow the current query sits adjacent to what matters, not buried.

Pseudocode

FUNCTION compact(history, keep_recent):
  IF length(history) <= keep_recent:
    RETURN history        # nothing to do

  old    = history[: -keep_recent]
  recent = history[-keep_recent :]

  # SEPARATE model call to summarize —
  # preserve every ID, number, decision;
  # drop retries, dupes, resolved detours
  gist = summarize(old)

  RETURN [ gist ] + recent

Never ask the main model to summarize in the same call — it can't summarize what it's currently reading.

Misconceptions

Test the core insight

One question per concept. Tap to reveal the answer.