M10 · The Hybrid Context Stack

Learning Objectives

Three Layers, Not Three Competitors

The viral framing of 2026 — "Graphify kills RAG," "OKF kills vector databases" — is backwards, and you have spent nine modules accumulating the evidence to see why. Graphify's own benchmarks (M04) showed it tying dense vector RAG on QA accuracy and slightly losing on recall. OKF's own spec (M06) ships no search layer at all. Neither of these things is trying to replace retrieval. They are specialized layers that sit alongside a vector index, each justified only by a specific failure mode.

The capability breakdown

Failure-Driven Adoption: The Two Triggers

Here is the discipline that separates architecture from hype-chasing: you add a layer when — and only when — you have observed a concrete failure mode that layer addresses. Not because a benchmark impressed you, not because a launch post went viral, and not because of vague dissatisfaction with your current RAG setup.

That last one deserves numbers. Industry surveys found that 72% of enterprise RAG deployments underdelivered or failed outright in year one — but the dominant causes were poor chunking strategies, outdated vector indices, and inappropriate embedding models, not the absence of a graph. Adding a knowledge graph to a flawed retrieval pipeline gives you a flawed retrieval pipeline with higher operational overhead. Fix the pipeline first; as Atlan's guidance puts it: "RAG first, graph when depth becomes a bottleneck."

Trigger 1 — multi-hop reasoning failures → add the structural graph

This failure appears when an answer requires linking facts across multiple files that are never co-located in one text chunk. The symptom sounds like: your agent cannot answer "What breaks in the API layer if I alter the schema in db/migrations/04_users.sql?" It retrieves the migration file (keyword-rich, semantically close) but misses the foreign-key references, interface implementations, and cross-file calls that make up the actual answer.

Why can't better prompting fix it? Because vector RAG embeds text blocks independently. The relationship between the migration and the API handler was never captured in vector space — there is nothing for a better prompt or a bigger context window to recover. M02 called this the structural blind spot; the fix is the layer whose entire job is relationships: the AST graph you built in Track 2.

Trigger 2 — tribal-knowledge sprawl → add OKF

This failure appears when critical institutional context is fragmented across Slack channels, outdated wikis, and unwritten developer memory. The symptom is subtler and nastier: the agent generates syntactically valid code that violates internal architectural guidelines, calls deprecated internal APIs, or ignores domain-specific business logic. Nothing crashes. Reviews catch it — sometimes.

Why can't the structural graph fix it? Because AST parsers inspect source code, and business intent isn't in the source. No parse of webhooks.py reveals that the team decided (ADR-001) never to make billing settlement block on notifications. That knowledge lives in prose — which is exactly what OKF's concept files are for, in a shape agents reliably parse.

The Staged Query Pipeline

When a system has earned all three layers, how do they compose at query time? Not as alternatives the agent picks between at random — as stages, ordered from cheapest-and-most-curated to broadest-and-noisiest. Follow one real query through: "What breaks if I modify UserService.py?"

Walk the stages in prose, because the ordering logic is the lesson:

1. Stage 1 — narrative orientation (~2K tokens). The agent reads the OKF catalog's entry for the target: what UserService is, who owns it, which modules relate. This is progressive disclosure from M09 — a curated 2K-token orientation instead of a 50-file exploration. Output: the specific nodes worth investigating.
2. Stage 2 — structural traversal (zero LLM cost). With targets identified, the agent queries the graph: callers, implementers, transitive dependents. Deterministic answers from the M09 server; the LLM spends nothing acquiring them and everything reasoning about them.
3. Stage 3 — the unstructured remainder. Whatever the curated layers can't answer — "did we have an incident last time someone touched this?" — goes to vector/sparse search over Slack logs, GitHub issues, PR comments. RAG does what RAG is actually for: the long tail.

The Router Pattern

The staged pipeline suits deep investigations. But plenty of agent traffic is single questions, and those deserve a cheaper dispatch: a router that classifies each query and sends it straight to the layer that owns it. A canonical question — "What is the SLA for Severity-1 incidents?" — routes to a deterministic OKF read. A structural one — "who calls this?" — routes to the graph server. An exploratory one — "what are common themes in churn interviews?" — falls back to vector search.

Let's build the skeleton, in three chunks. Chunk 1 — classification. What: decide the query class. Why: the class determines cost and failure characteristics — a canonical query answered probabilistically is how the stale-WAU disaster of M02 happened. Gotcha: keyword rules look naive but are transparent and debuggable; teams often run rules first with an LLM-classifier fallback, not the reverse.

STRUCTURAL_CUES = ("who calls", "what calls", "depends on", "call graph",
                   "imports", "blast radius", "what breaks", "cycle")
CANONICAL_CUES  = ("definition of", "canonical", "sla", "policy",
                   "runbook", "how do we compute", "what is our")

def classify(query: str) -> str:
    q = query.lower()
    if any(cue in q for cue in STRUCTURAL_CUES):
        return "structural"
    if any(cue in q for cue in CANONICAL_CUES):
        return "canonical"
    return "exploratory"          # default: the broad, noisy layer

const STRUCTURAL_CUES = ["who calls", "what calls", "depends on", "call graph",
                         "imports", "blast radius", "what breaks", "cycle"];
const CANONICAL_CUES  = ["definition of", "canonical", "sla", "policy",
                         "runbook", "how do we compute", "what is our"];

function classify(query) {
  const q = query.toLowerCase();
  if (STRUCTURAL_CUES.some(c => q.includes(c))) return "structural";
  if (CANONICAL_CUES.some(c => q.includes(c)))  return "canonical";
  return "exploratory";          // default: the broad, noisy layer
}

Chunk 2 — dispatch with fallbacks. What: send each class to its layer, and fall through when a layer misses. Why: the M09 server returns a graceful miss ("not in graph — try grep") precisely so a router can catch it and degrade to search instead of failing. Gotcha: log every route decision — when the router misroutes (it will), the log is how you tune the cues.

import logging
log = logging.getLogger("router")

def route(query: str) -> dict:
    kind = classify(query)
    log.info("route=%s query=%r", kind, query)
    try:
        if kind == "structural":
            answer = graph_query(query)          # M09 MCP tools
            if "error" in answer:                # graceful miss -> degrade
                log.warning("graph miss, falling back to search")
                return {"layer": "vector", "answer": vector_search(query)}
            return {"layer": "graph", "answer": answer}
        if kind == "canonical":
            return {"layer": "okf", "answer": read_concept(query)}  # exact file
        return {"layer": "vector", "answer": vector_search(query)}
    except Exception as exc:
        log.error("layer failed: %s — degrading to vector search", exc)
        return {"layer": "vector", "answer": vector_search(query)}

async function route(query) {
  const kind = classify(query);
  console.error(`route=${kind} query=${JSON.stringify(query)}`);
  try {
    if (kind === "structural") {
      const answer = await graphQuery(query);      // M09 MCP tools
      if (answer.error) {                          // graceful miss -> degrade
        console.error("graph miss, falling back to search");
        return { layer: "vector", answer: await vectorSearch(query) };
      }
      return { layer: "graph", answer };
    }
    if (kind === "canonical")
      return { layer: "okf", answer: await readConcept(query) };  // exact file
    return { layer: "vector", answer: await vectorSearch(query) };
  } catch (exc) {
    console.error(`layer failed: ${exc} — degrading to vector search`);
    return { layer: "vector", answer: await vectorSearch(query) };
  }
}

Chunk 3 — what the leaf functions are. Nothing new: graph_query wraps the M09 MCP tools; read_concept is a python-frontmatter read of the matching bundle file (M06); vector_search is whatever retrieval you already had. The router adds no storage of its own — it is pure traffic control over layers you built in earlier modules.

The Evidence

Does the assembled stack actually help? Two data points frame the answer — one for the upside, one for the cost of not having it.

The ERPNext case study

The best production-shaped benchmark in the corpus ran on ERPNext, an open-source enterprise repository of over 1,000,000 lines of Python and JavaScript — real scale, not a 52-file demo. Evaluations used Claude Opus 4.8, capped at 14 turns per task:

• Baseline agent (vector RAG / grep exploration): 70.8% key-fact coverage.
• Graphify + OKF hybrid pipeline: 82.0% key-fact coverage, averaging ~140,000 tokens per query session.

Sit with what an 11.2-percentage-point improvement means on a million-line codebase: roughly one in nine facts the baseline agent missed or got wrong, the hybrid agent got right. Missed facts are wrong refactors, broken callers, violated conventions. That accuracy delta is a far more reliable indicator of production value than any synthetic token-reduction multiple — token savings tell you the agent was cheaper; fact coverage tells you it was right.

The cost of retrieval-only

On the other side of the ledger: research into legal-domain RAG systems — a field with high-stakes, multi-document questions much like large codebases — found standard retrieval tools hallucinate in 17% to 33% of complex queries, because retrieval alone fails to model multi-document logic. That is the gap the structural and narrative layers exist to close: not making search better, but removing whole question classes from search's jurisdiction.

The Decision Matrix

Locate your codebase's shape and your observed failure mode; the cell tells you what to add next. (No observed failure → the answer is always "nothing yet.")

The matrix encodes this module's whole argument: the environment sets the cost of each layer, the failure mode sets the value, and you adopt when value exceeds cost — never on hype.

Walk it, step by step

One question sent to each layer in turn, including the layers that answer it badly. The routing rule at the end is what the whole module is for.

MCP vs RAG vs OKF: One Metric, Three Roles

A final source of confusion to dissolve: MCP, RAG, and OKF get name-dropped together so often that they blur. They occupy different positions in the timeline of a question. OKF is knowledge written down in advance. MCP is the live connection to systems at question time. RAG is on-the-spot retrieval over what nobody curated.

orderflow's WAU metric shows all three in one story. The definition of weekly active users — 7-day window, completed orders, testers excluded — lives in an OKF concept file, written once, versioned in git (M06). When an agent must compute this week's WAU, the definition doesn't run queries: the agent calls a database tool over MCP to execute the SQL against the live warehouse. And when someone asks "have we ever debated changing the WAU window?" — that discussion lives in old Slack threads and PR comments, which is RAG's territory.

One more composition worth knowing: OKF bundles can themselves be RAG targets. Once a bundle grows past what an agent browses by hand, a search layer indexes it — and clean, structured Markdown with explicit links is far better material to index than PDFs chopped into arbitrary chunks. The honest picture isn't "OKF instead of RAG"; it's "RAG, pointed at something worth retrieving from."

Hands-On: Route 10 Queries

What you'll do: act as the router. For each orderflow query, pick the layer that should own it. Immediate feedback per pick — this exercise is embedded (no lab folder). Time: 15 min.

Knowledge Check

Module Summary

What we built on orderflow: a router dispatching canonical → bundle, structural → graph server, exploratory → search — with all failures degrading toward the honestly-probabilistic layer.

Next module preview: the stack works — on the day you built it. M11 is about every day after: silent hook failures, artifact desync, the scale wall, and the observability that separates a production system from a demo that hasn't failed yet.

References

• "Graphify, OKF, or Both? Beyond RAG for Codebases" — the three-layer framing and decision matrix (Towards AI, Jul 2026)
• Graphify BENCHMARKS.md — LongMemEval-S / LOCOMO head-to-head results
• Atlan — "Knowledge Graphs vs RAG" (2026): "RAG first, graph when depth becomes a bottleneck"
• Prism Labs — "Beyond Vector Embeddings in 2026" (chunk co-location)
• ERPNext hybrid-pipeline evaluation (1M+ LOC case study)
• Course modules: M02 (RAG limits), M06–M08 (OKF), M09 (serving)