M04 · Graphify in Practice

Learning objectives Intermediate

In M03 you built a toy extractor. This module hands you the production version of the same idea — then teaches you to evaluate its claims the way an engineer should. Both halves matter: Graphify is genuinely useful, and the numbers on its posters need footnotes. By the end you'll know how to run it and exactly how much to believe.

What Graphify is

Every extracted relationship carries the provenance tag you met in M03 — EXTRACTED (parsed directly from the AST, 100% confidence) or INFERRED (derived via heuristic, lower confidence, with a confidence score). One reviewer who ran it end-to-end on a real 36-file infrastructure repository called this "the whole argument: it is the only one of these tools that tells you which of its answers are facts and which are guesses."

Install and first run

Three install routes, all landing in the same place. The uv route is recommended — it works on Mac and Linux with no PATH setup, and on Windows with a current uv:

# Recommended
uv tool install graphifyy && graphify install

# or with pipx
pipx install graphifyy && graphify install

# or plain pip
pip install graphifyy && graphify install

graphify install is the step people forget: it registers the /graphify skill with the assistants on your machine, so Claude Code (or Cursor, Codex, Gemini CLI…) can invoke the graph natively. Then, from your project root:

cd your-project
graphify update .
# or, inside your AI assistant:
/graphify .

A successful run logs like this — read it closely, because three of these lines encode design decisions you now understand:

Re-extracting code files in . (no LLM needed)...
[graphify watch] Rebuilt: 70 nodes, 89 edges, 15 communities
[graphify watch] graph.json, graph.html and GRAPH_REPORT.md updated in graphify-out
Code graph updated. For doc/paper/image changes run /graphify --update in your AI assistant.
Tip: set MOONSHOT_API_KEY to use Kimi K2.6 for semantic extraction — 3x cheaper, richer graphs.

• "no LLM needed" — the code pass is the deterministic, free, zero-hallucination pass from M03.
• "For doc/paper/image changes run /graphify --update" — the semantic pass is separate, optional, and the only part that costs tokens (and the only part that can hallucinate).
• "15 communities" — clustering ran automatically; we'll read those below, and the count is higher than the three services you might expect.

Two housekeeping facts: a .graphifyignore file excludes junk directories (it politely ignores .git and node_modules by default), and on very large codebases graph.html is skipped — the log tells you when. The cache/ directory means re-runs only touch changed files.

The three artifacts

Every run writes three files into graphify-out/. They are three views of the same graph for three different consumers — and the fact that there are three separate files becomes a production concern in M11 (they can desync; remember this).

Querying the graph

The CLI has two workhorse queries. path answers "how are these two things connected?" and query takes a natural-language question against the graph index:

# Dependency path between two components
graphify path "UserService" "DatabasePool"

# Architectural blast radius before a refactor
graphify query "What components depend on AuthTokenValidator?"

Inside an assistant, /graphify . plus a question does the same thing conversationally — the assistant fetches relevant nodes and edges and reasons over those, not over raw files. And for always-on integration, the graph can be served over MCPModel Context Protocol — the open standard that lets AI assistants call external tools. M09 builds a graph-serving MCP server from scratch. so the assistant fetches nodes on the fly:

python -m graphify.serve graphify-out/graph.json

Which of the three routes to use is a serving-architecture question — the entire subject of M09. For now: CLI for you, slash-command for conversations, MCP for agents that need to query mid-task.

Walk it, step by step

Below is the whole pipeline on the orderflow sample project — install, extract, inspect the artifacts, then ask it a question. Watch the node and edge counts appear, and note how few of the edges are EXTRACTED rather than INFERRED.

Communities and god nodes: the graph reads your architecture

Recall from M01: a community is a cluster of nodes more densely connected to each other than to the rest of the graph, and in code graphs communities ≈ natural subsystems. Graphify runs Leiden community detectionA graph-clustering algorithm (successor to Louvain) that partitions nodes to maximize modularity — the degree to which clusters are internally dense and externally sparse. M01 introduced it. automatically on every build. On orderflow it does not hand you three tidy services: Graphify reports 15 communities named after files (app.py, db.py, worker.py, webhooks.py, orders.py, invoice.py) plus one per document. The billing / orders / notifications split the architecture doc claims is recoverable from those clusters — but you read it out, you are not handed it. At 18 files there is too little call density for the clustering to coarsen further.

It also flags god nodes — the high-centrality files that many communities depend on. On orderflow that is shared/db.py — but for a narrower reason than it first appears. Billing and orders depend on it (notifications never persists, so it is not universal), and its claim to the title is blast radius: seven functions transitively depend on it, more than any other shared module. It does not win on betweenness — it scores 0.00, because nothing routes through a sink that calls nothing back. A god node isn't automatically bad, but it is automatically risky: any change to it has the widest blast radius in the repo, and (M01 foreshadowed this) computing centrality exactly is O(V×E) work — which becomes M11's scale-wall story on 450K-node graphs.

The honest benchmarks

Now the part most write-ups skip. Graphify's marketing number is famous: up to 71.5× reduction in query tokens. That number is real — and almost useless as a planning input. Here is the full picture, every number sourced, vendor claims labeled as such.

Where 71.5× came from

The headline (vendor) figure comes from a single favorable benchmark: a 52-file corpus — Karpathy's public repositories, 5 research papers, and 4 diagrams — where a naive assistant burned roughly 123,000 tokens answering an architecture question that the graph answered in about 1,700 tokens. That's a ceiling case: a favorable query, on a favorable corpus, measured once.

Independent replications tell the useful story. Across task types and repo sizes, the honest range came out to 6.8× on code-review tasks up to 49× on daily coding in 500+ file repositories. A from-scratch benchmark on a real, non-cherry-picked Python codebase landed at 7.3×. Still genuinely good — a different conversation than "70×." As engineering writer Andrus put it during the July 2026 scrutiny wave: "Whether you get anything close to the 70x savings comes down to a single property of your project that almost none of these posts bother to state out loud." That property is corpus topology — flat-package enterprise codebases and 50,000-file monorepos do not behave like a curated 52-file demo set.

The benchmark that contradicts the hype — and reframes the tool

Open Graphify's own BENCHMARKS.md and you find numbers the launch posts skip. On LongMemEval-S (n=50), graph-expand retrieval scores 76% QA accuracy with 0.844 recall@10. Dense vector RAG — same model (Kimi K2.6), same BGE-m3 embedder, identical token budgets — scores 76% accuracy with 0.848 recall@10. A tie on accuracy; a slight loss on recall. And this is Graphify's own vendor-reported, blind-validated dataset (90.6% grading agreement, Cohen's kappa 0.81), not a hostile third party.

On LOCOMO (n=300) the interesting metric shifts to cost: Graphify scores 45.3% accuracy at roughly $1.40 to ingest the dataset; Supermemory scores higher — 49.7% — but costs $15.67 to ingest, roughly 11× more.

Put together, the honest reframe: Graphify does not out-retrieve a well-tuned vector RAG pipeline. It matches RAG on accuracy while costing a fraction to build, and its two real advantages are exactly the M03 story — ingest cost efficiency (AST parsing needs zero LLM calls) and zero-hallucination structural extraction. If you evaluated it as a RAG replacement, you evaluated the wrong axis. One more caveat belongs in your notes: no external benchmark comparing Graphify's graph queries against a raw-file or RAG baseline on a common test set has been independently reproduced — treat all vendor figures as directional.

Limitations

• Noise on messy code: point it at a 5,000-line GodController and the graph looks like a ball of yarn after a cat attack. The graph reflects the code; it cannot untangle what the code tangled.
• Dynamic-language guessing: in JavaScript and friends, where "types are merely suggestions," edges lean on INFERRED heuristics (M03's ladder). The tags are your audit trail.
• Large-graph latency: at 500,000 nodes, querying can take a second — and rebuild-time centrality math gets far worse than that (the full scale-wall numbers, including a 96-core rebuild killed at 114 minutes, are in M11).
• Static snapshot: the graph does not update itself as developers commit. Without hooks wired in — and working — it silently drifts. This single sentence is the seed of the entire production track; M11 grows it into a checklist.
• Semantic pass hallucination: relationships extracted from docs, PDFs, and images go through an LLM and inherit its error rate. Only the code pass is deterministic.

Code walkthrough: reading graph.json programmatically

The lab's extractor writes the same shape Graphify does: a nodes array and an edges array. Let's load it and compute the top-degree nodes — a two-minute god-node detector. Tabs for both languages.

import json, sys
from collections import Counter
from pathlib import Path

path = Path("graphify-out/graph.json")
try:
    graph = json.loads(path.read_text(encoding="utf-8"))
except FileNotFoundError:
    sys.exit(f"{path} not found — run 'graphify update .' (or kg_extract.py) first")
except json.JSONDecodeError as exc:
    sys.exit(f"{path} is corrupt: {exc} — rebuild the graph")

degree = Counter()
for edge in graph["edges"]:
    degree[edge["source"]] += 1
    degree[edge["target"]] += 1

print("top nodes by degree:")
for node_id, deg in degree.most_common(5):
    print(f"  {deg:3d}  {node_id}")

# On orderflow: handle_payment_webhook (6), then the shared.auth
# pair (4 each). Degree finds busy nodes; it is not the whole
# story — rank by blast radius and shared/db.py wins instead,
# and by betweenness (not computed here) it scores zero.

import { readFileSync } from "node:fs";

const path = "graphify-out/graph.json";
let graph;
try {
  graph = JSON.parse(readFileSync(path, "utf8"));
} catch (err) {
  const hint = err.code === "ENOENT"
    ? "run 'graphify update .' (or kg_extract.py) first"
    : `corrupt JSON: ${err.message} — rebuild the graph`;
  console.error(`${path}: ${hint}`);
  process.exit(1);
}

const degree = new Map();
const bump = (id) => degree.set(id, (degree.get(id) ?? 0) + 1);
for (const edge of graph.edges) { bump(edge.source); bump(edge.target); }

const top = [...degree.entries()].sort((a, b) => b[1] - a[1]).slice(0, 5);
console.log("top nodes by degree:");
for (const [id, deg] of top) console.log(`  ${String(deg).padStart(3)}  ${id}`);

Hands-on lab

📂 Get the files: labs/M04-graphify-end-to-end on GitHub — or clone the whole course once: git clone https://github.com/varasrinivas/knowledge-graph-course.git

Run labs/M04-graphify-end-to-end/ now. The four steps:

1. Build — graphify update . on orderflow; match the expected log shape (nodes/edges/communities, "no LLM needed").
2. Read all three artifacts — find the service clusters among the file communities in graph.html, and read the God Nodes list in the report; check GRAPH_REPORT.md names the same ones; peek at graph.json.
3. Query — graphify path "post_invoice" "decode_jwt" and a dependency query on DatabasePool; verify both against the ground-truth edge list in sample-project/README.md. On a 15-file repo you can check the graph by hand — that's the whole point of the repo.
4. The honest measurement — ask your assistant a structural question with and without the graph, record real token counts, and keep the number: expect a small multiple, not 71.5×, because orderflow is far below the ~500-file floor. You will reuse this measurement in the capstone.

No install available? Every step has an offline path via labs/shared_tools/kg_extract.py.

Knowledge check

Module summary

What we built on orderflow: a complete graph build with all three artifacts, hand-verified against the repo's ground-truth edge list, plus a six-line god-node detector over graph.json.

Next module preview: Graphify is one point in a fast-moving design space. M05 maps the rest — CodeGraph's SQLite-and-watchers architecture (with the best benchmark table in the field), okf-rs's Markdown-bundle bet, and the cloud tools — and gives you the comparison axes to evaluate any tool that launches next month.

References

• Graphify — GitHub repository, BENCHMARKS.md, and docs (graphify.net)
• PyPI: graphifyy · install docs for uv/pipx/pip · /graphify skill registration
• Level Up Coding (Andrus) — scrutiny of the 70× token-reduction claims; corpus-topology dependence
• LongMemEval-S and LOCOMO benchmark figures — Graphify vendor-reported, blind-validated
• Course lab: labs/M04-graphify-end-to-end/ · offline fallback: labs/shared_tools/kg_extract.py