M06 — The Open Knowledge Format Specification

Learning Objectives

• Explain what the Open Knowledge Format (OKF) standardizes — and, just as importantly, everything it deliberately does not include.
• Author a valid OKF concept file from scratch: YAML frontmatter with the one required field, a Markdown body, and cross-links.
• Describe the role of the two reserved filenames, index.md and log.md, and how progressive disclosure works as a token-budget control.
• State the conformance leniency rules and argue both sides of the trade-off they create.
• Read a bundle programmatically in about three lines of Python or Node.js.

Bridge: From Structure to Narrative

In Track 2 you built and queried structural graphs: tree-sitter parsed the orderflow code, Graphify clustered it into communities, and you could answer "who calls decode_jwt?" deterministically. That graph knows everything about what the code is. It knows nothing about what your team knows.

Consider a question the structural graph cannot answer: "What is the canonical definition of Weekly Active Users?" The AST can find the function weekly_active_users(), but it cannot tell you that this function is the attested source of truth, that the exclusion of internal testers was a deliberate policy decision from a 2026 data-governance review, or that the old definition in a stale doc is wrong. That knowledge lives in people's heads, in wikis, in Slack threads — scattered, unversioned, and invisible to agents.

This module introduces the format designed to fix exactly that: a way to write down curated, canonical knowledge so that both humans and agents can read it deterministically. Track 3 is about the narrative layer.

What OKF Is (and Isn't)

Three consequences follow from "format, not platform," and each one matters in practice. First, there is nothing to install. The spec's own FAQ puts it memorably: "If you can cat a file, you can read OKF; if you can git clone a repo, you can ship it." Second, the knowledge is Git-native: version-controlled, auditable through pull requests, diffable line by line. When your company's definition of "Revenue" changes, the change has an author, a timestamp, a review trail. Third, it is human-and-agent readableat the same time: the same file renders beautifully on GitHub for a person and parses in three lines for an agent. Compare that to a vector index, which is meaningful to exactly one embedding model and opaque to everyone else.

It's worth pausing on what OKF deliberately leaves out, because the omissions are the design. There is no built-in search engine — no indexing, no ranking, no query language. There is no schema registry — nobody validates that your type values are drawn from an approved list. There is no sync mechanism — the spec has no opinion on how bundles get created or stay current (that's Module 8's entire subject). Prior enterprise metadata standards demanded schema registries, validation servers, and vendor SDKs before you could write a single entry; OKF's bet is that minimalism is what makes a standard adoptable at all.

Lineage & Release: How a One-Page Spec Happened

On June 12–13, 2026, Google Cloud's Sam McVeety and Amir Hormati published OKF v0.1 through the GoogleCloudPlatform/knowledge-catalog repository. Their announcement framed it plainly: "a vendor-neutral, agent- and human-friendly standard for representing the metadata, context, and curated knowledge that modern AI systems need." The problem it targets is what the Google Cloud blog calls the context-assembly problem: table schemas, metric definitions, JOIN paths, and incident guides live scattered across catalogs, wikis, code comments, and experienced employees' memories — so every new agent rebuilds the same context from scratch.

Google is explicit about the intellectual lineage. In April 2026, Andrej Karpathy published the LLM WikiKarpathy's April 2026 thought experiment: instead of a model re-searching raw documents on every query, let it build and maintain a living, cross-linked Markdown wiki as durable external memory. idea: an LLM should maintain a compiled, cross-linked Markdown wiki as durable external memory — the way source code is compiled once and reused rather than re-parsed on every read. His framing for why this works stuck with a lot of engineers, and Google's materials quote the spirit of it directly: "LLMs don't get bored, don't forget to update a cross-reference, and can touch 15 files in one pass. The bookkeeping that causes humans to abandon personal wikis is exactly what LLMs are good at."

OKF didn't invent a substrate, either. The AGENTS.md convention had already spread across tens of thousands of open-source projects; developers were already wiring Obsidian-style vaults straight into coding agents. As one analysis put it: OKF standardizes the substrate that already won — Markdown, frontmatter, and Git. The full v0.1 conformance spec fits on a single page: three conformance rules, total. That minimalism is not a gap to be filled later. It is the entire design philosophy — "the spec defines the interoperability surface, not the content model."

And it shipped with proof it scales past a toy: Google published reference sample bundles (a GA4 e-commerce bundle, Stack Overflow, Bitcoin) plus a BigQuery metadata-extraction agent. The GA4 demo bundle is 17 Markdown files — indexes, references, datasets, tables — demonstrating that this restraint holds up on something real.

Bundle Anatomy: Directories, Concepts, and Paths as Identity

Path-as-identity sounds like a small decision, but it kills an entire class of maintenance bugs. Every system that assigns synthetic IDs (UUIDs in a catalog, page IDs in a wiki) eventually faces drift between the ID registry and reality: the page moved, the record was cloned, the pointer dangles. When the path is the ID, moving a file is renaming a concept — visible in Git, reviewable in a PR, greppable everywhere it's referenced. One thing to update, one place to see it.

Here is what a real bundle looks like — an internal "company brain" with knowledge compiled into hyper-focused, singular concepts:

Notice the granularity. There is no misc_notes.md dumping ground. Information is compiled into hyper-focused, singular concepts: one file per API contract, one per financial metric, one per database table. That granularity is what makes progressive disclosure (next section) work — an agent can load exactly the concept a task needs, and nothing else.

Frontmatter: One Required Field

Every concept file follows a strict but minimal design: a YAML frontmatter block on top, a free-form Markdown body underneath. Of the recommended frontmatter fields, exactly one is required: type. Everything else — title, description, resource, tags, timestamp — is optional. As the spec's authors put it: "OKF requires exactly one thing of every concept: a type field."

Watch a canonical concept file come apart into its pieces:

Why does the frontmatter matter operationally? Because it gives the LLM structured, high-level context before it reads the body. An agent scanning a directory can filter on type: metric or tags: [billing] using explicit metadata rather than fuzzy semantics — a cheap, deterministic pre-filter that costs a few dozen tokens per file instead of the whole file.

Here's a second canonical example — the one this course keeps returning to. This is a real business-metric concept, the kind whose absence caused M02's churn disaster:

---
type: metric
id: analytics/metrics/active_users
title: Weekly Active Users (WAU)
owner: data-eng@company.com
updated_at: 2026-06-15
citations:
  - source: "https://github.com/internal-org/dbt/models/wau.sql"
---

# Weekly Active Users (WAU)

The total unique count of user IDs who have triggered at least one core
backend API transaction within a rolling 7-day window.

## Computation Rules
We explicitly exclude internal QA and test accounts:
`WHERE user_id NOT IN (SELECT user_id FROM staging.internal_testers)`

## Related Components
- See [[analytics/tables/customers]] for primary user dimension mappings.
- See [[analytics/tables/billing]] to correlate usage with subscription cycles.

Three details deserve attention. First, the frontmatter includes fields beyond the recommended set (owner, citations) — perfectly legal, because consumers must tolerate unknown keys (next section). Second, the body carries the actual, audited SQL exclusion rule — not a description of where to find it. When an agent needs to compute WAU, it extracts the absolute logic and follows the explicit link to the customers table concept for current join keys. It generates a correct query on the first attempt, citing the exact file, its update time, and its owner. Third — and this is a spec subtlety worth teaching precisely — the id field here is redundant with the path by design: the path is the identity; the field is a convenience mirror some producers include.

The Reserved Filenames: index.md and log.md

Two filenames have reserved meanings in every bundle, and they give the format its structure without adding a single required field.

index.md — the progressive disclosure lever

This is a genuine token-budget control, not a convenience file. An orchestrating agent reads index.md, decides which concept files a given subtask actually needs, and loads those — nobody pulls a thousand-file bundle into context for a change that touches one service. Watch the difference:

One production tip carries more weight than any other here: keep index entries to a title and a one-line description pulled straight from each concept's frontmatter. An index bloated with detail defeats the point of having one — it becomes just another big file the agent must wade through.

log.md — what the bundle knows, over time

log.md is a chronological, date-grouped record of changes to what the bundle knows — deliberately distinct from git log, which records what changed in the files. These are different questions. A service's code can change without its documented responsibilities changing (a refactor), and its documented responsibilities can change without a single code line moving (a policy decision). "What changed in this file" and "what changed in our understanding of this system" need separate answers, and log.md lets an agent — or a person — answer the second one directly instead of inferring it from commit archaeology.

Conformance Leniency: The Trade-Off at the Heart of the Spec

The v0.1 conformance rules are deliberately, radically lenient. A conformant consumer must not reject a bundle for:

• missing optional fields,
• unrecognized type values (there is no type registry — type is producer-defined),
• unknown frontmatter keys,
• or even broken links.

Why design it this way? Because strictness kills adoption. Every failed validation is a reason for a team to give up on the format; every required registry is infrastructure someone must stand up before writing entry one. Leniency means a team can start with a single file, today, and be conformant. That is why OKF spread in weeks while stricter predecessors — the schema-registry, validation-server kind — spent years stalled.

But the same leniency creates the risk this course spends Module 11 on: silent degradation. A bundle full of dangling links, drifted type names ("API Endpoint" here, "Endpoint" there, "Route" somewhere else), and stale timestamps is still technically conformant. Nothing errors. Nothing warns. The spec will never save you from a rotting bundle — only your own lintAn automated checker that enforces rules stricter than the format requires — e.g., "every concept must have a description" or "no dangling links." The okf CLI ships 13 built-in lint rules (M07). rules and review discipline can. Remember the asymmetry: the spec is lenient so that adoption is easy; your pipeline must be strict so that trust is possible.

Cross-Links: A Folder Becomes a Knowledge Graph

The links you saw in the WAU concept — [[analytics/tables/customers]], or standard relative Markdown links like [customers](/tables/customers.md) — are where the "graph" in this course's title comes back. Each link turns two files into two nodes with an edge between them. The result: a plain folder of Markdown becomes a deterministic knowledge graph an agent can step through logically, with no cosine-similarity guessing about what relates to what.

Contrast the retrieval story from M02 one more time, because this is the payoff of the whole design. Old way: the agent embeds your question, searches thousands of chunks, and gets back three fragments of conflicting vintage. OKF way: the agent reads the root index.md, traverses to analytics/metrics/churn_rate.md, extracts the audited logic, and follows the explicit link to the current schema definition. Deterministic navigation across a living graph, instead of probabilistic search over shredded files.

Be equally clear about the ceiling. OKF links are untyped: a link is just a link, with the relationship's meaning carried by surrounding prose ("Calls…", "Joined with…", "See also…"). Consumers must tolerate broken ones. That caps how much formal graph reasoning — automated dependency traversal, impact analysis with guarantees — an agent can safely do compared to a real RDF/OWLThe W3C semantic-web standards for formal knowledge graphs: typed relationships, machine-checkable ontologies, logical inference. Far more rigorous than Markdown links — and far heavier to author and maintain. knowledge graph. The structural layer you built in Track 2 (typed, provenance-tagged AST edges) and the narrative layer you're building now are complements, not substitutes: Graphify's edges are verified; OKF's links are curated.

Walk it, step by step

Author a concept file and get it rejected. The lint failure is the lesson: it tells you exactly which part of the spec you skipped, which is faster than reading the spec twice.

Code Walkthrough: Reading a Bundle Programmatically

The entire consumption story fits in a few lines — that's the point of standardizing on boring substrates. Let's read the WAU concept, then write the validation check every pipeline needs. Chunk 1 — load and inspect. WHAT: parse one concept into metadata + body. WHY: this is the exact operation an agent's tool layer performs before deciding whether the concept is relevant. GOTCHA: a file with no frontmatter loads fine with empty metadata — reading never throws just because metadata is missing, which is why validation must be explicit.

import frontmatter  # pip install python-frontmatter

concept = frontmatter.load("analytics/metrics/weekly_active_users.md")
print(concept["type"])        # "metric"
print(concept["title"])       # "Weekly Active Users (WAU)"
print(concept.content[:200])  # the markdown body: rules, links, everything

import fs from "node:fs";
import matter from "gray-matter";  // npm install gray-matter

const raw = fs.readFileSync("analytics/metrics/weekly_active_users.md", "utf8");
const concept = matter(raw);
console.log(concept.data.type);        // "metric"
console.log(concept.data.title);       // "Weekly Active Users (WAU)"
console.log(concept.content.slice(0, 200)); // the markdown body

Chunk 2 — validate. WHAT: enforce the one hard rule (type present) across a bundle, in CI. WHY: leniency means nothing else will ever catch a malformed concept; this tiny gate is the seed of the lint discipline M07 and M11 grow into. GOTCHA: catch parse errors too — a mangled YAML fence is worse than a missing field, because it can silently swallow the whole header.

import sys
from pathlib import Path
import frontmatter

def validate_bundle(root: Path) -> int:
    errors = 0
    for path in sorted(root.rglob("*.md")):
        if path.name in {"index.md", "log.md"}:   # reserved files: no type needed
            continue
        try:
            post = frontmatter.load(path)
        except Exception as exc:       # mangled YAML is an error, not a shrug
            print(f"ERROR {path}: unparseable frontmatter ({exc})")
            errors += 1
            continue
        if not post.metadata.get("type"):
            print(f"ERROR {path}: missing required field 'type'")
            errors += 1
    print(f"{errors} error(s)")
    return 1 if errors else 0

if __name__ == "__main__":
    raise SystemExit(validate_bundle(Path(sys.argv[1])))

import fs from "node:fs";
import path from "node:path";
import matter from "gray-matter";

function walk(dir) {
  return fs.readdirSync(dir, { withFileTypes: true }).flatMap((e) =>
    e.isDirectory() ? walk(path.join(dir, e.name))
      : e.name.endsWith(".md") ? [path.join(dir, e.name)] : []);
}

let errors = 0;
for (const file of walk(process.argv[2])) {
  const base = path.basename(file);
  if (base === "index.md" || base === "log.md") continue;
  try {
    const { data } = matter(fs.readFileSync(file, "utf8"));
    if (!data.type) { console.error(`ERROR ${file}: missing required field 'type'`); errors++; }
  } catch (exc) {
    console.error(`ERROR ${file}: unparseable frontmatter (${exc.message})`); errors++;
  }
}
console.log(`${errors} error(s)`);
process.exit(errors ? 1 : 0);

Hands-On Exercise: Author Your First Concepts

📂 Get the files: labs/M06-okf-authoring on GitHub — or clone the whole course once: git clone https://github.com/varasrinivas/knowledge-graph-course.git

What you'll build: two valid OKF concepts for orderflow — the WAU metric and the orders_fact table — plus the frontmatter validator, in labs/M06-okf-authoring/. Time: 25–35 min. Files: two .md concepts and validate.py.

1. Step 1 — Study the shape. Open starter/knowledge/analytics/metrics/weekly_active_users.md. The frontmatter is stubbed; only type is required, but your lint will warn on missing recommended fields.
   ✅ Checkpoint

   You can name all five recommended optional fields without looking. (title, description, resource, tags, timestamp.)

2. Step 2 — Complete the two concepts. WAU: 7-day rolling window over completed orders, explicit tester exclusions, resource: pointing at services/orders/metrics.py, cross-link to [orders_fact](../tables/orders_fact.md). orders_fact: schema table, Joins section, link back to the metric.
3. Step 3 — Write the validator. Complete starter/validate.py per the walkthrough above: hard-fail on missing type, warn on missing recommended fields.
4. Step 4 — Verify. Run:
   terminalCopy

   cd labs/M06-okf-authoring/starter
   python validate.py knowledge

   Expected output: 2 concepts valid, 0 errors.
   ✅ Checkpoint

   Now break it on purpose: delete type: from one file and re-run. You must get a non-zero exit and the offending path. If the validator still passes, you're reproducing the spec's leniency instead of guarding against it.

Troubleshooting: a YAML ScannerError usually means your --- fences aren't alone on their lines; a validator that passes empty files means you forgot that frontmatter.load returns empty metadata rather than raising. Stretch goal: add a dangling-link check — every relative .md link must resolve inside the bundle.

Knowledge Check

Module Summary

What we built: orderflow's first two concepts — the WAU metric with its attested exclusion rule, and the orders_fact table — plus the validator that will grow into M07's lint gate.

Next module preview: Google's reference use case is BigQuery tables. M07 makes the pivot that matters for this course: the same concept shape, aimed at code — services, dependencies, responsibilities — and the okf CLI with its 13 lint rules.

References

• Google Cloud: Introducing the Open Knowledge Format — cloud.google.com/blog/products/data-analytics (June 2026)
• OKF Specification v0.1 — github.com/GoogleCloudPlatform/knowledge-catalog (okf/SPEC.md)
• OKF FAQ — okf.md/faq
• Karpathy, A. — LLM Wiki / knowledge-compiler pattern (April 2026)
• Course corpus: "Beyond RAG: How Google's OKF is Replacing the Vector Database"; "Did Google Just Kill RAG?"; "Standardizing Agent Memory"; "Your Vector DB is Shredding Context"