Basic RAG (from M09) retrieves the top-k chunks by cosine similarity and hands them to Claude. This works well for simple queries but hits a ceiling fast. Advanced RAG wraps that pipeline with three optimization layers: pre-retrieval (transform the query first), retrieval enhancement (combine keyword + semantic search, then re-rank), and post-retrieval processing (compress chunks, remove noise). Each layer targets a different failure mode.
Why the ceiling? Naive RAG fails because: (1) vector similarity ≠ relevance — the most similar chunk isn't always the most useful; (2) one vague query might miss documents that use different phrasing; (3) 500-token chunks contain mostly irrelevant context that dilutes the generation.
💡 Why It Matters
Benchmarks show naive RAG achieves 55–65% answer accuracy. Adding hybrid search lifts that to 70–75%. Adding re-ranking pushes to 80–85%. Query transformation on top gets to 85–90%. Each pattern is modular — add only what your use case needs.
Concept 1 of 6
Analogy
Google vs. Research Librarian
💡 Everyday Analogy
BEFORE: Naive RAG is like searching Google by typing your exact question and reading only the first result — sometimes it's right, but often it's tangential, uses different wording, or misses the nuance of what you really needed.
PAIN: You waste time reading irrelevant pages, miss the one article that actually answers your question because it used different wording, and have no way to know if the answer you got is trustworthy.
MAPPING: Advanced RAG is like having a research librarian who first rephrases your question three different ways, searches both a keyword index and a topic index simultaneously, carefully reads the top candidates to rank them by actual relevance, highlights only the sentences that matter, and then gives you a sourced answer. Every stage the librarian adds makes the final answer more accurate.
Concept 1 of 6
How It Works
Three Optimization Layers
1Pre-retrieval (Query Transformation): Rephrase or expand the user's query before searching. A vague question becomes 3 specific ones; a short question becomes a hypothetical document.
2Retrieval Enhancement — Hybrid Search: Run BM25 keyword search AND vector similarity search in parallel. Fuse results with Reciprocal Rank Fusion (RRF).
3Retrieval Enhancement — Re-Ranking: Take the top-50 fused candidates. Run a cross-encoder (reads query + doc together) to score true relevance. Keep only top-5.
4Post-Retrieval — Compression: Extract only query-relevant sentences from each chunk. Discard background noise. This can reduce context by 80–90%.
5Generate: Compressed, precisely-ranked chunks go to Claude. Result: higher accuracy, fewer tokens, lower hallucination.
Concept 1 of 6
Decision Framework
Which Advanced Pattern Do You Need?
IF queries contain IDs, codes, proper nouns:
→ Add HYBRID_SEARCH (BM25 catches exact terms)
IF correct document is retrieved but ranked low:
→ Add RERANKING (cross-encoder improves order)
IF user questions are vague or multi-part:
→ Add QUERY_TRANSFORMATION (HyDE / Multi-Query)
IF chunks contain too much irrelevant context:
→ Add CONTEXTUAL_COMPRESSIONIF questions span multiple corpora or need multi-hop:
→ Use AGENTIC_RAG (search as a tool)
# Start with hybrid search — biggest ROI# Add others based on measured eval gaps# Never add all patterns blindly — measure first
Concept 1 of 6
Misconceptions + Takeaway
Advanced RAG Misunderstandings
❌ "More stages always means better quality"
✅ Each stage adds latency and cost. After hybrid search + re-ranking, additional stages often yield 2–3% accuracy gains while tripling latency. Measure your eval set before adding complexity.
❌ "Advanced RAG is one thing you deploy"
✅ It's a menu of independent patterns. You can add hybrid search without re-ranking, or re-ranking without query transformation. Pick the layers that address your actual failure modes.
💡 Key Takeaway
Advanced RAG wraps basic retrieve-then-generate with modular optimization layers. Start with hybrid search (biggest ROI for most use cases), measure with eval metrics, then add re-ranking or query transformation based on what the metrics say is failing.
Concept 2 of 6
Big Idea
Hybrid Search
Vector search has a blind spot: exact terms. If a user asks about "invoice #INV-2024-0847," vector search might return chunks about invoices in general rather than that specific one. BM25 keyword search finds exact matches instantly but misses synonyms and semantic relationships. Hybrid search runs both in parallel and fuses the results — covering what either alone would miss.
BM25 (keyword)
Matches exact terms, IDs, proper nouns. Great for "invoice #INV-2024-0847". Poor for "tell me about payment issues."
Vector (semantic)
Matches meaning, catches synonyms. Great for "payment issues." Poor for exact document IDs it never "saw" as words.
💡 Why It Matters
In real-world benchmarks (BEIR, MTEB), hybrid search improves nDCG@10 by 8–15% over vector-only search, with the biggest gains on queries containing proper nouns, IDs, or technical terms.
Concept 2 of 6
Analogy
Name Search + Description Search
💡 Everyday Analogy
BEFORE: You're searching for a restaurant you know as "Trattoria Verde" — but you also remember it as "that cozy Italian place near Central Park."
PAIN: A name-only search finds the exact restaurant but misses similar ones you might like. A description-only search finds cozy Italian places but might miss Trattoria Verde if its listing doesn't say "cozy."
MAPPING: Hybrid search runs both queries and merges the results. BM25 is the name search — fast, literal, great for proper nouns. Vector search is the description search — understands meaning, catches synonyms. Together, they cover cases that either alone would miss. Reciprocal Rank Fusion (RRF) merges the two lists without needing to normalize incompatible score formats.
Concept 2 of 6
How It Works
BM25 + Vector + RRF Fusion
1BM25 search: Tokenize query into words. Score each document by term frequency × inverse document frequency. High score for rare matching terms (e.g., exact invoice IDs).
2Vector search: Embed query into a high-dimensional vector. Find top-k documents by cosine similarity. Catches synonyms and conceptual matches.
3RRF merge: For each document, compute score = Σ 1/(k + rank) across all result lists (k=60 typically). Sum and rank by fused score. No score normalization needed — pure rank-based.
4Unified result: Documents that appear high in both lists get boosted. Documents missed by one but found by the other still make it in. Best of both worlds.
Concept 2 of 6
Pseudocode
Hybrid Search + RRF
FUNCTION hybrid_search(query, k=5):
# Run both in parallel
bm25_results = BM25_SEARCH(query, top_n=20)
vector_results = VECTOR_SEARCH(EMBED(query), top_n=20)
# Reciprocal Rank Fusion
scores = {}
FOR rank, doc INENUMERATE(bm25_results):
scores[doc] += 1 / (60 + rank) # k=60FOR rank, doc INENUMERATE(vector_results):
scores[doc] += 1 / (60 + rank)
# Sort by fused score, return top-k
fused = SORT_BY_VALUE_DESC(scores)
RETURN fused[:k]
Concept 2 of 6
Misconceptions + Takeaway
Hybrid Search Misunderstandings
❌ "Hybrid search is always better than pure vector search"
✅ For abstract, conceptual queries ("explain debtor risk factors"), vector-only can outperform hybrid because BM25 adds noise by matching irrelevant keyword hits. Test with your actual queries before committing.
❌ "RRF requires score normalization between BM25 and vector scores"
✅ RRF uses rank positions, not raw scores. Score(doc) = Σ 1/(k + rank). You never need to normalize BM25's 12.4 against cosine's 0.87 — ranks are always comparable.
💡 Key Takeaway
Hybrid search = BM25 (exact terms) + vector (meaning) + RRF (rank-based merge). It's the highest ROI advanced RAG pattern — implement it first. Biggest gains appear on queries with IDs, codes, proper nouns, or technical terms that vector embeddings treat as opaque tokens.
Concept 3 of 6
Big Idea
Re-Ranking
Hybrid search gets better documents into your candidate set. But cosine similarity and BM25 scores are rough proxies for "actually answers this question." Re-ranking applies a more powerful cross-encoder model that reads the query and each document together, jointly, and scores their true relevance. The most important document that was buried at position #4 gets promoted to #1.
The two-stage pattern: Retrieve broadly with fast methods (top-50 candidates). Re-rank that short list precisely with a cross-encoder (top-5 final). Never apply the expensive step to all documents — it's the database B-tree index analogy: fast index scan first, expensive filter on the short list.
💡 Why It Matters
MS MARCO benchmarks show re-ranking improves precision@5 by 15–25%. In healthcare RAG processing 200 prior-auth claims/day, moving the correct clinical guideline from position #8 to #2 means it's in the context window instead of being dropped — the difference between correct and incorrect coverage decisions.
Concept 3 of 6
Analogy
The Careful Recruiter
💡 Everyday Analogy
BEFORE: A job recruiter searches LinkedIn for "senior backend engineer, Rust experience, distributed systems." The search returns 200 profiles ranked by keyword match.
PAIN: Profile #3 mentions "Rust" once in a side project, while profile #47 has 5 years of Rust at a distributed-systems company but listed it as "systems programming." The initial keyword ranking is completely misleading.
MAPPING: Re-ranking is the recruiter reading the top-50 profiles carefully and reordering them by actual fit — not just keyword count. It's slower (reading takes time), but the final shortlist is dramatically better. In RAG, the "recruiter" is a cross-encoder that reads both query and document together and assigns a true relevance score.
Concept 3 of 6
How It Works
Two-Stage Retrieval + Re-Ranking
1Broad retrieval: Use hybrid search to get top-50 candidate documents. Fast, approximate — gets the right docs into consideration.
2Cross-encoder scoring: Feed each (query, document) pair to a cross-encoder. It reads both together and outputs a relevance score 1–10. Slower but far more accurate.
3Re-sort: Order the 50 candidates by cross-encoder scores. The document that was #4 by cosine similarity may jump to #1 by true relevance.
4Keep top-5: Only the top-5 re-ranked documents go into Claude's context window. This cuts irrelevant noise before generation.
Concept 3 of 6
Pseudocode
Two-Stage Re-Ranking Pipeline
FUNCTION retrieve_and_rerank(query, final_k=5):
# Stage 1: broad retrieval (fast)
candidates = HYBRID_SEARCH(query, k=50)
# Stage 2: re-rank with cross-encoder (slow but precise)
scored = []
FOR doc IN candidates:
relevance = CROSS_ENCODER_SCORE(query, doc)
# Or: relevance = CLAUDE_SCORE(query, doc, scale=10)
scored.APPEND((relevance, doc))
# Sort by true relevance, return best few
scored.SORT(by=relevance, descending=True)
RETURN [doc FOR _, doc IN scored[:final_k]]
Cost note: Re-ranking 20 chunks with Claude costs ~$0.01–0.015 per query. For 10K queries/day: ~$100–150/day. Use dedicated models (Cohere Rerank, BGE-reranker) for high-volume systems.
Concept 3 of 6
Misconceptions + Takeaway
Re-Ranking Misunderstandings
❌ "Re-ranking is free improvement you should always add"
✅ Re-ranking adds 100–300ms of latency and one extra LLM call per query. For simple factual lookups where initial retrieval is already good, it just slows things down without meaningfully changing results.
❌ "Cosine similarity is a good measure of document relevance"
✅ Cosine similarity measures vector distance in embedding space, not query-document relevance. Two documents can be embedding-close without either actually answering the question. Cross-encoders fix this by reading both together.
💡 Key Takeaway
Re-ranking = read query + document together → true relevance score. Two-stage pattern: broad fast retrieval (top-50) then precise expensive re-ranking (top-5). Reserve it for complex or high-stakes queries where approximate cosine ordering leads to wrong context in the generation step.
Concept 4 of 6
Big Idea
Query Transformation
What if the user's question is the problem? Vague, ambiguous, or poorly worded queries lead to poor retrieval no matter how good your search infrastructure is. Query transformation fixes the input before it reaches the search engine. Three strategies: HyDE (generate a hypothetical answer, embed that instead), Multi-Query (3–5 reformulations, union results), and Step-Back (abstract to a broader question first).
The core insight: Document embeddings are trained on document language, not question language. A hypothetical answer uses the same vocabulary and phrasing as real documents — so its embedding lands much closer to actual relevant content in vector space.
Concept 4 of 6
Analogy
The Skilled Librarian
💡 Everyday Analogy
BEFORE: You walk into a library and ask the librarian "I need info about that thing where companies report their debts." The librarian stares blankly — that could mean annual reports, credit filings, bankruptcy proceedings, or SEC disclosures.
PAIN: With a vague question, the librarian might hand you a random book on corporate finance, wasting your time. One search for one vague phrase misses documents using different vocabulary.
MAPPING: A skilled librarian rephrases your question into three specific queries: "UCC-1 financing statement filings," "commercial debt disclosure requirements," and "secured transaction public records." Each version captures a different angle, and the combined results cover what you actually needed. That's what query transformation does — it improves the question before the search engine ever sees it.
Concept 4 of 6
How It Works
Three Query Transformation Strategies
1HyDE (Hypothetical Document Embedding): Ask Claude to generate a hypothetical answer paragraph. Embed THAT instead of the raw question. The hypothesis uses document-like language, so its embedding lands closer to real relevant documents.
2Multi-Query: Generate 3–5 different phrasings of the original question. Run retrieval for each. Take the union of all results (deduplicating). Catches documents that match one phrasing but not another.
3Step-Back Prompting: Abstract the specific question to a broader one. "How does filing #2024-FL-0093 affect debtor?" → "How do UCC filings generally affect debtor risk?" Retrieve broader context first, then use it alongside the specific query.
Concept 4 of 6
Pseudocode
HyDE + Multi-Query Pipeline
# Strategy A: HyDEFUNCTION hyde_search(question, k=5):
hypothesis = CLAUDE_GENERATE(
"Write a 2-sentence paragraph that would answer: " + question
)
embedding = EMBED(hypothesis) # NOT the question!RETURNVECTOR_SEARCH(embedding, k)
# Strategy B: Multi-QueryFUNCTION multi_query_search(question, k=5):
variants = CLAUDE_GENERATE(
"Rephrase this question 3 ways: " + question
)
all_results = []
FOR q IN [question] + variants:
all_results.EXTEND(HYBRID_SEARCH(q, k))
RETURNDEDUPLICATE_AND_RERANK(all_results)[:k]
Concept 4 of 6
Misconceptions + Takeaway
Query Transformation Misunderstandings
❌ "Query transformation helps every query"
✅ Simple factual queries ("What is the filing date for UCC-0093?") don't benefit from HyDE or Multi-Query. These techniques shine on complex, multi-part, or ambiguous questions where a single search cannot capture all aspects needed.
❌ "HyDE always produces better embeddings than embedding the question directly"
✅ HyDE works best when the hypothesis accurately represents the answer domain. If Claude generates a poor hypothesis (hallucinating content far from real documents), HyDE can actually hurt retrieval. Always evaluate on your dataset.
💡 Key Takeaway
Query transformation fixes bad questions before they reach search. HyDE generates a hypothetical answer to bridge the vocabulary gap between user questions and document language. Multi-Query expands retrieval surface by rephrasing. Use these for complex, ambiguous queries — skip them for simple factual lookups.
Concept 5 of 6
Big Idea
Agentic RAG
All previous patterns are still "passive" — one retrieval call per user turn. Agentic RAG gives the agent a search tool and lets it call that tool multiple times, adapting each query based on what it saw before, routing to different corpora, and stopping only when it has enough to answer. It's the M12 ReAct loop applied to retrieval.
Passive RAG
One query → fixed top-k chunks → Claude generates. Simple, fast. Fails when the answer spans multiple retrievals.
Agentic RAG
Agent calls search tool, reads chunks, decides if it needs more, re-queries, stops when ready. 2–6 LLM calls per question.
💡 When to Use It
Use when: questions span multiple corpora; multi-hop retrieval needed; single-shot retrieval keeps failing. Skip when: hybrid + re-rank already works; latency budget is tight; you can't measure improvement on your eval set.
Concept 5 of 6
Analogy
Vending Machine vs. Research Assistant
💡 Everyday Analogy
BEFORE: Naive RAG is a vending machine — you put in a query, it spits out top-k chunks, you eat what comes out. Advanced patterns (hybrid, re-rank, transform) are a better vending machine — smarter sorting, more selection — but still one-shot: one query in, one set of chunks out.
PAIN: The pain shows up the moment a question can't be answered from a single retrieval. "Compare Acme's 2024 and 2025 quarterly filings" needs two retrievals — one per year — then a synthesis step. A vending machine can't do this.
MAPPING: Agentic RAG hands retrieval over to a research assistant — the agent itself. It treats the retriever as a tool, calls it, looks at what came back, decides whether to call it again with a refined query, routes to a different document collection, and stops when it has enough. Same ReAct patterns you know from M12, applied to the search problem.
Concept 5 of 6
How It Works
Multi-Step Retrieval Loop
1Define search as a tool: Register a search(corpus, query, k) tool with Claude. The tool description tells it when to use each corpus.
2Agent reads the question: Claude decides which corpus to query and how to phrase the search. It calls the tool.
3Read returned chunks: If the chunks fully answer the question, Claude returns end_turn with the answer.
4Refine and retry: If chunks are insufficient, Claude refines the query, tries a different corpus, or searches for additional context. Repeat up to max_iters.
5Escalate on failure: If max_iters reached without sufficient context, escalate to a human — never fabricate. Same M17 guardrail pattern.
Concept 5 of 6
Pseudocode
Agentic RAG Loop
# search is registered as a tool with ClaudeDEFINE tool "search":
inputs: corpus (which index), query, k
returns: top-k relevant chunks
FUNCTION agentic_rag(question, max_iters=6):
messages = [{role: "user", content: question}]
FOR step IN range(max_iters):
response = CLAUDE(tools=[search], messages)
IF response.stop_reason == "end_turn":
RETURN response.text # agent has enough# Execute the search tool Claude requested
tool_results = RUN_TOOLS(response.tool_calls)
messages.APPEND(response, tool_results)
# Claude reads results → decides next actionRAISE"Escalate: max iterations reached"
Concept 5 of 6
Misconceptions + Takeaway
Agentic RAG Misunderstandings
❌ "Agentic RAG replaces hybrid search and re-ranking"
✅ The agent uses them. Inside the search tool you should still do hybrid retrieval and cross-encoder re-ranking. Agentic RAG is the outer orchestration loop; hybrid + re-rank is the inner implementation of each search call.
❌ "If passive RAG works, agentic RAG works better"
✅ For single-fact lookups, agentic RAG can underperform because the agent second-guesses correct first-shot results and makes unnecessary extra calls. Measure on your eval set before switching — 4× the LLM cost only makes sense if accuracy improves measurably.
💡 Key Takeaway
Agentic RAG = search as a tool + ReAct loop. Use it when questions require multi-hop retrieval, span multiple corpora, or when single-shot retrieval keeps failing despite well-tuned hybrid + re-rank. Cost: ~4× the LLM calls of passive RAG. Always cap iterations and escalate rather than fabricate.
Concept 6 of 6
Big Idea
Compression & Evaluation
Contextual compression uses an LLM to extract only query-relevant sentences from each retrieved chunk before sending them to Claude. A 487-token chunk becomes 62 tokens — 87% reduction — with only the relevant sentences surviving. Less noise = better answers + lower cost.
RAG evaluation measures whether your pipeline actually works. Three metrics: Precision (how many retrieved docs were relevant?), Recall (did you miss any relevant docs?), and Faithfulness (does the answer stick to the retrieved context, not hallucinate?).
💡 Why It Matters
Without evaluation metrics, you're optimizing blindly. A team spent 3 weeks tuning embeddings only to discover their bottleneck was chunking — 40% of relevant paragraphs were split across two chunks and neither half was retrievable alone. Measure first, then optimize the right thing.
Concept 6 of 6
Analogy
Highlighted Textbook Pages
💡 Everyday Analogy
BEFORE: You're studying for an exam and a friend hands you five full textbook pages that "contain the answer somewhere."
PAIN: You have to scan 5,000 words to find the three sentences that actually matter. The surrounding text might confuse you into thinking irrelevant details are important — and you're now paying for all 5,000 words in your context window.
MAPPING: Contextual compression is your friend highlighting only the relevant sentences before handing you the pages. Claude reads each retrieved chunk in context of the query, extracts only the parts that answer the question, and throws away the rest. Less noise means better answers, fewer distraction paths for hallucination, and 80–90% fewer tokens billed for generation.
Concept 6 of 6
How It Works
Compression + Evaluation Metrics
1Extractive compression: Give Claude a query + chunk. Ask it to copy verbatim only the sentences that directly answer the query. Preserves exact wording for citations.
2Compressed context: Send only the extracts (not full chunks) to the generation prompt. 87% token savings reduces cost and hallucination surface.
3Precision measurement: Of retrieved chunks, how many were actually relevant? (3 relevant / 5 retrieved = 60%). Tells you if you're retrieving junk.
4Recall measurement: Of all relevant documents in the corpus, how many did you retrieve? Tells you if you're missing important docs.
5Faithfulness measurement: Does the generated answer only use claims from the retrieved context? Requires LLM-based evaluation — separate API call from generation.
❌ "One aggregate accuracy number is enough to evaluate RAG"
✅ Aggregate metrics hide per-query-type failures. A system might hit 85% on factual lookups but 40% on multi-hop questions. Track Precision, Recall, and Faithfulness separately per query type to find real bottlenecks.
❌ "Use Claude to evaluate its own RAG outputs in the same session"
✅ Same-session self-review creates confirmation bias — the model retains reasoning context from generation. For faithful evaluation, use separate API calls (separate sessions) for generation and quality assessment.
💡 Key Takeaway
Compression extracts only query-relevant sentences (80–90% token reduction, lower hallucination). Evaluation with Precision, Recall, and Faithfulness tells you exactly which pipeline stage to fix. Measure per query type — averages lie. Evaluate generation and quality in separate sessions to avoid self-confirmation bias.
Quick Quiz
Test Your Understanding
Tap each question to reveal the answer.
Q1: Why does naive RAG fail on queries like "invoice #INV-2024-0847"?
Tap to reveal ↓
Vector search embeds the query semantically and finds "invoices in general" — but the specific ID "INV-2024-0847" is a rare, exact string that vector embeddings treat as opaque. BM25 keyword search finds the exact match immediately. Hybrid search solves this by combining both strategies and fusing with RRF.
Q2: Why does HyDE improve retrieval over embedding the raw question?
Tap to reveal ↓
Documents and questions use different vocabulary and style. A question like "How does UCC filing affect the debtor?" has a different embedding than the document that answers it. HyDE generates a hypothetical answer paragraph, which uses the same document-like language and vocabulary, so its embedding lands much closer to real relevant documents in vector space.
Q3: What is the difference between a bi-encoder and a cross-encoder in re-ranking?
Tap to reveal ↓
A bi-encoder creates separate embeddings for query and document, then compares them with cosine similarity. Fast (pre-compute document embeddings) but misses subtle relevance signals. A cross-encoder reads the query and document together as a single input and scores their joint relevance — much more accurate but too slow to run on all documents. Use bi-encoders for first-pass retrieval, cross-encoders for re-ranking the short list.
Q4: What does "faithfulness" measure in RAG evaluation, and why is it distinct from precision/recall?
Tap to reveal ↓
Faithfulness measures whether the generated answer only uses information from the retrieved context — does the LLM stick to what it was given, or does it hallucinate facts? Precision and recall measure retrieval quality (did you find the right documents?). Faithfulness measures generation quality (did Claude stay grounded in those documents?). A system can have perfect retrieval but low faithfulness if Claude fabricates beyond the context.
Q5: When should you NOT use agentic RAG?
Tap to reveal ↓
Skip agentic RAG when: (1) hybrid search + re-ranking already retrieves the right chunk on the first try — don't pay for extra LLM calls you don't need; (2) latency budget is sub-second — each retrieval step adds a full model round-trip; (3) your eval set is too small to measure whether extra calls actually improve answer quality. Start with passive RAG + hybrid + re-rank; only add agentic retrieval when measured eval shows it helps.
💻
Ready to build an advanced RAG pipeline?
The desktop version includes full code walkthroughs for BM25 + hybrid search, cross-encoder re-ranking, HyDE, and agentic RAG — with Python and Node.js, copy-paste ready.