The 4 concepts of M17

Tap any concept to jump to its 5-card cluster.

• 1Output ValidationHallucination, toxicity, format — catch defects before shipment
• 2Cost ControlsBudget ceilings, loop detection, fallback responses
• 3Human-in-the-Loop PatternsApproval, modification, and escalation gates
• 4Circuit Breaker PatternTrip on N failures, recover via half-open testing

Output Validation

Every Claude response is a potential failure mode. The model can hallucinate a fact, leak PII it pulled from a database row, or generate off-policy text. Output validation is the family of cheap, layered checks that intercept the response before it ever reaches the user — the same defense-in-depth shape as M16, just on the way out.

Three core checks: format validation (is the JSON well-formed and complete?), hallucination detection (is each factual claim supported by the source documents?), and toxicity / policy filtering (does the response stay inside your content rules?). Cheap checks first, expensive ones last. Verdicts are limited to three: allow, redact-with-disclosure, or block.

The factory quality inspector

BEFORE: A factory with no quality control. Products roll off the assembly line and ship straight to the customer. The machines work perfectly 99% of the time, the line moves fast, and most boxes arrive without complaint.

PAIN: The 1% is enough. One car with a faulty brake line, one toy with a sharp edge, one bottle of medicine with the wrong dosage. The machine doesn't know it made a mistake — it has no way to inspect its own output. For high-value items even a tiny defect rate is unacceptable.

MAPPING: Real factories put inspectors at the end of the line. Most products pass; defects get pulled. Output guardrails are the same: format checkers, hallucination detectors, and toxicity classifiers inspect every Claude response. Routine outputs auto-pass. Defective ones get blocked or redacted.

Three checks, cheapest first

1. Format validationPure local parsing — no API call. Is the JSON syntactically valid? Are all required fields present? Are values inside expected ranges (positive amount, non-future date)? Cheapest, fastest, catches the most common failures.
2. Hallucination / groundedness checkFor RAG agents: a separate Claude-as-judge call receives the response plus the retrieved sources. It classifies each factual claim as supported, unsupported, or contradicted. One contradicted claim → block and ask the agent to retry against the actual evidence.
3. Toxicity / policy classifierA separate Claude call rates the response on a severity scale against your content policy. Toxic, off-brand, defamatory, or out-of-scope text gets blocked. Different prompt context than the main agent so the response can't talk the judge out of its verdict.

Order matters: format check (free) runs first, then hallucination, then toxicity. Skip downstream checks if upstream blocks — you don't pay $0.003 to score text you're already throwing away.

Pseudocode

The validation pipeline that wraps every Claude response:

FUNCTION validate_output(response, sources):
  # 1. Format check — cheapest, run first
  IF NOT parses_as_json(response):
    RETURN block("format: invalid JSON")
  IF missing_required_fields(response):
    RETURN block("format: required field missing")
  IF NOT values_in_range(response):
    RETURN block("format: value out of range")

  # 2. Hallucination — separate verifier vs sources
  claims = verify_claims(response, sources)
  FOR c IN claims:
    IF c.status == "contradicted":
      RETURN block("hallucination", claim=c)

  # 3. Toxicity — Claude-as-judge, severity 1–5
  verdict = classify_toxicity(response)
  IF verdict.severity >= 4:
    RETURN block("policy")

  RETURN allow(response)

Three legitimate verdicts only: allow, redact-with-disclosure, block. Never silently rewrite Claude's answer.

Misconceptions

Cost Controls

An agent loop that costs $0.02 in testing can cost $15 in production when an edge case triggers 50 iterations. Cost controls are the middleware layer that wraps every Claude call, tracks cumulative tokens and dollars, and short-circuits the loop when the next call would exceed a per-request, per-user, or wall-clock budget.

Without them, a single bug can burn through your monthly budget in hours. With them, runaway loops cap at 8–10 iterations instead of 50, and your worst-case cost per request is a known number, not a surprise on next month's bill.

The teenager's phone bill

BEFORE: A parent gives their teenager a phone with an unlimited-everything plan. The phone works perfectly — calls, texts, data, video.

PAIN: A $2,000 bill arrives. The teenager discovered international calling. The plan didn't malfunction; it just had no ceiling between "reasonable use" and "catastrophic cost." Nothing in the system said "stop at $50."

MAPPING: An agent without cost controls is the same story. It works perfectly in testing — 3-4 iterations, $0.02 per request. In production, an edge case spawns 50 iterations of 100K-token prompts, and one user request costs $15. A per-request budget is the prepaid card: when the balance hits zero, the next call declines and the agent returns a graceful fallback instead of continuing to spend.

Four levels of cost control

1. Per-request budgetCap on tokens or dollars for a single agent invocation. Agent terminates with a fallback when the next call would exceed the cap. Like a prepaid card — balance hits zero, card declines.
2. Per-user budgetDaily or monthly limit per user. Prevents one power user from eating your entire monthly allocation with expensive queries all day.
3. Loop detectionSpots the same tool being called repeatedly without making progress — an agent retrying a failing API forever, each retry costing tokens but producing nothing. Break the cycle.
4. Execution time limitsWall-clock timeout that kills the process. Catches infinite reasoning loops that generate few tokens per iteration but never terminate — the case token caps miss.

Implement as middleware that wraps every Claude API call. Your agent code stays clean — it doesn't need to know budgets exist.

Pseudocode

A cost-tracking middleware that sits between agent code and the API client:

CLASS CostMiddleware:
  budget       = $0.50
  spent        = $0.00
  call_history = []

  FUNCTION call_claude(prompt, tools):
    estimated = estimate_cost(prompt, tools)

    # 1. Per-request ceiling
    IF spent + estimated > budget:
      RETURN fallback("budget_exceeded")

    # 2. Loop detection — same tool, same args, 3x
    IF repeated_call(call_history, prompt, n=3):
      RETURN fallback("loop_detected")

    # 3. Wall-clock timeout
    IF elapsed() > MAX_SECONDS:
      RETURN fallback("timeout")

    response = anthropic.create(prompt, tools)
    spent += actual_cost(response)
    call_history.append(prompt)
    RETURN response

Estimate cost before the call; record actual cost after. The estimate is for the gate; the actual is for the running total.

Misconceptions

Human-in-the-Loop Patterns

Some decisions are too important to fully automate. Human-in-the-loop (HITL) is the design pattern where the agent handles the routine 95%, then pauses at predefined gates — high blast radius, irreversible action, low confidence — and hands the proposed action to a human reviewer with full context.

Three flavors. Approval gates say "do you want me to do this?" before an irreversible action like a refund or external email. Modification gates draft content and let a human edit before send. Escalation gates route the entire problem to a human specialist when the agent recognizes it's hit its competence boundary.

The autopilot handoff

BEFORE: Early flight required active human control every second — pilots couldn't step away from the yoke.

PAIN: Modern autopilot handles routine cruise perfectly for hours. But "always on" autopilot would be unsafe — takeoff, landing, turbulence, system warnings, and unusual ATC instructions all need human judgment. A system that never hands control back is as dangerous as one that never takes it in the first place.

MAPPING: The fix isn't more or less automation — it's well-defined handoff points. Modern aircraft autopilot disengages on takeoff, landing, and exception conditions. Your agent does the same: routine inquiries auto-resolve, but at refund gates, escalation gates, and confidence cliffs it pauses and surfaces the decision to a human reviewer with all the context.

Three gate patterns + state machine

1. Approval gateAgent proposes "Send $450 refund to customer X" with full context. Human clicks Approve / Deny. Used for: payments, external emails, database modifications, deploys.
2. Modification gateAgent generates a draft (customer email, report, contract clause). Human edits before send. Used for: customer communications, legal docs, anything that needs a human voice.
3. Escalation gateAgent recognizes "I can't resolve this" — policy gap, capability limit, or ambiguity — and routes the whole problem to a human specialist with a structured handoff summary.
4. State machine wraps itThe agent enters waiting_for_human, the proposal goes to a queue, the human gets notified, and the agent resumes only on approve/edit/escalate. Timeout → auto-escalate to a manager.

The human should see the proposed action, the full context, and one-click options. Forcing them to reconstruct the conversation defeats the entire pattern.

When to escalate

# For each proposed action, check these axes:

IF action is irreversible
  (refund issued, email sent, record deleted):
  GATE: approval

ELIF action exceeds business threshold
  (refund > $X, contract over Y days,
   payment to new account):
  GATE: approval

ELIF output goes to a human + needs voice
  (customer email, legal letter):
  GATE: modification

ELIF agent hit a policy gap or capability limit
  ("I don't know how to handle this"):
  GATE: escalation

ELSE:
  AUTO-resolve  # the routine 95%

# Tier on blast radius & reversibility, NOT volume.
# Anti-pattern: escalate on customer sentiment alone.

An angry customer with a simple request does NOT need a human. A calm customer hitting a policy gap DOES. Sentiment is a poor escalation signal.

Misconceptions

Circuit Breaker Pattern

HITL gates handle individual decisions. The circuit breaker handles systemic failures — when a downstream API has been failing for two minutes straight, when your hallucination detector starts blocking everything, when 429 rate limits cascade. After N consecutive failures, the breaker trips and routes all subsequent requests to a fallback response — no API call made — until a cooldown expires.

It's a state machine with three states: Closed (normal flow, count failures), Open (tripped, all requests get fallback, cooldown timer running), Half-Open (cooldown done, one test request allowed). Test passes → close. Test fails → re-open with longer backoff.

The household breaker box

BEFORE: A house wired without circuit breakers. Power flows directly to every outlet, all the time, with nothing in between.

PAIN: A faulty appliance, a lightning strike, or one too many devices on a single circuit pushes current past the safe limit. The wiring overheats, insulation melts, a fire starts. Without a breaker, one bad appliance can burn down the house.

MAPPING: The household breaker monitors current and TRIPS when it crosses a danger threshold — cutting power instantly. Once you fix the problem (unplug the appliance), you flip the breaker back. An agent breaker does the same with API failures: 3 consecutive errors → trip, all requests get fallback responses, no money burned on doomed calls. After 60s, one test request feels things out. Pass → close. Fail → longer cooldown.

Three states, two transitions

1. CLOSED — normalRequests flow through to Claude. Failures are counted. Successes reset the counter. If failures cross the threshold (e.g., 3 in 5 minutes) → trip.
2. OPEN — trippedALL requests immediately receive the fallback response — no API calls made, no money spent on failing calls. A cooldown timer starts (e.g., 60s).
3. HALF-OPEN — testingCooldown expires. ONE probe request is allowed through. If it succeeds → close (back to normal). If it fails → re-open with exponential backoff (60s, 2min, 4min…).
4. What counts as a failure?API 5xx, 429 rate limits, hallucination-detector blocks, guardrail violations, timeouts. Each failure type can have its own threshold and cooldown.

Tolerate 5 rate-limit errors (transient) but trip on just 2 consecutive hallucination blocks (suggests something deeper is wrong).

Pseudocode

The breaker as a state machine wrapped around your Claude call:

CLASS CircuitBreaker:
  state    = "closed"
  failures = 0
  threshold = 3
  cooldown_until = NONE

  FUNCTION call(fn):
    # OPEN: short-circuit until cooldown expires
    IF state == "open":
      IF now() < cooldown_until:
        RETURN fallback("breaker_open")
      state = "half_open"   # promote to test

    TRY:
      result = fn()
      IF state == "half_open":
        state = "closed"      # probe passed
      failures = 0
      RETURN result

    CATCH err:
      failures += 1
      IF failures >= threshold OR state == "half_open":
        state = "open"
        cooldown_until = now() + backoff(failures)
      RETURN fallback(err)

Backoff doubles on each re-trip: 60s → 120s → 240s. Don't hammer a service that's already failing.

Misconceptions

Tap a card to reveal the answer