← M02: Structured Output 🏠 Home M04: Memory →

M03 — Multi-Tool Orchestration

One tool made Acme an agent. A toolbox makes it useful. In this module you give Acme three tools — check an order, search products, process a refund — and let the model pick the right one, or several at once. The surprise: the loop you wrote in M01 already handles all of it. You'll barely change it.

Learning Objectives

  • Register multiple tools and let the model route each request to the right one
  • Understand that tool descriptionsThe natural-language "description" field on each tool is what the model reads to decide when to use it. Vague descriptions cause wrong-tool bugs; precise ones are your routing logic. ARE the routing logic — write them like they matter, because they do
  • Handle parallel tool calls: one model turn can request several tools at once
  • Replace M01's if/else with a dispatch table that scales to any number of tools
  • Steer behavior with tool_choice (auto / required / a specific tool / none) and know when to disable parallel calls
  • Recognize which tools are dangerous (a refund writes money) and flag them for the guardrails you'll add later

From One Tool to a Toolbox

Everyday Analogy

BEFORE: A front-desk receptionist at a big company doesn't personally handle billing, tech support, and shipping. They hold a directory. You say "I need a refund and I'm also wondering if you have blue running shoes," and they route you: refund goes to Billing, product question goes to Sales — often both calls placed at the same time.

PAIN: If the receptionist only had one phone line to one department, every off-topic question would dead-end. And if their directory descriptions were vague ("Dave — stuff"), they'd transfer you to the wrong desk constantly.

MAPPING: Your agent is the receptionist; your tools are the departments; the tool descriptions are the directory entries. Give the model several well-described tools and it routes each request correctly — and, like a good receptionist, it can place two calls at once when a message needs two departments. The loop doesn't change; the toolbox and the descriptions do.

Why It Matters

Real agents are defined by their tool surface. A support agent with only get_order_status is a lookup bot; add search_products and process_refund and it can actually resolve tickets end to end. But more tools means more ways to route wrong — and the fix is almost never "better model," it's "better descriptions." This module is where you learn to think in tool surfaces, the skill every framework in M06–M07 assumes you already have.

How the Model Chooses

Technical Definition — Descriptions Are Routing

When you pass several tools, the model reads each tool's name and description and decides — per request — which one(s) fit. There's no separate router you configure; the descriptions are the router. "Look up an order's status by ID" and "Search the product catalog by keyword" are different enough that the model routes cleanly. Two tools described as "get info" and "fetch data" would collide constantly. The rule: write each description to answer "when should I use this, and when should I NOT?" Include the trigger ("Use only when the customer explicitly asks for a refund") for anything risky.

Parallel Tool Calls

Here's the capability that trips up newcomers: a single model turn can request more than one tool at once. Ask Acme "Where's my order AC-1042, and do you have blue running shoes?" and the model returns two tool requests in one response — get_order_status and search_products. All three SDKs do this by default. Your job: run them all, return all the results together, and loop.

Watch one two-part question fan out into two parallel tool calls, then merge back into a single answer:

Parallel Orchestration — One Turn, Two Tools
"Where's order AC-1042, and do you have blue running shoes?"one user message
↓
Model returns two tool requests at onceparallel tool_use / function_calls
↙↘
get_order_status("AC-1042")→ shipped, UPS, Tue
search_products("blue running shoes")→ 1 match, $89
↘↙
Both results returned in one turn → final answer"Your order ships Tuesday, and yes — the blue Trailblazer is $89."
The #1 Parallel-Tool Bug

When the model requests two tools, you must return both results before the next model call — in a single turn (Anthropic: one user message with two tool_result blocks; Gemini: one tool Content with two functionResponse parts; OpenAI: two function_call_output items appended together). Return only one, or split them across turns, and the SDK errors or the model silently stops making parallel calls. Loop over all the requests; answer all of them.

Step 1 — The Toolbox

Three plain functions — still no AI. We extend M01's get_order_status with search_products and process_refund. Notice process_refund is a write action: it (pretends to) move money. We'll let the model call it now, but flag it — in M06–M07 you'll gate write-tools behind confirmation.

# acme_tools.py — three real functions, shared by all three SDKs.
FAKE_ORDERS = {
    "AC-1042": {"status": "shipped", "carrier": "UPS", "eta": "Tuesday"},
    "AC-2088": {"status": "processing", "carrier": None, "eta": "unknown"},
}
PRODUCTS = [
    {"sku": "SH-01", "name": "Trailblazer Running Shoe", "color": "blue",  "price": 89.0},
    {"sku": "SH-02", "name": "Trailblazer Running Shoe", "color": "black", "price": 89.0},
    {"sku": "BP-07", "name": "Summit Daypack",           "color": "green", "price": 54.0},
]

def get_order_status(order_id: str) -> dict:
    o = FAKE_ORDERS.get(order_id.upper())
    return {"error": f"No order {order_id}."} if o is None else {"order_id": order_id.upper(), **o}

def search_products(query: str) -> dict:
    q = query.lower()
    hits = [p for p in PRODUCTS if q in p["name"].lower() or q in p["color"].lower()]
    return {"query": query, "results": hits}

def process_refund(order_id: str, reason: str) -> dict:
    # WRITE action — moves money. Gated behind confirmation in M06–M07.
    return {"refund_id": f"RF-{order_id[-4:]}", "order_id": order_id.upper(),
            "status": "approved", "reason": reason}
// acmeTools.mjs — three real functions, shared by all three SDKs.
const FAKE_ORDERS = {
  "AC-1042": { status: "shipped", carrier: "UPS", eta: "Tuesday" },
  "AC-2088": { status: "processing", carrier: null, eta: "unknown" },
};
const PRODUCTS = [
  { sku: "SH-01", name: "Trailblazer Running Shoe", color: "blue",  price: 89.0 },
  { sku: "SH-02", name: "Trailblazer Running Shoe", color: "black", price: 89.0 },
  { sku: "BP-07", name: "Summit Daypack",           color: "green", price: 54.0 },
];

export function getOrderStatus(orderId) {
  const o = FAKE_ORDERS[orderId.toUpperCase()];
  return o ? { order_id: orderId.toUpperCase(), ...o } : { error: `No order ${orderId}.` };
}
export function searchProducts(query) {
  const q = query.toLowerCase();
  const results = PRODUCTS.filter(p => p.name.toLowerCase().includes(q) || p.color.includes(q));
  return { query, results };
}
export function processRefund(orderId, reason) {
  // WRITE action — moves money. Gated behind confirmation in M06–M07.
  return { refund_id: `RF-${orderId.slice(-4)}`, order_id: orderId.toUpperCase(),
           status: "approved", reason };
}

Step 2 — The Orchestration Loop

Now the loop. Compare it to M01's: the structure is identical. Two changes only — the tool list has three entries, and we dispatch through a lookup table (DISPATCH[name]) instead of an if/else. The loop already iterates over all requested tools, so parallel calls just work. Each tab is a complete, runnable file.

# orchestrate_anthropic.py
import json
from anthropic import Anthropic
from acme_tools import get_order_status, search_products, process_refund

client = Anthropic()
SYSTEM = "You are Acme Support. Use the tools for orders, product search, and refunds."

TOOLS = [
    {"name": "get_order_status", "description": "Look up an order's status by its ID (e.g. AC-1042).",
     "input_schema": {"type": "object", "properties": {"order_id": {"type": "string"}}, "required": ["order_id"]}},
    {"name": "search_products", "description": "Search the product catalog by keyword or color.",
     "input_schema": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}},
    {"name": "process_refund", "description": "Issue a refund for an order. Use ONLY when the customer explicitly asks for a refund.",
     "input_schema": {"type": "object", "properties": {"order_id": {"type": "string"}, "reason": {"type": "string"}}, "required": ["order_id", "reason"]}},
]
DISPATCH = {"get_order_status": get_order_status, "search_products": search_products, "process_refund": process_refund}

def run_agent(user_message: str) -> str:
    messages = [{"role": "user", "content": user_message}]
    for _ in range(8):                          # loop cap (see M01)
        resp = client.messages.create(model="claude-sonnet-5", max_tokens=1024,
                                      system=SYSTEM, tools=TOOLS, messages=messages)
        if resp.stop_reason != "tool_use":
            return "".join(b.text for b in resp.content if b.type == "text")
        messages.append({"role": "assistant", "content": resp.content})
        results = []
        for block in resp.content:              # may be SEVERAL tool_use blocks (parallel)
            if block.type == "tool_use":
                out = DISPATCH[block.name](**block.input)     # dispatch table, not if/else
                results.append({"type": "tool_result", "tool_use_id": block.id, "content": json.dumps(out)})
        messages.append({"role": "user", "content": results})  # ALL results in ONE turn
    return "Sorry, I couldn't finish that — let me get a human."

if __name__ == "__main__":
    print(run_agent("Where's order AC-1042, and do you have blue running shoes?"))
// orchestrate_anthropic.mjs
import Anthropic from "@anthropic-ai/sdk";
import { getOrderStatus, searchProducts, processRefund } from "./acmeTools.mjs";

const client = new Anthropic();
const SYSTEM = "You are Acme Support. Use the tools for orders, product search, and refunds.";

const TOOLS = [
  { name: "get_order_status", description: "Look up an order's status by its ID (e.g. AC-1042).",
    input_schema: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"] } },
  { name: "search_products", description: "Search the product catalog by keyword or color.",
    input_schema: { type: "object", properties: { query: { type: "string" } }, required: ["query"] } },
  { name: "process_refund", description: "Issue a refund for an order. Use ONLY when the customer explicitly asks.",
    input_schema: { type: "object", properties: { order_id: { type: "string" }, reason: { type: "string" } }, required: ["order_id", "reason"] } },
];
const DISPATCH = { get_order_status: (a) => getOrderStatus(a.order_id),
                   search_products: (a) => searchProducts(a.query),
                   process_refund: (a) => processRefund(a.order_id, a.reason) };

async function runAgent(userMessage) {
  const messages = [{ role: "user", content: userMessage }];
  for (let i = 0; i < 8; i++) {
    const resp = await client.messages.create({ model: "claude-sonnet-5", max_tokens: 1024,
      system: SYSTEM, tools: TOOLS, messages });
    if (resp.stop_reason !== "tool_use")
      return resp.content.filter(b => b.type === "text").map(b => b.text).join("");
    messages.push({ role: "assistant", content: resp.content });
    const results = [];
    for (const block of resp.content) {          // may be several (parallel)
      if (block.type === "tool_use") {
        const out = DISPATCH[block.name](block.input);
        results.push({ type: "tool_result", tool_use_id: block.id, content: JSON.stringify(out) });
      }
    }
    messages.push({ role: "user", content: results });   // ALL results in ONE turn
  }
  return "Sorry, I couldn't finish that — let me get a human.";
}

console.log(await runAgent("Where's order AC-1042, and do you have blue running shoes?"));
# orchestrate_gemini.py
from google import genai
from google.genai import types
from acme_tools import get_order_status, search_products, process_refund

client = genai.Client()
SYSTEM = "You are Acme Support. Use the tools for orders, product search, and refunds."

toolbox = types.Tool(function_declarations=[
    types.FunctionDeclaration(name="get_order_status", description="Look up an order's status by its ID (e.g. AC-1042).",
        parameters_json_schema={"type": "object", "properties": {"order_id": {"type": "string"}}, "required": ["order_id"]}),
    types.FunctionDeclaration(name="search_products", description="Search the product catalog by keyword or color.",
        parameters_json_schema={"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]}),
    types.FunctionDeclaration(name="process_refund", description="Issue a refund. Use ONLY when the customer explicitly asks.",
        parameters_json_schema={"type": "object", "properties": {"order_id": {"type": "string"}, "reason": {"type": "string"}}, "required": ["order_id", "reason"]}),
])
config = types.GenerateContentConfig(system_instruction=SYSTEM, tools=[toolbox])
DISPATCH = {"get_order_status": get_order_status, "search_products": search_products, "process_refund": process_refund}

def run_agent(user_message: str) -> str:
    contents = [types.Content(role="user", parts=[types.Part(text=user_message)])]
    for _ in range(8):
        resp = client.models.generate_content(model="gemini-2.5-flash", contents=contents, config=config)
        if not resp.function_calls:
            return resp.text
        contents.append(resp.candidates[0].content)
        parts = []
        for fc in resp.function_calls:          # may be SEVERAL (parallel)
            out = DISPATCH[fc.name](**dict(fc.args))
            parts.append(types.Part.from_function_response(name=fc.name, response={"result": out}))
        contents.append(types.Content(role="tool", parts=parts))   # ALL results in ONE turn
    return "Sorry, I couldn't finish that — let me get a human."

if __name__ == "__main__":
    print(run_agent("Where's order AC-1042, and do you have blue running shoes?"))
// orchestrate_gemini.mjs
import { GoogleGenAI, Type } from "@google/genai";
import { getOrderStatus, searchProducts, processRefund } from "./acmeTools.mjs";

const ai = new GoogleGenAI({});
const SYSTEM = "You are Acme Support. Use the tools for orders, product search, and refunds.";

const toolbox = { functionDeclarations: [
  { name: "get_order_status", description: "Look up an order's status by its ID (e.g. AC-1042).",
    parameters: { type: Type.OBJECT, properties: { order_id: { type: Type.STRING } }, required: ["order_id"] } },
  { name: "search_products", description: "Search the product catalog by keyword or color.",
    parameters: { type: Type.OBJECT, properties: { query: { type: Type.STRING } }, required: ["query"] } },
  { name: "process_refund", description: "Issue a refund. Use ONLY when the customer explicitly asks.",
    parameters: { type: Type.OBJECT, properties: { order_id: { type: Type.STRING }, reason: { type: Type.STRING } }, required: ["order_id", "reason"] } },
]};
const config = { systemInstruction: SYSTEM, tools: [toolbox] };
const DISPATCH = { get_order_status: (a) => getOrderStatus(a.order_id),
                   search_products: (a) => searchProducts(a.query),
                   process_refund: (a) => processRefund(a.order_id, a.reason) };

async function runAgent(userMessage) {
  const contents = [{ role: "user", parts: [{ text: userMessage }] }];
  for (let i = 0; i < 8; i++) {
    const resp = await ai.models.generateContent({ model: "gemini-2.5-flash", contents, config });
    if (!resp.functionCalls || resp.functionCalls.length === 0) return resp.text;
    contents.push(resp.candidates[0].content);
    const parts = [];
    for (const fc of resp.functionCalls) {       // may be several (parallel)
      const out = DISPATCH[fc.name](fc.args);
      parts.push({ functionResponse: { name: fc.name, response: { result: out } } });
    }
    contents.push({ role: "tool", parts });      // ALL results in ONE turn
  }
  return "Sorry, I couldn't finish that — let me get a human.";
}

console.log(await runAgent("Where's order AC-1042, and do you have blue running shoes?"));
# orchestrate_openai.py  (Responses API)
import json
from openai import OpenAI
from acme_tools import get_order_status, search_products, process_refund

client = OpenAI()
SYSTEM = "You are Acme Support. Use the tools for orders, product search, and refunds."

TOOLS = [
    {"type": "function", "name": "get_order_status", "description": "Look up an order's status by its ID (e.g. AC-1042).",
     "parameters": {"type": "object", "properties": {"order_id": {"type": "string"}}, "required": ["order_id"], "additionalProperties": False}},
    {"type": "function", "name": "search_products", "description": "Search the product catalog by keyword or color.",
     "parameters": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"], "additionalProperties": False}},
    {"type": "function", "name": "process_refund", "description": "Issue a refund. Use ONLY when the customer explicitly asks.",
     "parameters": {"type": "object", "properties": {"order_id": {"type": "string"}, "reason": {"type": "string"}}, "required": ["order_id", "reason"], "additionalProperties": False}},
]
DISPATCH = {"get_order_status": get_order_status, "search_products": search_products, "process_refund": process_refund}

def run_agent(user_message: str) -> str:
    input_list = [{"role": "user", "content": user_message}]
    for _ in range(8):
        resp = client.responses.create(model="gpt-5.5", instructions=SYSTEM, tools=TOOLS, input=input_list)
        calls = [item for item in resp.output if item.type == "function_call"]
        if not calls:
            return resp.output_text
        input_list += resp.output
        for call in calls:                      # may be SEVERAL (parallel)
            out = DISPATCH[call.name](**json.loads(call.arguments))
            input_list.append({"type": "function_call_output", "call_id": call.call_id, "output": json.dumps(out)})
    return "Sorry, I couldn't finish that — let me get a human."

if __name__ == "__main__":
    print(run_agent("Where's order AC-1042, and do you have blue running shoes?"))
// orchestrate_openai.mjs  (Responses API)
import OpenAI from "openai";
import { getOrderStatus, searchProducts, processRefund } from "./acmeTools.mjs";

const client = new OpenAI();
const SYSTEM = "You are Acme Support. Use the tools for orders, product search, and refunds.";

const TOOLS = [
  { type: "function", name: "get_order_status", description: "Look up an order's status by its ID (e.g. AC-1042).",
    parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"], additionalProperties: false } },
  { type: "function", name: "search_products", description: "Search the product catalog by keyword or color.",
    parameters: { type: "object", properties: { query: { type: "string" } }, required: ["query"], additionalProperties: false } },
  { type: "function", name: "process_refund", description: "Issue a refund. Use ONLY when the customer explicitly asks.",
    parameters: { type: "object", properties: { order_id: { type: "string" }, reason: { type: "string" } }, required: ["order_id", "reason"], additionalProperties: false } },
];
const DISPATCH = { get_order_status: (a) => getOrderStatus(a.order_id),
                   search_products: (a) => searchProducts(a.query),
                   process_refund: (a) => processRefund(a.order_id, a.reason) };

async function runAgent(userMessage) {
  let input = [{ role: "user", content: userMessage }];
  for (let i = 0; i < 8; i++) {
    const resp = await client.responses.create({ model: "gpt-5.5", instructions: SYSTEM, tools: TOOLS, input });
    const calls = resp.output.filter(o => o.type === "function_call");
    if (calls.length === 0) return resp.output_text;
    input = input.concat(resp.output);
    for (const call of calls) {                  // may be several (parallel)
      const out = DISPATCH[call.name](JSON.parse(call.arguments));
      input.push({ type: "function_call_output", call_id: call.call_id, output: JSON.stringify(out) });
    }
  }
  return "Sorry, I couldn't finish that — let me get a human.";
}

console.log(await runAgent("Where's order AC-1042, and do you have blue running shoes?"));
What Just Happened? — Three Changes From M01

(1) The tool list grew from one to three. (2) The if block.name == "..." became DISPATCH[block.name] — a table that scales to 3 tools or 30 without touching the loop. (3) We swapped while True for a capped for _ in range(8). That's it. The parallel-call handling was already there in M01, because we always looped over every tool request. Multi-tool orchestration isn't a new mechanism — it's the same loop, fed a bigger toolbox.

Run It

Expected output (the two-part question triggers two parallel tools)
Your order AC-1042 has shipped via UPS and should arrive Tuesday. And yes —
we carry the blue Trailblazer Running Shoe for $89. Anything else?
✅ Checkpoint: Add logging inside the loop (print(block.name) / console.log(fc.name)). On the two-part question you should see both get_order_status and search_products fire in the same iteration — proof the model requested them in parallel. Ask a refund question and watch process_refund fire instead.

Controlling Tool Choice

By default the model decides freely (auto). Sometimes you want to steer it. All three SDKs expose the same four modes; here's what each means and when to use it.

ModeBehaviorUse when
autoModel chooses whether and which tools to call (default)Almost always — normal agent behavior
required / anyModel must call at least one toolYou know a tool is needed (e.g. a lookup flow)
specific toolForce one named toolDeterministic extraction / a fixed step
noneModel may not call any toolYou want a plain text turn only
The Spelling, Per SDK
  • Anthropic: tool_choice={"type": "auto" | "any" | "tool" | "none", "name": ...}. Add "disable_parallel_tool_use": true to force at most one tool per turn.
  • Gemini: a tool_config with function_calling_config.mode = "AUTO" | "ANY" | "NONE" (and optional allowed_function_names).
  • OpenAI: tool_choice="auto" | "required" | "none" or {"type": "function", "name": "..."}; set parallel_tool_calls=False to disable parallelism.
Write-Tools Need a Leash

Read tools (get_order_status, search_products) are safe to run automatically. process_refund is a write — it moves money. Right now the model can trigger it from a persuasive message alone. In production you'd (1) require human confirmation before executing a write tool, and (2) validate inputs server-side. That's exactly the human-in-the-loop and guardrail work coming in M06–M07. For now, just notice which of your tools are dangerous — that instinct is half the battle.

Three Ways, One Idea

ConceptAnthropicGoogle GeminiOpenAI (Responses)
Many toolslist of tool dicts in tools=many FunctionDeclarations in one Toollist of flat function tools in tools=
Parallel callsseveral tool_use blocks in one responseseveral entries in resp.function_callsseveral function_call items in output
Return all resultsone user msg, many tool_resultone tool Content, many functionResponsemany function_call_output items appended
Steer choicetool_choice (auto/any/tool/none)tool_config.function_calling_config.modetool_choice (auto/required/none/tool)
Disable paralleldisable_parallel_tool_use: true(model / prompt controlled)parallel_tool_calls=False
Why the Differences?

There barely are any — this is the module where the three SDKs converge most. All represent "many tools" as a list, all can return "many calls" in one turn, and all want "many results" back together. The dispatch-table pattern you wrote is portable across all three with only the field names swapped. If M01 taught you the loop, M03 taught you it doesn't grow when the toolbox does. That property is exactly why the M06–M07 frameworks can automate it.

Knowledge Check

Q1: With three tools registered, what decides which one the model calls?

AA separate router you configure in the SDK
BThe model, by reading each tool's name and description
CThe order the tools appear in the list
DRandom selection
Correct! Descriptions ARE the routing logic. Vague, overlapping descriptions cause wrong-tool bugs; precise ones with clear "use when" triggers route cleanly.

Q2: The model requests two tools in one turn. What must you do?

ARun only the first one; ignore the second
BReturn each result in its own separate turn
CRun both and return both results together in a single turn, then loop
DRestart the conversation
Correct! Run all requested tools and return all results in one turn. Splitting them across turns (or dropping one) makes the SDK error or the model stop calling in parallel.

Q3: How much did the M01 loop change to support three tools and parallel calls?

ACompletely rewritten with a new async framework
BIt needed a special parallel-execution library
CParallel calls required a different loop entirely
DBarely — a bigger tool list and a dispatch table; the loop already looped over all requests
Correct! Because M01's loop always iterated over every tool request, parallel calls "just worked." Orchestration is the same engine with a fuller toolbox.

Q4: Why does process_refund deserve extra caution that search_products doesn't?

AIt's a write action — it changes state (moves money) — so it should be gated behind confirmation
BIt's slower to run
CIt uses more tokens
DIt's not really different
Correct! Read tools are safe to auto-run; write tools change the world. Reversibility is the tell — hard-to-undo actions belong behind human confirmation (M06–M07).

Q5: You want to force the model to always call a tool this turn. Which tool_choice-style mode?

Anone
Brequired / any (or ANY mode in Gemini)
Cauto
DThere's no way to require a tool
Correct! required (OpenAI) / any (Anthropic) / ANY (Gemini) forces at least one tool call. auto lets the model decide; none forbids tools.

Module Summary

Key Takeaways

  • An agent is its tool surface. Add well-described tools and the model routes requests to them — descriptions are your router.
  • Parallel tool calls are built in: one turn can request several tools; run them all and return all results together.
  • The M01 loop didn't change — a dispatch table replaces if/else, and it scales to any number of tools.
  • Steer with tool_choice (auto / required / specific / none) and disable parallelism when you need serialized, gate-able calls.
  • Flag write-tools. A refund moves money — it belongs behind confirmation, which you'll build in the guardrails modules.

Next: M04 — Memory & Conversation

Every run so far started from a blank slate. Real support is a conversation — "where's my order?" then "actually, cancel it." In M04 you'll give Acme memory across turns, and meet the three SDKs' very different answers: resend-the-history (stateless), chat sessions, and server-stored state with previous_response_id.

← M02: Structured Output 🏠 Home M04: Memory →