The 4 concepts of function calling

Tap any concept to jump to its 5-card cluster.

• 1What Is Tool Use?The model proposes, your code executes
• 2Defining ToolsName, description, JSON Schema
• 3The Tool Use Loopsend → detect → execute → return
• 4Error HandlingErrors are data, not crashes

The model proposes, your code executes

Function calling lets you hand the model a set of tools — functions it can ask you to run. When the model decides it needs one, it stops writing text and returns a structured tool call: "I'd like to call get_weather with {city: "Tokyo"}."

The key idea: the model never runs your code. It only asks. Your application reads the request, runs the real function, and sends the result back. That separation of "deciding" (model) from "doing" (your code) is what makes tool use safe for production.

The brilliant clerk in a windowless room

BEFORE: Picture a super-smart assistant locked in a windowless room — no phone, no internet, can't even see the clock. Every answer about the outside world is a guess from what they already memorized.

PAIN: Ask "What's the weather in Tokyo right now?" and they have to make something up or shrug. Their intelligence is trapped — brilliant, but blind to anything real or current.

MAPPING: Function calling hands them a set of labeled buttons — get_weather(), calculate(), get_time() — each wired to the real world. Now they press get_weather with "Tokyo", read what comes back, and answer accurately. You decide what each button actually does.

The round trip

1. You send tools + messageThe request includes a tools array describing every function the model may request.
2. The model returns a tool callInstead of text, it emits {name, arguments} — a structured request, not an execution.
3. Your code executesYou validate the arguments and run the real function on your server. You are the gatekeeper.
4. You feed the result backSend it in a role: "tool" message referencing the call's id, and the model writes the final answer.

Pseudocode

# the model ASKS — it never runs code
reply = model.chat(messages, tools)

IF reply has a tool_call:
  name = tool_call.name       # "get_weather"
  args = tool_call.arguments  # {city:"Tokyo"}

  # YOUR code does the real work
  result = run_function(name, args)

  # hand the answer back to the model
  send_back(role="tool",
            id=tool_call.id,
            content=result)

Misconceptions

A tool is name + description + schema

A tool definition is a JSON object with three parts: a name (how your code identifies the call), a description (plain English the model reads to decide when the tool fits), and parameters — a JSON Schema listing the arguments, their types, and which are required.

Of the three, the description matters most. The model picks tools by understanding descriptions, not by matching names. "How hot is it in Tokyo?" fires a tool described as "Get current weather for a city" even though the word "weather" never appears.

A menu the model reads

BEFORE: Think of tool definitions as API documentation — but instead of a human developer reading the docs, the model reads them to figure out what each function does.

PAIN: A menu item labeled just "Food" tells a diner nothing — they can't order what they want. A tool named search with the description "Search" is the same: the web? a database? files? The model has nothing to go on and picks wrong.

MAPPING: A good description is a rich menu entry — what the tool does, what data source it hits, what the output looks like, and when to use it over others. Write it well and the model orders the right dish on the first try.

The four parts of a definition

1. nameA snake_case identifier your dispatcher uses to route the call to the right function.
2. descriptionThe most important field — what it does, its data source, output format, and when to use it.
3. parameters (JSON Schema)Each argument gets a type and its own description; add examples like "e.g. 'Tokyo'".
4. requiredThe list of arguments the model MUST supply. Everything else is optional.

Writing a strong description

FOR every tool description, state:

  1. WHAT it does
     → "Get current weather"

  2. WHICH data source
     → "for a city, live API"

  3. WHAT output shape
     → "temp C, condition, humidity"

  4. WHEN to use it vs others
     → "use for weather questions"

# smaller local models lean on this
# harder than a frontier model does

Spend more time on the description than on the implementation — it's the #1 lever for selection accuracy.

Misconceptions

A while loop, not one request

The tool use loop is what turns a chatbot into an agent. You send a message; each time the model replies you check a field called finish_reason. If it's "tool_calls", the model is waiting on a tool — you execute, append the result, and call again. If it's "stop", the model is done and you return its text.

So one user question might trigger 2, 3, or 10 API round-trips as the model gathers what it needs. Every round-trip adds to the same conversation, so the model always sees what it has already tried. The loop exits only on "stop".

A chef sending tickets to the kitchen

BEFORE: A simple API call is like ordering one dish and getting one plate — one message in, one reply out, done.

PAIN: But a real task often needs several steps: check stock, prep, taste, adjust. One shot can't cover a request that only reveals its next step after the last one finishes.

MAPPING: The loop is a chef firing tickets to the kitchen. Each ticket (tool call) comes back with a result; the chef reads it and decides whether to fire another or plate the dish. The finish_reason is the chef calling "service!" — only then does the plate (final answer) go out.

One iteration of the loop

1. Send messagesPost the full conversation so far to the model and read the response.
2. Check finish_reasonIs it "tool_calls" (act) or "stop" (return)? This field is the only reliable signal.
3. Execute + appendRun the tool, then append BOTH the assistant message and the role: "tool" result.
4. Loop or exitCall again with the grown history — repeat until "stop". Add a max_iterations cap for safety.

Pseudocode

messages = [ user_message ]

WHILE true:
  reply = model.chat(messages, tools)

  IF reply.finish_reason == "tool_calls":
    FOR call IN reply.tool_calls:
      result = run(call.name, call.args)
      messages.append(tool_result(call.id,
                                  result))
    # keep the assistant turn too, then loop
  ELSE:                # finish_reason == "stop"
    RETURN reply.text  # the model is done

Misconceptions

Errors are data, not crashes

Tools will fail — and the worst move is letting the failure crash the loop. Instead: catch the error, describe it in plain language, and return that description as the tool's result. The model reads it and can inform the user, try another approach, or gracefully give up.

This flips normal error handling. The "caller" here isn't your code — it's a model that can reason about failures. So you report errors as data ({"error": "City not found: Tokyoo"}) and let the model decide what to do next.

The assistant who reports back, not quits

BEFORE: Imagine an assistant you send to fetch a file. If the cabinet is locked, you'd want them to come back and say "it's locked" — not vanish for the rest of the day.

PAIN: An unhandled exception is exactly that vanishing: the tool throws, the loop dies, and the user gets an ugly stack trace instead of an answer. The whole agent falls over because one lookup failed.

MAPPING: Returning the error as the tool result is the assistant reporting back: "Couldn't find 'Tokyoo'." The model hears that and responds gracefully — "Did you mean Tokyo?" The loop keeps running; only your unhandled exception can kill it.

Four failure modes to handle

1. Tool timeoutCap each call (e.g. 10s). On overrun, return a timeout message so the model can tell the user to retry.
2. Tool failureReturn a human-readable error, not a raw stack trace. The model needs to understand, not debug your code.
3. Invalid argumentsValidate before executing. If the model sends "city": 123, return a message explaining the expected format.
4. Non-existent toolIf the model names an unknown tool, return {"error": "Unknown tool"} instead of crashing with a KeyError.

Structured errors beat generic ones

ON tool failure, RETURN data:

  {
    isError: true,
    category: "auth_failure",
    isRetryable: true,
    context: "token expired"
  }

# anti-pattern: "Operation failed"
#   the model can't decide to retry,
#   try an alternative, or escalate.

# also: [] means "found nothing",
#   isError:true means "couldn't check"

And never let the model build arbitrary code or shell commands — tools are YOUR fixed functions.

Misconceptions

Test the core insight

One question per concept. Tap to reveal the answer.