Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Function calling: how the model actually invokes your code

There is a persistent misconception that when a model “calls a tool,” something inside the model executes your function. It does not. The model has no network access, no filesystem, no Python interpreter.

What actually happens is narrower and much easier to reason about once you see it: the model emits text in a structured format that says I would like you to run this function with these arguments, your code notices that, runs the function, and puts the answer back into the conversation as a new message. The model then continues generating with that answer in its context.

That is the whole mechanism. Everything else — parallel calls, retries, agent loops — is bookkeeping on top of that one exchange. This chapter walks the full round trip and then catalogues what goes wrong.

We will use the Anthropic Python SDK (anthropic, version 0.120.x at the time of writing) for concrete code, because its tool-use shape is clean and explicit. The same structure applies to Gemini and OpenAI with different field names; the differences are noted where they matter.

Step 1: declaring the tool

A tool declaration has exactly three parts: a name, a description, and a JSON Schema for the input.

WEATHER_TOOL = {
    "name": "get_current_weather",
    "description": (
        "Get the current weather conditions at a specific location. "
        "Use this for 'right now' questions. For forecasts, use get_forecast instead."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "city": {
                "type": "string",
                "description": "City name, optionally with country, e.g. 'Amsterdam, NL'.",
            },
            "units": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"],
                "description": "Temperature unit. Defaults to celsius.",
                "default": "celsius",
            },
        },
        "required": ["city"],
        "additionalProperties": False,
    },
}

input_schema is ordinary JSON Schema, and the model is given it verbatim. Every description in there is doing prompt work — go back to Chapter 1 if that is not yet obvious.

Two details that catch people out.

additionalProperties: False is worth setting. Without it, nothing stops the model from inventing an extra field, and an invented field that your dispatcher silently ignores is a bug you will find weeks later.

default is documentation only. The model reads it and often honors it, but nothing enforces it — if the model omits units, your function’s own Python default is what actually applies. Keep the two in sync.

Naming differs by provider: Anthropic uses input_schema, OpenAI uses parameters nested under a function object, Gemini uses parameters inside a FunctionDeclaration. The content is the same JSON Schema in all three.

Saying it out loud. A tool declaration is three things: a name, a description, and a JSON Schema for the input. That’s it, and the model gets the schema verbatim, so every description field in there is doing prompt work. Two details bite people. Set additionalProperties: false, because otherwise nothing stops the model inventing an extra field, and an invented field your dispatcher silently ignores is a bug you find weeks later. And default in the schema is documentation only — the model reads it and usually honors it, but nothing enforces it, so if it omits the field your Python default is what actually applies. Keep those two in sync. The provider differences are just naming: Anthropic calls it input_schema, OpenAI and Gemini call it parameters, and it’s the same JSON Schema underneath.

Step 2: the model emits a call

You send the tools alongside the messages.

import anthropic

client = anthropic.Anthropic()

resp = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    tools=[WEATHER_TOOL],
    messages=[{"role": "user", "content": "What's the weather in Amsterdam?"}],
)

The response comes back with stop_reason == "tool_use" and a content list that contains one or more blocks. There is usually a text block (the model narrating its intent) followed by one or more tool_use blocks:

resp.stop_reason
# 'tool_use'

resp.content
# [TextBlock(type='text', text="I'll look that up."),
#  ToolUseBlock(type='tool_use', id='toolu_01A9...', name='get_current_weather',
#               input={'city': 'Amsterdam, NL', 'units': 'celsius'})]

Three fields matter on a ToolUseBlock.

id is the correlation handle. When you send the result back you must echo this exact ID, or the API rejects the request. With parallel calls it is the only thing linking each result to its call.

name is the tool the model chose. Do not assume it is one you registered — see the failure catalogue below.

input is a parsed dict. The SDK has already done the JSON parsing for you; what it has not done is validate it against your schema.

The key thing to internalize: stop_reason == "tool_use" means the turn is not over. The model is waiting. Your job is to answer.

Saying it out loud. When the model wants a tool, the response comes back with stop_reason of tool_use and a content list holding one or more tool-use blocks, usually after a bit of text narrating intent. Three fields matter: the id, which is the correlation handle you must echo back exactly or the API rejects your request — with parallel calls it’s the only thing linking a result to its call; the name, which you should never assume is a tool you actually registered; and the input, which the SDK has parsed into a dict but has emphatically not validated against your schema. The thing to internalize is that tool_use means the turn is not over. The model is sitting there waiting, and your job is to answer.

Step 3: dispatch

Dispatch is a lookup and a call, wrapped in enough defense to survive bad input.

def get_current_weather(city: str, units: str = "celsius") -> dict:
    # In a real system this calls a weather API.
    data = {"Amsterdam, NL": {"c": 14, "sky": "overcast"}}
    row = data.get(city)
    if row is None:
        raise LookupError(
            f"No weather station matches {city!r}. Try a major city name with a "
            "country code, e.g. 'Amsterdam, NL'."
        )
    temp = row["c"] if units == "celsius" else round(row["c"] * 9 / 5 + 32)
    return {"city": city, "temperature": temp, "units": units, "sky": row["sky"]}


HANDLERS = {"get_current_weather": get_current_weather}

import json
import jsonschema

SCHEMAS = {t["name"]: t["input_schema"] for t in [WEATHER_TOOL]}

def dispatch(block) -> tuple[str, bool]:
    """Return (result_text, is_error) for one tool_use block."""
    fn = HANDLERS.get(block.name)
    if fn is None:
        return (f"Unknown tool {block.name!r}. Available tools: "
                f"{', '.join(sorted(HANDLERS))}. Call one of those."), True
    try:
        jsonschema.validate(block.input, SCHEMAS[block.name])
    except jsonschema.ValidationError as e:
        return (f"Invalid arguments for {block.name}: {e.message}. "
                "Check the schema and call again with corrected arguments."), True
    try:
        return json.dumps(fn(**block.input)), False
    except LookupError as e:
        return str(e), True
    except Exception as e:
        return (f"{block.name} failed internally ({type(e).__name__}). "
                "This is not an argument problem; do not retry."), True

Note the return type. Every branch produces text destined for the model, and a flag saying whether it was a failure. That is the entire contract between your code and the model, and keeping it uniform is what makes the loop simple.

Saying it out loud. Dispatch is a lookup and a call wrapped in enough defense to survive bad input. Look the handler up in a map rather than indexing it, so an unknown name becomes an error listing the real tools instead of a KeyError. Validate the arguments against the schema before you execute anything. Then run the function, and turn every possible outcome — success, bad arguments, a business-level miss, an internal crash — into the same shape: a string of text plus a boolean saying whether it failed. That uniform return type is the entire contract between your code and the model, and keeping it uniform is exactly why the agent loop stays about ten lines long.

Step 4: returning the result

Results go back as a user-role message containing tool_result blocks. This surprises people — the result did not come from the user — but that is the wire format: the assistant asked, the “user” side of the conversation answers.

tool_results = []
for block in resp.content:
    if block.type != "tool_use":
        continue
    text, is_error = dispatch(block)
    tool_results.append({
        "type": "tool_result",
        "tool_use_id": block.id,
        "content": text,
        "is_error": is_error,
    })

messages = [
    {"role": "user", "content": "What's the weather in Amsterdam?"},
    {"role": "assistant", "content": resp.content},   # echo the model's turn back verbatim
    {"role": "user", "content": tool_results},
]

Three rules that are not optional.

Echo the assistant turn back unchanged. resp.content includes the tool_use blocks; the API needs them present to match your results against.

Every tool_use gets exactly one tool_result. Miss one and the request is rejected. Send two for the same ID and it is rejected.

Put all results for one assistant turn in a single user message. Not one message per result.

is_error: true is a small thing with a real effect: it marks the block as a failure so the model treats it as something to recover from rather than as data.

Saying it out loud. Results go back as a user-role message containing tool_result blocks, which surprises people because the result obviously didn’t come from the user — but that’s the wire format: the assistant asked, the user side answers. Three rules aren’t optional. Echo the assistant turn back unchanged, because the API needs the original tool-use blocks present to match against. Every tool-use gets exactly one tool-result — miss one and the request is rejected, send two for the same ID and it’s rejected. And all results for one assistant turn go in a single user message, not one message each. The small detail with real effect is the is_error flag: it marks the block as a failure so the model treats it as something to recover from rather than as data.

Step 5: the loop

Put those steps in a while and you have an agent.

def run(user_message: str, tools: list[dict], max_turns: int = 10) -> str:
    messages = [{"role": "user", "content": user_message}]

    for turn in range(max_turns):
        resp = client.messages.create(
            model="claude-sonnet-4-5",
            max_tokens=2048,
            tools=tools,
            messages=messages,
        )
        messages.append({"role": "assistant", "content": resp.content})

        if resp.stop_reason != "tool_use":
            return "".join(b.text for b in resp.content if b.type == "text")

        results = []
        for block in resp.content:
            if block.type == "tool_use":
                text, is_error = dispatch(block)
                results.append({
                    "type": "tool_result",
                    "tool_use_id": block.id,
                    "content": text,
                    "is_error": is_error,
                })
        messages.append({"role": "user", "content": results})

    return "I couldn't finish this within the allowed number of steps."

Read the termination condition carefully, because it is the part people get wrong. The loop ends when the model stops asking for tools — not when a tool succeeds, not when some goal predicate is satisfied. The model decides it is done.

The max_turns guard is not optional. Without it, a model that keeps re-calling a failing tool will run until your budget or your patience gives out. Ten to twenty is a reasonable ceiling for most agents; if you legitimately need more, that is a signal your tools are too granular for the task, not that the ceiling is too low.

Saying it out loud. Put those steps in a while-loop and you have an agent. The part people get wrong is the termination condition: the loop ends when the model stops asking for tools — not when a tool succeeds, not when some goal predicate is satisfied. The model decides it’s done. And the turn cap is not optional, because without it a model that keeps re-calling a failing tool runs until your budget or your patience gives out. Ten to twenty is a reasonable ceiling for most agents, and if you genuinely need more than that, read it as a signal your tools are too granular for the task rather than that the ceiling is too low.

Parallel tool calls

Modern models routinely emit several tool_use blocks in one turn when the calls are independent. “Compare the weather in Amsterdam and Singapore” produces two blocks, not two turns.

The loop above already handles this — it iterates over every tool_use block — but it executes them serially. If your tools do I/O, run them concurrently:

import concurrent.futures

def dispatch_all(blocks):
    with concurrent.futures.ThreadPoolExecutor(max_workers=8) as pool:
        futures = {pool.submit(dispatch, b): b for b in blocks}
        out = []
        for fut in concurrent.futures.as_completed(futures):
            b = futures[fut]
            text, is_error = fut.result()
            out.append({"type": "tool_result", "tool_use_id": b.id,
                        "content": text, "is_error": is_error})
    return out

Order does not matter — the tool_use_id does the correlating — so as_completed is safe.

Two cautions. Parallel calls to a mutating tool can race in ways you did not design for; if a tool writes, consider serializing it. And if you need deterministic ordering for a specific reason, you can disable parallelism per request with tool_choice={"type": "auto", "disable_parallel_tool_use": True}. Reach for that rarely; parallelism is usually a large latency win.

Saying it out loud. Modern models emit several tool-use blocks in one turn when the calls are independent — “compare the weather in Amsterdam and Singapore” is two blocks in one turn, not two turns. The naive loop already handles that, it just runs them serially, so if your tools do I/O, put them on a thread pool; ordering doesn’t matter because the tool-use ID does the correlating. Two cautions. Parallel calls into a mutating tool can race in ways you never designed for, so serialize writes. And you can disable parallelism per request if you need deterministic ordering, but reach for that rarely — parallelism is usually a big latency win.

What breaks, and how to defend

Everything below happens in production. Not occasionally — routinely, at rates that are small per call and inevitable across millions of calls.

Hallucinated tool names

The model calls chekc_stock, or calls a tool you removed last week, or invents get_user_email because it seems like something that ought to exist.

Defend: never index your handler map directly. Look it up, and on a miss return an error listing the real tool names. The model recovers from that almost every time. Also log it — a spike in unknown-tool calls usually means two of your descriptions overlap.

Saying it out loud. Models will call tools that don’t exist — a typo like chekc_stock, a tool you deleted last week, or something like get_user_email that just seems like it ought to exist. So never index your handler map directly; look it up, and on a miss hand back an error that lists the real tool names, because the model recovers from that almost every time. The part people skip is logging it: a spike in unknown-tool calls is usually telling you that two of your tool descriptions overlap and the model is trying to split the difference.

Wrong types and malformed arguments

{"sku": 1001} where you wanted "SKU-1001". {"limit": "ten"}. A date as "next Tuesday". Nested objects flattened into strings.

Defend: validate against the schema before dispatch, always, and return the validator’s message plus an instruction to try again. Do not coerce silently. Coercion hides the problem from you and teaches the model nothing; a clear rejection gets corrected on the next turn. The one exception worth making is stripping whitespace, which is harmless.

Saying it out loud. You’ll get a SKU as an integer where you wanted a string, a limit as the word “ten,” a date as “next Tuesday.” The defense is to validate against the schema before dispatch, every time, and hand back the validator’s own message plus an instruction to try again. The tempting thing — and the wrong thing — is to coerce silently. Coercion hides the problem from you and teaches the model nothing, whereas a clear rejection gets corrected on the very next turn. Stripping whitespace is about the only coercion I’d allow.

Missing required parameters

The model calls check_stock with no sku because it does not have one and hopes the tool will figure it out.

Defend: the required list catches it, and the error message should tell the model where to get the missing value: “sku is required. Call find_sku with the product name first.” This turns a dead end into a two-step plan.

Non-idempotent double-fires

The nastiest one. The model calls charge_card, your network hiccups, the model does not see a result, and it calls charge_card again. Or a retry wrapper you forgot about fires it twice. Or the model, seeing an ambiguous result, decides to “make sure” and repeats it.

Defend with idempotency keys. Any tool with a side effect should accept a caller-supplied key and deduplicate on it.

def charge_card(order_id: str, amount_eur: float, idempotency_key: str) -> dict:
    """Charge the saved card for one order. Safe to retry with the same key.

    Args:
        order_id: The order being paid for.
        amount_eur: Amount in euros.
        idempotency_key: A unique string for this charge attempt. If you call
            this tool twice with the same key, only one charge happens and the
            second call returns the original result.
    """
    if existing := CHARGES.get(idempotency_key):
        return existing | {"deduplicated": True}
    ...

You can also let the framework generate the key by hashing the tool name plus arguments within a conversation, which requires nothing of the model. That is often the better design precisely because it does not depend on the model getting anything right.

Layer a second defense on top for anything genuinely irreversible: a confirmation step, a spending cap, or a human approval gate. Chapter 1’s advice about telling the model not to retry belongs here too.

Saying it out loud. The nastiest failure is the non-idempotent double-fire: the model calls charge_card, the network hiccups so it never sees a result, and it calls charge_card again. Or it sees an ambiguous result and repeats it to “make sure.” The fix is idempotency keys — any side-effecting tool takes a caller-supplied key and deduplicates on it, so a second call returns the original result instead of charging twice. Better still, have the framework generate that key by hashing the tool name and arguments within the conversation, because then the safety property doesn’t depend on the model getting anything right. And for anything genuinely irreversible, layer a second defense on top: a spending cap or a human approval gate.

Runaway loops

The model calls the same tool with the same arguments five turns running because the error message did not tell it to stop.

Defend: cap turns, and detect repetition. If the same (name, arguments) pair appears three times, return an error that says so explicitly: “You have called this tool with identical arguments three times and received the same failure. Stop calling it and explain the problem to the user.”

Saying it out loud. Runaway loops usually aren’t the model being stupid, they’re the error message failing to tell it to stop — so it calls the same tool with the same arguments five turns running. Two defenses, and you want both. Cap the turns, and detect repetition: if the same tool-name-plus-arguments pair shows up three times, return an error that says exactly that — you’ve called this three times and gotten the same failure, stop and explain the problem to the user. That’s the difference between a bounded failure and an agent quietly burning your budget while looking busy.

Oversized results

A tool returns the entire table. Covered in Chapter 1, but enforce it here too, in the dispatcher, where it applies to every tool including the ones you did not write.

Tool call arrives, but the schema changed

You deployed a rename mid-conversation. The model, working from history, keeps using the old name.

Defend: keep deprecated names as aliases for a while, and have the alias return the result plus a note: “old_name is deprecated; use new_name in future calls.”

The error contract

Pulling the thread together: your dispatcher should guarantee a small, uniform contract regardless of what the tool does.

Every dispatch returns text and an error flag. It never raises into the loop. It never returns None. It never returns more than a fixed number of characters. Every error message names the cause and prescribes a next action. Every success is JSON the model can parse.

That uniformity is what lets the loop stay ten lines long, and it is what you will build properly in the next chapter.

Saying it out loud. Pulling it together, my dispatcher guarantees a small uniform contract no matter what the underlying tool does. Every dispatch returns text plus an error flag. It never raises into the loop, never returns None, and never returns more than a fixed number of characters. Every error names the cause and prescribes the next action; every success is JSON the model can parse. That uniformity is the whole reason the loop can stay ten lines long — all the messy, tool-specific variation gets absorbed at exactly one boundary.

What you should be able to do now

  • Declare a tool as JSON Schema, send it with a request, and read the tool_use blocks out of the response.
  • Write a dispatcher that validates arguments before execution and returns a uniform (text, is_error) result for every outcome.
  • Construct a correctly-shaped tool_result message — echoing the assistant turn, matching every tool_use_id exactly once, in a single user message.
  • Write the multi-turn loop with a turn cap and the right termination condition, and execute parallel tool calls concurrently.
  • Name at least five ways function calling fails in production and implement the defense for each, including idempotency keys for side-effecting tools.

Further reading