Designing tools an LLM can actually use

Here is the single most useful sentence in this book.

The tool description is a prompt, not documentation.

Sit with that for a moment, because it inverts how most engineers approach the problem. When you write a docstring for a human colleague, you are writing a reference. The colleague reads it when they are confused, skips it when they are not, and can always open the source file if the docstring is wrong.

The model has none of those options. It cannot open your source file. It cannot ask you a follow-up question before deciding. It sees the tool name, the description, and the parameter schema — and then it must commit, in one shot, to a decision about whether to call your function and with what arguments.

That text is not a reference the model consults. It is the entire universe of information the model has about your code, injected into its context on every single request, competing for attention with the user’s message and the conversation history. It is a prompt. Write it like one.

Everything in this chapter is a consequence of that reframing.

Principle 1: documentation is the model’s only view

The tool name, the description, and every parameter description are all serialized into the request context. If a fact is not in that text, the model does not have it.

Let us make this concrete. Here is a tool as it is usually first written.

# BAD
def fetchpd(pid):
    """Retrieves product data.

    Args:
        pid: id

    Returns:
        dict of data
    """

Now put yourself in the model’s position. What is a pid? Is it an integer, a UUID, a SKU string? Does “product data” include the price, the stock level, both, neither? What comes back if the product does not exist — an empty dict, None, an exception? And is fetchpd the tool you want, or is fetchp (which you also registered) the right one?

The model will guess. It will usually guess plausibly, which is worse than guessing badly, because plausible-but-wrong failures are the ones that reach production.

Here is the same tool written as a prompt.

# GOOD
def get_product_details(product_id: str) -> dict:
    """Look up the catalogue record for one product by its exact product ID.

    Use this when you already have a product ID. If you only have a product
    name or a shopper's description, call search_products first to get the ID.

    Args:
        product_id: The exact catalogue identifier, formatted like "SKU-1001".
            Case-sensitive. Not the same as the supplier part number.

    Returns:
        A dict with these keys:
          product_name: display name, e.g. "Astro Zoom Trainers"
          brand:        brand name, e.g. "Cymbal Athletic"
          category:     e.g. "Children's Shoes"
          status:       one of "active", "discontinued", "suspended"
        Raises a not_found error, with recovery instructions, if the ID is unknown.
    """

Count what changed. The name now says what it does in words a person would use. The description says when to call it and when not to, which is the part that resolves ambiguity between overlapping tools. The parameter description gives the format, gives an example, and rules out the most likely confusion (supplier part numbers). The return description enumerates the keys and the allowed values of status, so the model can reason about the result before it ever sees one.

A checklist for this principle:

Name it descriptively. create_critical_bug_in_jira beats update_jira. Clear names also make your audit logs readable, which matters the first time someone asks what an agent did last Tuesday.
Describe every parameter, including its type, its format, and what the tool will actually do with it.
Keep parameter lists short. Long signatures confuse models the same way they confuse people, only faster.
Avoid jargon and internal shorthand. Returns the canonical CDS entity means nothing to a model that has never seen your codebase.
Add a targeted example where a distinction is genuinely subtle. One example in the description is cheaper than fine-tuning and works immediately.
Give defaults, and document them. Models use documented defaults correctly and surprisingly often. Undocumented defaults are invisible.

Saying it out loud. The one-liner I’d open with is that a tool description is a prompt, not documentation. A human colleague reads a docstring when they’re confused and opens the source when it’s wrong; the model can’t do either. It sees a name, a description, and a parameter schema, and then it has to commit in one shot. So fetchpd(pid) forces it to guess what a pid is and what comes back, and it’ll guess plausibly — which is worse than guessing badly, because plausible-but-wrong is what reaches production. A good description says when to call the tool and when not to, gives the parameter format with an example, and enumerates the return keys and their allowed values so the model can reason about a result before it’s ever seen one.

Principle 2: describe actions, not implementations

This one is about your system prompt, not your tool descriptions, and it is the mirror image of Principle 1.

Once the tools document themselves properly, the instructions should stop talking about tools at all. They should describe the goal and leave the mechanism alone.

# BAD system prompt
When the user reports a problem, call the create_bug tool. Pass the summary
in the `title` field and the description in the `body` field. The priority
field must be P0, P1 or P2. Then call notify_team with the returned bug ID.

# GOOD system prompt
When a user reports a problem, file a bug that captures what they described,
and make sure the on-call engineer knows about it.

The bad version has three separate failure modes baked in.

It duplicates the tool’s own documentation, so now you have two descriptions of the same schema that can drift apart. The day someone renames body to description, the tool schema updates automatically and the system prompt silently lies. It also hard-codes a workflow, which removes the model’s ability to do something sensible when reality does not match the script — say, when the bug already exists and the right move is to comment on it. And it names tools directly, which breaks the moment your tool set becomes dynamic, as it does with MCP.

The rules:

Say what, not how. “File a bug describing the issue,” not “use the create_bug tool.”
Do not restate tool documentation in the system prompt. One source of truth.
Do not dictate step sequences unless the sequence is a hard business requirement. Describe the objective and let the model plan.
Do document cross-tool side effects. If fetch_web_page writes the page to a scratch file and returns the path, the model must be told, or it will never think to read the file. That belongs in the tool’s own description, not in the system prompt.

Saying it out loud. Once your tools document themselves, your system prompt should stop mentioning tools at all — say what you want, not how to get it. “File a bug capturing what the user described and make sure on-call knows” beats spelling out which fields to pass to create_bug. Three things break when you spell it out: you’ve duplicated the schema so the two copies drift, and the day someone renames a field the tool spec updates itself while the prompt silently lies; you’ve hardcoded a workflow so the model can’t do the sensible thing when the bug already exists; and you’ve named tools directly, which breaks as soon as the tool set is dynamic, like with MCP. The one exception is cross-tool side effects — if a fetch tool writes to a scratch file and returns the path, that has to be documented, but it belongs in the tool’s own description.

Principle 3: publish tasks, not API calls

The laziest possible tool layer is a one-to-one wrapper over an existing REST API. It is also, reliably, the one that performs worst.

Enterprise APIs are designed for human developers who have read the docs, know the domain, and can iterate. They are broad on purpose: dozens of optional query parameters, expansion flags, pagination controls, field selectors. That breadth is a feature for a human integrator and a trap for a model that must choose everything at once, at runtime, from a description.

# BAD — a faithful wrapper over the underlying API
def orders_api(method: str, path: str, params: dict = None, body: dict = None) -> dict:
    """Call the Orders API.

    Args:
        method: HTTP method.
        path: API path.
        params: Query parameters.
        body: Request body.
    """

This is a tool that can do anything, which means the model has to invent the API surface from nothing. It will hallucinate paths. It will hallucinate parameter names. You have not built a tool; you have built a way for the model to guess at HTTP.

# GOOD — one tool per task the agent actually needs to perform
def find_orders_for_customer(email: str, since_days: int = 90) -> list[dict]:
    """List a customer's recent orders, newest first.

    Args:
        email: The customer's email address, exactly as they gave it.
        since_days: How far back to look. Defaults to 90 days.

    Returns:
        Up to 20 orders as {order_id, placed_on, status, total_eur}.
    """

def start_return_for_order(order_id: str, reason: str) -> dict:
    """Open a return request for one order and email the shopper a label.

    Only use this after confirming with the shopper which order they mean.

    Args:
        order_id: From find_orders_for_customer.
        reason: The shopper's own words, e.g. "wrong size".

    Returns:
        {return_id, label_url, expires_on}.
    """

Underneath, both of these call the same Orders API. The difference is that you did the hard thinking once, at design time, instead of asking the model to redo it on every request. You chose the endpoint, fixed the pagination, pinned the field selection, and named the thing after the job to be done.

The test to apply: can you describe this tool as something the user wants, in one sentence, without using the word “API”? If not, it is a wrapper, not a task.

Saying it out loud. The laziest tool layer is a one-to-one wrapper over your REST API, and it’s reliably the worst performer. Enterprise APIs are built for human developers who read the docs and iterate — dozens of optional params, expansion flags, pagination — and all that breadth is a trap for a model that has to choose everything at once from a description. A generic orders_api(method, path, params, body) tool isn’t a tool, it’s a way for the model to guess at HTTP, and it will hallucinate paths and parameter names. Instead publish tasks: find_orders_for_customer, start_return_for_order. Same backend underneath — you just did the hard thinking once at design time instead of asking the model to redo it every request. My test is whether you can describe the tool as something the user wants, in one sentence, without saying the word “API.”

Principle 4: make tools granular

Standard function design advice applies unchanged. One tool, one responsibility, clearly documented.

Granular tools are easier to describe, so the model chooses among them more consistently. They are easier to log, so your audit trail is meaningful. They are easier to permission, so you can let an agent read stock levels without letting it issue refunds.

For each tool you should be able to answer four questions in one sentence each: what does it do, when should it be called, does it have side effects, and what does it return. If any answer needs a paragraph, split the tool.

# BAD — a multi-tool with a hidden workflow
def handle_customer_issue(email: str, issue: str, action: str) -> dict:
    """Look up the customer, find their orders, decide what to do, and do it.

    Args:
        action: One of "refund", "replace", "escalate", "close", "investigate".
    """

Nobody can write a good description for that, including you. Its behavior depends on a branch the model cannot see, its side effects vary by argument, and half of its action values are irreversible while the other half are read-only.

The honest exception: when a fixed sequence of calls is genuinely always performed together, collapsing it into one tool saves round trips and reduces the chance the model stops halfway. A create_order_and_reserve_stock tool is defensible precisely because doing one without the other is a bug. If you do this, document the full set of effects explicitly — the model needs to know it just did two things.

Saying it out loud. One tool, one responsibility — the ordinary function design advice holds. Granular tools are easier to describe so the model picks among them consistently, easier to log so your audit trail means something, and easier to permission, so an agent can read stock levels without being able to issue refunds. My check is four questions, one sentence each: what does it do, when should it be called, does it have side effects, what does it return. If any answer needs a paragraph, split it. A handle_customer_issue tool with an action parameter of refund-or-replace-or-escalate is the failure case, because its side effects change per argument and half those values are irreversible while the other half are read-only. The honest exception is a sequence that’s always performed together — bundling create-order-and-reserve-stock is defensible precisely because doing one without the other is a bug.

Principle 5: design for concise output

This is the principle engineers most often skip, and it causes the most expensive class of failure.

The output of a tool does not just get read once. It gets appended to the conversation history and re-sent to the model on every subsequent turn. A tool that returns 30,000 tokens of JSON has not cost you 30,000 tokens. It has cost you 30,000 tokens multiplied by every remaining turn in the conversation, plus the reasoning quality you lost when the useful context got crowded out.

That is what people mean by context poisoning. The model does not get more capable when you give it more data; past a point it gets measurably worse at finding the relevant part.

# BAD
def query_sales(sql: str) -> list[dict]:
    """Run a SQL query against the sales warehouse and return all rows."""
    return db.execute(sql).fetchall()   # could be 400,000 rows

# GOOD
def query_sales(sql: str, preview_rows: int = 20) -> dict:
    """Run a read-only SQL query against the sales warehouse.

    The full result is written to a temporary table. Only a preview comes back
    to you. To work with the whole result, pass the returned table name to
    summarize_table or export_table -- do not try to re-query for all rows.

    Args:
        sql: A SELECT statement. Writes are rejected.
        preview_rows: How many sample rows to include inline. Max 50.

    Returns:
        {row_count, columns, preview, result_table} where result_table is the
        temporary table name holding the complete result.
    """

The pattern generalizes. Large results should be stored, not returned. Return a handle — a table name, a file path, an object key, an artifact ID — plus enough of a summary that the model can decide what to do next. Most agent frameworks give you somewhere to put this; if yours does not, a temp table or a scratch directory works fine.

Apply the same discipline to files, images, and binary blobs. Return the path and the metadata, not the bytes.

And truncate defensively at the framework level, not just per tool. You will eventually register a tool that returns more than you expected, and a global cap on tool-result size turns a disaster into a mild annoyance. Make the truncation message actionable: not [truncated] but [truncated 18,000 chars — add a filter or lower the limit and call again].

Saying it out loud. This is the one engineers skip and it’s the most expensive. A tool’s output doesn’t get read once — it’s appended to the history and re-sent on every later turn, so a tool that returns 30,000 tokens of JSON hasn’t cost you 30,000 tokens, it’s cost you that times every remaining turn, plus the reasoning quality you lost when the useful context got crowded out. That’s context poisoning: past a point, more data makes the model measurably worse at finding the relevant part. The fix is a pattern — large results get stored, not returned. Hand back a handle plus a summary: a table name, a file path, a row count, twenty preview rows. Same for files and images, return the path and metadata, not the bytes. And cap tool-result size globally, with a truncation message that tells the model what to do — not [truncated] but [truncated 18,000 chars, add a filter and call again].

Principle 6: use validation effectively

Schemas do two jobs at once, and it is worth naming them separately because they pull in the same direction.

At design time, the schema is more documentation. An enum of allowed warehouse codes tells the model exactly which values exist, far more reliably than a sentence saying “one of AMS, SIN”. A required list tells it what it cannot omit. A pattern or a format tells it the shape of an ID.

At runtime, the schema is a guard. Models produce malformed arguments — wrong types, missing fields, invented enum members — at a low but nonzero rate that never reaches zero no matter how good the model gets. Validation is what stops a malformed call from reaching your database.

# BAD — permissive schema, no runtime check
def schedule_delivery(date: str, window: str, priority: int) -> dict:
    """Schedule a delivery."""

Nothing here constrains anything. date could be "next Tuesday", window could be "morning-ish", priority could be 9999.

# GOOD
def schedule_delivery(
    date: str,                                   # pattern: ^\d{4}-\d{2}-\d{2}$
    window: Literal["08-12", "12-16", "16-20"],
    priority: Literal[1, 2, 3] = 3,
) -> dict:
    """Book a delivery slot for a confirmed order.

    Args:
        date: Delivery date in YYYY-MM-DD format. Must be at least 2 days
            from today; earlier dates are rejected with an explanation.
        window: The four-hour delivery window, in local warehouse time.
        priority: 1 = same-day premium, 2 = express, 3 = standard. Defaults to 3.
    """

Where the type system cannot express a rule — “at least two days out” — put the rule in the description and enforce it in code. The description is what usually prevents the mistake; the check is what catches it when prevention fails.

Also validate on the way out. An output schema lets you catch the day your backend starts returning null where it used to return a number, before that null becomes a confident sentence in front of a customer.

Saying it out loud. A schema is doing two jobs at once. At design time it’s more documentation — an enum of warehouse codes tells the model which values exist far more reliably than a sentence saying “one of AMS, SIN,” and a pattern tells it the shape of an ID. At runtime it’s a guard, because models emit malformed arguments — wrong types, missing fields, invented enum members — at a low but nonzero rate that never hits zero no matter how good the model gets. So constrain every field you can constrain. Where the type system can’t express the rule, like “the date must be at least two days out,” put it in the description and enforce it in code: the description usually prevents the mistake, the check catches it when prevention fails. And validate on the way out too, so you catch the day your backend starts returning null before that null becomes a confident sentence in front of a customer.

Principle 7: error messages are your last prompt

This deserves its own section because it is so consistently neglected.

When a tool fails, the failure message goes back into the model’s context. That means an error message is another chance to steer behavior — arguably the most important one, because the model is at that moment stuck and looking for direction.

# BAD
raise ValueError("404")

# BAD
return {"error": "Product not found"}

The second is better than the first and still tells the model nothing about what to do. It will typically retry the identical call, get the identical error, and then either loop or give up.

# GOOD
raise ToolError(
    f"No product with ID {product_id!r}. This usually means the ID came from "
    "the shopper rather than from search_products. Ask the shopper for the "
    "product name, call search_products to get a valid ID, then try again.",
    code="not_found",
)

A good tool error answers three questions: what went wrong, why it probably happened, and what to do next. That last part is the one that changes outcomes.

Some patterns worth stealing:

Rate limited: "Rate limit hit. Wait 15 seconds before calling this tool again. Do not call it in a loop."
Bad enum value: "'LHR' is not a warehouse code. Valid codes are AMS, SIN, NYC. Pick one of those."
Ambiguous input: "Three customers match 'j.smith'. Ask the user which one: j.smith@corp.com, jsmith@corp.com, john.smith@corp.com."
Permission denied: "This account cannot issue refunds over 500 EUR. Tell the user the request needs a supervisor and stop."
Genuine internal bug: "Backend returned malformed data. This is a system fault, not an argument problem. Do not retry; tell the user the service is unavailable."

That last one matters more than it looks. Telling the model explicitly not to retry is how you prevent an agent from burning ten turns and a lot of money re-running a call that will never succeed.

Saying it out loud. An error message is your last prompt. When a tool fails, that text goes straight into the model’s context at the exact moment it’s stuck and looking for direction — so it’s arguably your highest-leverage sentence. ValueError("404") tells it nothing; even “Product not found” tells it nothing actionable, so it retries the identical call, gets the identical error, and either loops or gives up. A good tool error answers three things: what went wrong, why it probably happened, and what to do next — “that ID probably came from the shopper rather than from search_products, so ask for the product name, call search_products, then try again.” The one people forget is telling the model explicitly not to retry on a genuine backend fault, and that single sentence is what stops an agent burning ten turns and real money re-running a call that will never succeed.

Putting it together

Before you register any tool, run it past these questions.

Could a competent stranger, reading only the name and description, call this correctly on the first try?
Does the description say when not to use this tool?
Is this a task the user cares about, or an endpoint your backend happens to expose?
Can you state its side effects in one sentence?
What is the largest thing it can possibly return, and what happens then?
Does the schema constrain every field it can constrain?
For each way it can fail, does the error message tell the model what to do next?

Seven questions, a few minutes each. They will save you more debugging time than any other habit in this book.

Saying it out loud. Before I register any tool I run seven questions. Could a competent stranger call this right on the first try from just the name and description? Does the description say when not to use it? Is this a task the user cares about, or an endpoint my backend happens to expose? Can I state the side effects in one sentence? What’s the biggest thing it can return, and what happens then? Does the schema constrain everything it can? And for each failure mode, does the error tell the model what to do next? That’s a few minutes per tool, and it saves more debugging time than any other habit I know.

What you should be able to do now

Rewrite a vague tool docstring into one that functions as a prompt — with usage boundaries, parameter formats, examples, and an enumerated return shape.
Refactor a generic API wrapper into a small set of task-shaped tools, and explain why the wrapper was going to fail.
Identify a tool that will poison the context window, and redesign it to return a handle plus a summary instead of the full payload.
Write tool error messages that state the cause and prescribe the model’s next action, including when to stop retrying.
Audit an existing agent’s tool set against the seven-question checklist and produce a prioritized fix list.

Production Agent Engineering — A Practical Guide