Agents vs workflows: what actually makes something an agent

Someone on your team is going to say “let’s make that an agent” this quarter. Before you agree, you need a definition sharp enough to disagree with.

This chapter gives you one. It also gives you the far more valuable skill of recognizing when the honest answer is “that should be a for-loop and three API calls.”

Three generations, and what each one could not do

It helps to see where agents came from, because each generation exists to fix a specific limitation of the one before it.

Predictive AI was the state of the art for a long time, and it is still the right tool for enormous numbers of problems. You train a model on labelled examples and it produces a number or a label: this transaction is fraudulent with probability 0.87, this image contains a cat, this customer will churn. The interface is narrow and the output space is fixed. You know in advance the shape of everything it can say. That constraint is a feature — it makes the system testable in the ordinary way, with a held-out set and an accuracy number.

What it cannot do is handle a request it was not trained for. A churn model cannot answer “why is this customer unhappy.”

Generative AI widened the output space to “any text.” You give a language model a prompt and it produces a completion. Suddenly one model handles summarization, translation, classification, drafting, and explanation, without being retrained for each.

But it is still fundamentally passive. It answers the question in front of it, once, using only what is inside its weights plus whatever you pasted into the prompt. Ask it about something that happened after its training data was collected and it will either say it does not know or — worse — invent something plausible. Ask it to actually do the thing rather than describe it, and it produces a description of doing it.

Every call is independent. There is no notion of “I need more information before I answer,” because there is no mechanism by which it could go get any.

Agentic AI closes that gap by putting the model in a loop with the ability to act.

Saying it out loud. Each generation exists because the previous one couldn’t do something. Predictive models give you a fixed output space — a probability, a label — so they’re easy to test but they can only answer the question you trained them on. Generative models blew the output space open to “any text,” but they’re still passive: one call, one answer, no way to go get a fact they don’t have. Agents add the loop and the tools, so the system can decide it needs more information and go find it. The tradeoff you’re accepting when you move up that ladder is testability — you go from assert accuracy > 0.9 on a held-out set to needing a whole eval harness.

The working definition

An agent is a language model running in a loop, with access to tools, pursuing a goal it was given without being told the steps.

Four words in that sentence are load-bearing, and if you drop any one of them you no longer have an agent.

Goal. The input is an outcome, not an instruction. “Find out why order 12345 is late and email the customer” is a goal. “Call find_order('12345'), then call get_shipping_status(), then call send_email()” is a script. The distinguishing test: does the caller know, at the moment of calling, exactly which operations will run? If yes, you have a workflow with an LLM inside it.

Autonomy. The system decides its own next step. Nobody approves each hop. This is the property that makes agents useful and also the property that makes them dangerous, and most of the engineering in this book exists to give you that usefulness with a bounded amount of that danger.

Tools. The model can reach outside its own weights — read a database, call an API, run code, ask a human. Without tools it is a very articulate prisoner. It can plan a trip perfectly and book nothing.

Iteration. The output of an action feeds back in as input to the next decision. This is where the actual intelligence of the system lives. A single model call is a guess. A loop that observes the consequence of each guess and adjusts is a problem-solving process.

Saying it out loud. My working definition is: an agent is a language model running in a loop, with tools, chasing a goal nobody gave it the steps for. Four things have to be true — it gets a goal instead of a script, it picks its own next step, it can reach outside its own weights, and each result feeds back into the next decision. The cleanest test I use is: at the moment you call it, do you already know which operations will run? If yes, that’s a workflow with a model in it, no matter what the repo is called. And the autonomy bullet is the one that cuts both ways — it’s exactly why agents are useful and exactly why they’re dangerous.

The agentic problem-solving process

The Google whitepaper this book builds on breaks the loop into five phases, and it is a genuinely useful decomposition because each phase is a different place where things go wrong.

1. Get the mission. Something arrives: a user types a request, a webhook fires, a scheduled job wakes up. The critical property is that the mission states a desired end state, not a procedure.

2. Scan the scene. Before reasoning, the orchestration layer gathers what is knowable. What did the user actually ask? What is in the conversation so far? What does long-term memory hold about this user — did they ask something similar last week, did they already tell us their preferences? Which tools are available right now? This is the assembly step, and it is where most quality problems originate. A model reasoning over an incomplete scene will reason confidently and wrongly.

3. Think it through. The model reasons over the mission and the scene and produces a plan or a next step. Note that this is not necessarily one thought. For a hard problem it is a chain: to answer this I need X, to get X I need Y, so first I will do Y.

4. Take action. The orchestration layer executes exactly one concrete step — invoking a tool, running a function, querying a store. Not the whole plan. One step. This matters because the world may not match the plan, and you want to find that out after one action rather than seven.

5. Observe and iterate. The result comes back and is folded into the context. Then you go back to step 3 with more information than you had before.

That cycle — think, act, observe — repeats until the model decides it is done or until you stop it.

Saying it out loud. The loop has five phases and it’s worth naming them separately because each one fails differently. You get a mission, you scan the scene, you think, you take one action, you observe and go around again. The part people skip is “scan” — gathering the conversation, the memory, the available tools — and that’s where most quality problems actually start, because a model reasoning over an incomplete scene reasons confidently and wrongly. The other detail that matters is that you execute one step, not the whole plan, because the world may not match the plan and you’d rather find that out after one action than after seven.

A concrete trace

A customer asks: “Where is my order #12345?”

The model thinks: to answer this I need the order record, and from the record the tracking number, and from the tracking number the carrier’s current status. Three unknowns, resolvable in sequence.

It acts: find_order("12345"). It observes: a record containing tracking number ZYX987.

It thinks: I have the tracking number, now I need live status. It acts: get_shipping_status("ZYX987"). It observes: "Out for delivery".

It thinks: I have everything. It acts by producing the final message: “Your order #12345 is out for delivery.”

Nothing in that trace was hardcoded. The model chose to call find_order first, extracted ZYX987 from a JSON blob without being told which field to look at, and decided on its own that two lookups were enough. Change the question to “where are my last three orders” and the same code produces a different, correct trajectory.

That flexibility is what you are buying. Now let us talk about the price.

Saying it out loud. Here’s the trace that makes it concrete. Somebody asks where order 12345 is; the model reasons that it needs the order record, then the tracking number out of that record, then the carrier’s status. So it calls find_order, gets back a JSON blob, pulls the tracking number out without anyone telling it which field to look at, calls get_shipping_status, and decides two lookups is enough. Nothing in that path was hardcoded — and that’s the point, because if you change the question to “where are my last three orders,” the exact same code produces a different, correct trajectory. That flexibility is the whole thing you’re paying for.

Why reaching for an agent is usually wrong

Here is the uncomfortable engineering reality. The customer-support example above — the one everybody uses to introduce agents, this book included — is a workflow.

You know the steps. They are always the same steps. find_order then get_shipping_status then format a message. You could write that as forty lines of Python with a template string at the end, and it would be faster, cheaper, fully deterministic, testable with assert output == expected, and debuggable with a print statement.

The agent version costs you:

Latency. Every hop is a model round trip. A three-step agent trajectory is three sequential inference calls, each of which is hundreds of milliseconds to several seconds. Your forty-line workflow does the same work in one round trip or none.

Money. Every hop resends the entire accumulated context. Step 5 of a trajectory is paying to re-read steps 1 through 4. Token cost in a loop grows superlinearly with trajectory length, and this surprises people badly the first time they see the bill.

Non-determinism. The same input can produce different trajectories. Sometimes the model calls the tools in a different order. Sometimes it calls one twice. Sometimes it decides it has enough after one call and answers with half the facts. You cannot unit-test this with equality assertions, which is why Part 5 of this book is entirely about evaluation.

A new attack surface. If any content the agent reads can influence what it does next — and by construction it can — then anyone who can put text in front of your agent has partial control of it. Prompt injection is not a theoretical concern; it is the default state of a naive agent with tools.

Operational weight. Traces, evals, step caps, cost budgets, tool permissions, human-in-the-loop gates. None of that is optional in production, and none of it is needed for a for-loop.

You pay all of that for one thing: the ability to handle inputs you did not enumerate in advance.

If your inputs are enumerable, you are paying for nothing.

Saying it out loud. The honest answer is that most things people want to make agents shouldn’t be agents. Even the customer-support example everyone uses to teach this is a workflow — the steps are always the same, so forty lines of Python would be faster, cheaper, deterministic, and debuggable with a print statement. What you’re paying for the agent version is latency, since every hop is a model round trip; money, since every hop resends the whole accumulated context so token cost grows superlinearly with trajectory length; non-determinism, so you can’t unit-test it with equality assertions; and a real attack surface, because anything the agent reads can steer it. You buy all of that for exactly one capability: handling inputs you couldn’t enumerate in advance. If your inputs are enumerable, you’re paying for nothing.

The decision table

Work down this table honestly. The right-hand column is not a consolation prize — it is usually the better system.

A rough heuristic that has served me well: if you can draw the flowchart, build the flowchart. Agents earn their cost exactly when the flowchart would need a node for every possible situation and you cannot enumerate the situations.

Saying it out loud. My heuristic is: if you can draw the flowchart, build the flowchart. When I’m deciding, I’m really asking a handful of questions — can I enumerate the steps, does step N depend on the actual content of step N-1, what happens if it does the wrong thing, and what’s my latency and cost budget. Sub-second latency or fractions-of-a-cent per request rules out an agent basically on its own, and so does anything irreversible or regulated without a human gate. Agents earn their cost in exactly one place: when the flowchart would need a node for every possible situation and you can’t list the situations.

The hybrid is usually the answer

The framing of “agent vs workflow” is a little false, and the best production systems in 2026 are neither pure. They are deterministic workflows with an agentic step in the middle, or agents whose risky operations are wrapped in deterministic gates.

Some patterns worth stealing:

LLM as a workflow step. Your pipeline is fixed, but one node is “classify this ticket into one of nine categories” or “extract the shipping address from this email.” No loop, no tools, just a model call with structured output. This is generative AI inside a workflow and it is enormously effective.

Agent with a deterministic shell. The agent reasons and plans freely, but before any mutating tool executes, a plain Python policy check runs — over the spending limit, refuse; touching a record outside this customer’s account, refuse. The model’s judgment is an input to the decision, never the decision itself.

Router to specialists. One cheap model call classifies the incoming request. The eighty percent of requests that match known shapes go to hardcoded workflows. The long tail goes to the agent. You get workflow economics on the bulk of your traffic and agent flexibility where you actually need it.

That last one is, in my experience, the single highest-leverage architecture for a first production system. It lets you ship the known cases fast and reliably while the agent handles the residue, and it gives you a natural place to measure how much residue there actually is.

Saying it out loud. In practice the agent-versus-workflow framing is a bit false, because the good production systems are hybrids. The one I’d reach for first is a router: a cheap classifier call up front, the eighty percent of traffic that matches known shapes goes to hardcoded workflows, and the long tail goes to the agent. You get workflow economics on the bulk of your volume and agent flexibility only where you actually need it, plus a free measurement of how big the long tail really is. The other pattern is an agent inside a deterministic shell — the model can plan whatever it wants, but before any mutating tool runs, a plain Python policy check decides. The model’s judgment is an input to the decision, never the decision.

A checklist before you commit

Before you write a line of agent code, answer these out loud:

1. What is the goal, stated as an outcome? If you cannot state it without listing steps, it is a workflow.
2. What is the worst thing this system could do? If the answer involves money leaving, data being deleted, or a message going to a customer, you need a human gate or a hard-coded guardrail before launch.
3. How will you know it is working? “It looked good when I tried it” is not an answer. If you cannot name a metric, you cannot ship this.
4. What is your per-request cost ceiling, and what is your step cap? Pick numbers now, while you are calm.
5. What is the fallback when the agent fails? Every agent needs an exit that hands the problem to a human or a simpler system.

If you have good answers to all five, build the agent. If you have good answers to none of them, you have just saved yourself a quarter.

Saying it out loud. Before I write any agent code I want five answers. What’s the goal as an outcome — if I can’t say it without listing steps, it’s a workflow. What’s the worst thing this thing could do, and if that involves money moving or a message reaching a customer, there’s a human gate before launch. How will I know it’s working, named as a metric, because “it looked good when I tried it” isn’t one. What’s my per-request cost ceiling and my step cap — pick those numbers while you’re calm, not during an incident. And what’s the fallback when it fails, because every agent needs an exit that hands the problem to a human.

What you should be able to do now

• State a precise, defensible definition of an agent and apply it to classify any system someone shows you — including catching systems that are called agents but are workflows with a model in them.
• Walk through the five-phase agentic loop (mission, scan, think, act, observe) and identify which phase a given failure belongs to.
• Use the decision table to argue for or against building an agent for a specific problem, with the cost side of the argument made concrete.
• Design a hybrid architecture — router to specialists, or agent inside a deterministic shell — instead of defaulting to one extreme.
• Run the five-question pre-commit checklist on a proposal and identify which answers are missing.

Further reading

• ReAct: Synergizing Reasoning and Acting in Language Models — the paper the loop in Chapter 4 comes from: https://arxiv.org/abs/2210.03629
• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models: https://arxiv.org/abs/2201.11903
• Anthropic, “Building effective agents” — the clearest short argument for preferring workflows: https://www.anthropic.com/engineering/building-effective-agents
• Simon Willison’s ongoing series on prompt injection, which is the security reason to keep agent autonomy narrow: https://simonwillison.net/series/prompt-injection/