Control flow: loops, graphs, and state machines

Every agent framework you will ever use is an answer to one question: what runs next?

Strip away the decorators, the observability integrations, and the marketing, and what remains is a scheduler. Something has to decide which piece of code executes, with what inputs, holding what state, and under what limits. In Part 1 your answer was a for loop with a step cap. This chapter is about the answers you need when that stops being enough, and about understanding them well enough that you could have written them yourself — which, in the next chapter, you will.

The organising idea is that there are exactly three control-flow shapes in practice, and they sit on a spectrum of how much of the decision-making you have handed to the model.

Shape	Who decides what runs next	Good at	Bad at
Loop	The model, every turn	Open-ended goals, unknown step counts	Parallelism, resumption, auditability
Graph	You, with model-driven branches	Structured work with dynamic edges	Genuinely unbounded exploration
State machine	You, entirely	Compliance, cost control, determinism	Anything you cannot enumerate

Most production systems are graphs with loops inside some of the nodes. That sentence is the whole chapter, but it will not mean much until we take the pieces apart.

The bare loop, and exactly where it stops

Here is the Part 1 loop again, compressed:

state = initial(mission)
for step in range(max_steps):
    decision = model(state)
    if decision.done:
        return decision.answer
    state = state + observe(execute(decision.actions))

Three properties make this work as well as it does.

It is model-driven: the transition function is the language model, so you do not have to enumerate the paths. It is stateful by accumulation: everything the agent knows lives in one growing message list, so no information is ever accidentally dropped. And it is trivially correct: there is one thread of control and one place where it can exit.

Those same three properties are also the failure modes.

Model-driven means undiagnosable. When the loop does something strange, the transition function is a black box you cannot step through. You have a trace of what it decided, never a reason.

Accumulate-everything means the context degrades. Step nine sees every byte of steps one through eight, including the forty-kilobyte JSON blob from the tool call that turned out to be irrelevant. Cost per step rises monotonically, and so does the chance the model latches onto something stale. If a run takes \( n \) steps and each step adds roughly \( c \) tokens of context, total tokens processed scale as \( O(n^2 c) \) — not \( O(nc) \), because every step resends everything before it. That quadratic is why a run that takes twice as many steps can cost four times as much.

One thread of control means no parallelism. Modern APIs let the model request several tools in one turn, and you should absolutely execute those concurrently. But that is parallelism within a step. The loop cannot run two different lines of reasoning at once, because there is one message list and one model call driving it.

And then there are the things the shape simply has no vocabulary for.

You cannot express “this step is allowed to fail; if it does, do that instead” — a try block around the whole loop is not the same thing, because it loses the work already done. You cannot express “pause here until a human approves,” because the loop’s entire state is Python locals in a process that will not be alive on Tuesday. You cannot express “these four lookups are independent; do them together.” You cannot resume from the middle after a crash. You cannot say “spend at most three dollars and forty seconds on this.”

Every one of those is a real production requirement, and all of them are asking for the same thing: the control flow needs to be data, not code.

Saying it out loud. So the simple agent loop is just “ask the model, run the tool, append the result, repeat” — and it works surprisingly well, right up until you need something it has no words for. It can’t pause for a human approval, because all its state is Python variables in a process that’s about to be redeployed. It can’t run two branches at once, because there’s one message list and one model call driving it. And it gets expensive in a way people don’t expect: every step resends everything before it, so the token cost grows with the square of the number of steps, not linearly — double the steps and you’ve roughly quadrupled the bill. The fix for all of it is the same idea: make the control flow data you can inspect and save, instead of the position of the program counter.

Making control flow into data: the graph model

The move that unlocks all of it is small. Stop expressing “what runs next” as the position of the program counter, and start expressing it as a value you can inspect, log, checkpoint, and reason about.

A graph does this. Four concepts, and that is genuinely all of it.

Nodes

A node is a unit of work with a name. It takes the current state and returns a partial update — not a new state, just the fields it changed.

def classify(state):
    return {"category": "billing"}

Returning a partial update rather than a whole state is not stylistic. It is what makes parallelism safe: two nodes running at once produce two small dictionaries you can merge, instead of two whole states you would have to reconcile.

A node can be anything. A pure function. A single model call. A database query. An entire ReAct loop, complete with its own step cap — this is the “graphs with loops inside the nodes” point, and it is how almost every real system is built.

Saying it out loud. A node is just a named piece of work that takes the current state and hands back only the fields it changed. The reason it returns a partial update rather than a whole new state is parallelism. If two nodes run at the same time and each returns a complete state object, you now have to reconcile two full snapshots and decide which one wins — whereas two small dictionaries you can merge field by field. So the tradeoff is a tiny bit of ceremony in every node in exchange for concurrency that doesn’t silently lose work.

Edges

An edge says which node runs after which. A plain edge is static, known when you write the graph:

graph.edge("load", "classify")

Static edges are the boring, valuable part. They encode what you already know about your problem, and everything they encode is something the model cannot get wrong. If a policy check must always run before a refund, that is an edge, not a sentence in a prompt.

Saying it out loud. An edge is the part of the flow you already know, written down as structure instead of as a sentence in a prompt. If the policy check absolutely has to run before the refund, make that an edge — because an edge can’t be talked out of running, and a prompt can. That’s the real dividing line: anything you can enumerate should be an edge, and only the genuinely judgment-shaped decisions should be left to the model. The failure mode you’re avoiding is the one where a persuasive customer message convinces the agent to skip a compliance step.

Conditional edges

A conditional edge is a function from state to the next node or nodes.

def route(state):
    return "refund_path" if state.category == "billing" else "tech_path"

This is where dynamism enters, and it is worth being precise about what “dynamic” means here. The router is ordinary code. It may consult the model’s output — usually the classification a previous node produced — but the branching itself is deterministic and testable. You get model-driven behaviour without giving up a control flow you can unit-test.

The distinction matters for debugging. When a request went down the wrong path, you can ask “did the router misfire, or did the classifier hand it a bad label?” — and answer it. In a bare loop those two failures are the same event.

Saying it out loud. A conditional edge is just a Python function that looks at the state and says which node runs next. The key thing is that it’s ordinary code — it may read a label the model produced, but the branching itself is deterministic and you can unit-test it. That buys you a debugging property you really want: when a request goes down the wrong path, you can tell whether the router misfired or the classifier handed it a bad label. In a bare loop those two failures are the same event, and you have no way to separate them.

Shared state with reducers

Nodes need to communicate, and the graph gives them exactly one channel: a shared state object.

The obvious implementation — a dictionary that every node calls .update() on — breaks the moment two nodes run in parallel. Both return {"findings": [...]}, one overwrites the other, and you silently lose half your work. This is the single most common bug in hand-rolled parallel workflows.

The fix is to declare, per field, how concurrent writes combine. That function is a reducer.

findings: Annotated[list[str], operator.add]     # concurrent writes concatenate
category: str                                    # single writer; collision is a bug

Now merge is total: for reduced fields it folds, and for non-reduced fields a collision from two parallel nodes is an error you raise loudly rather than a data loss you never notice. Reducers are also where you put deduplication, capped-length windows, and “keep the highest-confidence value” logic — anywhere you would otherwise be tempted to write merge logic inside a node.

Saying it out loud. A reducer is a per-field rule for what happens when two nodes write to the same key at the same time. This sounds like plumbing, but it’s the single most common bug in hand-rolled parallel workflows: two branches both return a findings list, one dict update clobbers the other, and you quietly lose half your results with no error anywhere. So you declare it up front — findings concatenate, category has one writer and a collision is a bug you raise loudly. The tradeoff is that you have to think about merge semantics before you have the bug, instead of after.

The scheduler: supersteps

Given nodes, edges, and a merge function, execution is a loop over supersteps.

Start with a frontier: the set of (node, payload) pairs ready to run.
Run everything in the frontier concurrently, all against the same frozen snapshot of state.
Collect the partial updates and merge them, in a deterministic order.
Evaluate the outgoing edges of every node that just ran to compute the next frontier.
Repeat until the frontier is empty or a budget fires.

This is the Bulk Synchronous Parallel model, borrowed from graph processing systems like Pregel, and LangGraph names the debt explicitly (https://docs.langchain.com/oss/python/langgraph/graph-api). It buys you something specific and valuable: parallel execution with deterministic results.

Because every node in a superstep sees the same input snapshot, nothing depends on which thread happened to finish first. Because merges happen at a barrier in a fixed order, the merged state is reproducible. You get concurrency without the class of bug where the same inputs produce different answers on Tuesday.

The cost is real too. A superstep is only as fast as its slowest node, so one straggler holds up the barrier. And you cannot have a node inside a superstep read what a sibling just wrote — if you need that, they belong in different supersteps, which is a design constraint you will hit and should recognise when you do.

Saying it out loud. A superstep is a round: run everything that’s ready right now, all against the same frozen snapshot of the state, then merge the results at a barrier before starting the next round. That’s the Pregel model borrowed from graph processing, and what it buys you is parallel execution that’s still deterministic — nothing depends on which thread happened to finish first, so the same inputs give the same answer on Tuesday. The price is two things. A superstep is only as fast as its slowest node, so one straggler stalls the whole barrier, and a node can’t read what a sibling just wrote — if it needs to, they belong in different supersteps.

Fan-out with per-item payloads

One more primitive and the model is complete.

Sometimes you do not know until runtime how many parallel branches you need — one summariser per document, one worker per sub-task, one validator per extracted field. Static edges cannot express that, because you would have to name every target when you build the graph.

The answer is a conditional edge that returns a list of dispatches, each carrying its own payload. LangGraph calls this Send:

from langgraph.types import Send

def fan_out(state):
    return [Send("summarise", {"doc": d}) for d in state["docs"]]

Send targets a node with an input that is not the shared state, which is the important part. The worker sees only what you handed it. That is context isolation as a control-flow primitive, and it is the mechanism behind every map-reduce and orchestrator-worker pattern you will build.

Saying it out loud. Sometimes you don’t know until runtime how many parallel branches you need — one summariser per document, and you don’t know the document count when you’re writing the graph. So instead of naming targets ahead of time, you have a router return a list of dispatches, each carrying its own payload. The important detail is that the worker sees only the payload you handed it, not the whole shared state. That’s context isolation as a control-flow primitive, and it’s the mechanism behind every map-reduce and orchestrator-worker pattern you’ll build.

Put the five pieces together — nodes, edges, conditional edges, reduced state, supersteps — and you have expressed chaining, routing, parallelization, and dynamic fan-out with no special cases. That is why the graph model won.

Saying it out loud. The reason the graph model won is that five small primitives — nodes, edges, conditional edges, reduced state, and superstep scheduling — cover chaining, routing, parallelism, and dynamic fan-out with no special cases. Once “what runs next” is a value rather than a program counter, you can log it, checkpoint it, replay it, and test the routing separately from the reasoning. And the graph doesn’t force you to choose between structure and autonomy: the deterministic skeleton is edges, and any single node can be a full agent loop with its own step cap. That’s why most real production systems are described as graphs with loops inside some of the nodes.

Durable execution: surviving the gap between steps

Here is a fact that surprises people the first time it bites.

Agent runs are long. Not “long” as in a slow HTTP request — long as in minutes to hours, and for anything with a human approval in it, days. Meanwhile the machine your agent runs on has a mean time between deployments measured in hours.

If your run’s entire state is in memory, every deploy, every OOM kill, every autoscaler decision, and every transient network partition destroys work that cost real money to produce. At a 1% chance of interruption per step, a 40-step run has about a \( 1 - 0.99^{40} \approx 33% \) chance of dying before it finishes. That is not an edge case. That is a third of your traffic.

Durable execution is the property that a run’s progress is persisted as it goes, so it can be resumed from the last completed point rather than restarted.

The graph model makes this almost free, which is the second reason it won. After every superstep there is a barrier where the state is a single well-defined value and the frontier is a small list of node names. Write that pair to a database, keyed by a thread ID, and you have a checkpoint. On restart, load the last checkpoint and carry on.

In LangGraph this is a compile-time argument and a config key:

from langgraph.checkpoint.memory import InMemorySaver

graph = builder.compile(checkpointer=InMemorySaver())
config = {"configurable": {"thread_id": "ticket-901"}}
result = graph.invoke({"ticket_id": "T-901"}, config)

InMemorySaver is for development only — it holds checkpoints in RAM and loses them on restart, which is precisely the thing you are trying to prevent. Production uses PostgresSaver or SqliteSaver (https://docs.langchain.com/oss/python/langgraph/persistence).

Checkpointing is not free, so there is a knob. LangGraph exposes a durability parameter on invoke with three settings, from fastest to safest: "exit" writes a checkpoint only when the run finishes, "async" writes checkpoints in the background while the next step proceeds, and "sync" blocks each step until its checkpoint is committed (https://docs.langchain.com/oss/python/langgraph/durable-execution). Pick by what a lost step costs you. A research agent that reads public web pages can use "exit"; anything that spends money or touches a customer should be "sync".

There is a design consequence that is easy to miss and expensive to learn. A resumable run will re-execute the step that was in flight when it died. So every node that touches the outside world must be idempotent, or protected by an idempotency key — the same discipline Part 2 applied to mutating tools, now applied at the node level. If your charge_card node is not idempotent, durable execution will eventually charge someone twice.

The mature version of this idea is a dedicated durable-execution engine: Temporal, Restate, or DBOS, which give you the same guarantee for arbitrary code with retries, timers, and versioning built in (https://docs.temporal.io/evaluate/understanding-temporal). If your agent orchestrates work that takes days or coordinates with systems that have their own failure semantics, that is where you will end up. For most agents, checkpointed supersteps are enough.

Saying it out loud. Durable execution just means the run’s progress is written down as it goes, so a crash resumes from the last completed point instead of starting over. You need it because agent runs are long — minutes to hours, days if a human approval is in there — and the machine underneath redeploys far more often than that. Put numbers on it: at a one percent chance of interruption per step, a forty-step run has about a thirty-three percent chance of dying before it finishes. That’s a third of your traffic, not an edge case. The catch you have to name is that resuming re-runs the step that was in flight when it died, so every node touching the outside world needs to be idempotent — otherwise durable execution will eventually charge somebody twice.

Human-in-the-loop is a control-flow feature

Once state is checkpointed and the frontier is data, one more thing becomes possible that was structurally impossible in the bare loop.

You can stop.

A run that hits a node requiring approval writes its checkpoint, returns to the caller with “waiting on approval,” and exits the process. No thread parked on a queue, no connection held open, no timeout to tune. Days later, a human clicks approve, and you resume from the checkpoint with the decision injected into state.

LangGraph implements this with an interrupt() call inside a node and resumption via Command(resume=value) (https://docs.langchain.com/oss/python/langgraph/human-in-the-loop). Whatever the API, the underlying requirement is the same one: the pause is only cheap if the state was already durable.

This is why the whitepaper’s human-in-the-loop pattern is listed as a design pattern but is really an infrastructure capability. You cannot prompt your way to it. Either your orchestration layer can suspend and resume, or it cannot.

Saying it out loud. The point here is that a human approval step isn’t a prompting pattern, it’s an infrastructure capability. When the agent hits a step needing sign-off, it writes its checkpoint, tells the caller it’s waiting, and exits the process entirely — no thread parked on a queue, no connection held open, no timeout to tune. Three days later somebody clicks approve and you resume from the checkpoint with their decision injected into state. And that’s only cheap because the state was already durable; either your orchestration layer can suspend and resume or it can’t, and no amount of prompting gets you there.

When a state machine beats an agent

Now the other direction.

A state machine is a graph with no model in the routing. You enumerate the states, you enumerate the transitions, and the language model — if it appears at all — is a node that classifies or generates, never one that decides where to go next.

The instinct in 2026 is that this is the primitive, boring option and agents are the sophisticated one. That instinct costs companies a lot of money.

Use a state machine when any of these is true.

You can enumerate the paths. If a whiteboard session produces a complete diagram of what should happen, the model’s freedom to choose adds variance and nothing else. Refund processing, KYC onboarding, and tier-one triage are almost always in this category, and people build agents for all three.

A wrong path is a compliance event. When “the model decided to skip the eligibility check” is a sentence that ends in a regulator’s inbox, do not let the model decide. Encode the check as an edge. An edge cannot be talked out of running by a persuasive customer message, and prompt injection has no purchase on a line of Python.

Latency or cost has a hard ceiling. Every routing decision the model makes is a round trip: hundreds of milliseconds and a token bill. A five-node state machine with one classifier call costs one model call. The same flow as an agent loop costs five to nine, and you cannot bound it tightly in advance.

You need reproducibility. Same input, same path, every time. Deterministic routing gives you that; a model in the router does not, at any temperature.

Use an agent loop when the path genuinely cannot be enumerated: the number of steps depends on what is discovered along the way, the tool sequence varies per request, or the task is open-ended enough that any state diagram you draw will be missing a branch you have not imagined. Debugging an unfamiliar production failure is a real agent task. Processing a refund is not.

The honest answer for most systems is a mix, and the graph model is what lets you express the mix in one artifact. A deterministic skeleton — validate, classify, route, act, verify, respond — with a model in the classifier and a full agent loop inside exactly one node, the one doing open-ended investigation. The rest is edges.

There is a heuristic worth carrying: give the model the smallest decision that still requires judgment, and encode everything else. Not “here is a goal and eleven tools, good luck,” and not a hardcoded script that cannot handle a case you did not anticipate. The middle is where the reliable systems are.

Saying it out loud. The honest answer is that if you can draw the whole flow on a whiteboard, you don’t want an agent — you want a state machine with a model inside one or two of the nodes. Refund processing, KYC onboarding, tier-one triage: people build agents for all three, and the model’s freedom to choose adds variance and nothing else. There are four tests I’d apply — can you enumerate the paths, is a wrong path a compliance event, is there a hard latency or cost ceiling, and do you need the same input to take the same path every time. On cost alone it’s stark: a five-node state machine with one classifier is one model call, and the same flow as an agent loop is five to nine that you can’t bound in advance. The heuristic I’d give is to hand the model the smallest decision that still genuinely requires judgment, and encode everything else.

What this means for the next chapter

You now have the full vocabulary: nodes, edges, conditional edges, reduced shared state, supersteps, dispatch with per-item payloads, checkpoints, budgets.

You could go and use LangGraph right now, and it would work. But you would be using a scheduler you have never seen the inside of, and when it does something you did not expect — a state key silently overwritten, a superstep that ran a node twice, a fan-out that did not fan out — you would be reading GitHub issues instead of reading code.

So in the next chapter you write the scheduler. It is about two hundred lines. Then we express the same workflow in LangGraph and you will recognise every piece.

Saying it out loud. You could pick up LangGraph today and be productive, but you’d be trusting a scheduler you’ve never looked inside. The reason to write your own in a couple hundred lines is that when it eventually does something surprising — a state key silently overwritten, a node that ran twice, a fan-out that didn’t fan out — you want to be reading code, not GitHub issues. That’s the tradeoff between using a framework and understanding one, and interviewers can tell in about thirty seconds which side you’re on.

What you should be able to do now

Name the three control-flow shapes — loop, graph, state machine — and place a given requirement on that spectrum with an argument for why.
Explain why a bare agent loop cannot express parallel branches, resumable pauses, or per-step failure policy, and identify which of those a given production requirement needs.
Describe the graph model precisely: nodes returning partial updates, static and conditional edges, reducers on shared state, and superstep scheduling — and say why the superstep barrier is what makes parallel execution deterministic.
Explain what a checkpoint contains, why durable execution requires idempotent nodes, and choose sensibly between checkpoint-on-exit and checkpoint-per-step for a given workload.
Argue the case for a state machine over an agent for a specific business process, using enumerability, compliance exposure, cost ceiling, and reproducibility as the tests.

Production Agent Engineering — A Practical Guide