Part 6 — Production

You have an agent.

It has a loop with hard limits, a tool belt drawn from real MCP servers, a memory system, a workflow engine underneath it, and an eval harness that tells you whether a change made it better or worse. On your laptop, it works.

Now someone asks the question that ends the fun part.

“Can we give this to customers?”

That question is not about the agent. It is about everything wrapped around the agent: who gets paged when it breaks, what happens when a tool it can call is also a way to email money out of the company, how you know today’s version is better than yesterday’s, and what you do at 2 a.m. when it is worse.

The gap between “it works on my machine” and “I trust it with customers” is the subject of this part. The Google Prototype to Production whitepaper puts a number on that gap that matches what most teams find: roughly four-fifths of the total effort on a shipped agent goes not into the agent’s intelligence but into the validation, infrastructure, and safety work that makes it dependable. That ratio sounds discouraging until you notice what it implies — the last mile is where the actual engineering is, and it is learnable.

Why agents make this harder than normal software

You already know how to ship a web service. Three properties of agents break the habits you brought with you.

The execution path is assembled at runtime. A traditional service does the same thing every time. An agent picks its own trajectory — which tools, in what order, how many times — so “the code path” is not a thing you can enumerate, test exhaustively, or reason about from a diff. Versioning has to cover the prompt, the tool schemas, and the model, because any one of them changes behaviour without changing a line of application logic.

The tests are nondeterministic. Your CI has always answered a yes/no question. Now it answers a statistical one: is this version’s task success rate meaningfully worse than the baseline’s, given that both are sampled from noisy distributions? A gate that fails on any single regression will be red constantly and will get disabled within a month. A gate that is too loose lets real regressions through. Getting this right is Chapter 2’s whole job.

The system can act. A retrieval-augmented chatbot that gets tricked produces embarrassing text. An agent that gets tricked issues a refund, sends an email, or writes to a database. Prompt injection stops being a content-safety concern and becomes an authorization concern, and the defense lives in code that runs before the tool does — not in a paragraph of your system prompt.

None of these are exotic. They are all tractable. But they need different machinery than the deploy pipeline you already have, and bolting that machinery on after launch is how teams end up with the incident that shows up in a postmortem template.

What this part covers

Chapter 1 — People, process, and the road to production. The organizational half everyone skips. Who owns an agent in production, what changes about your development process, the stages from prototype to general availability, and the one principle everything else hangs from: evaluation-gated deployment, where an agent ships only when it clears a measured bar.

Chapter 2 — CI/CD when your tests are nondeterministic. Building the pipeline. What runs on every pull request versus what runs nightly, how to wire the eval harness in as a blocking gate, statistical gates that survive contact with a flaky sampler, versioning prompts and tools and models as one atomic artifact, and promotion of a tested build from staging into production. With a complete GitHub Actions workflow you can copy.

Chapter 3 — Shipping without breaking things. Rollout strategies for agents specifically. Shadow mode, canary with quality gates rather than only latency and error gates, staged rollout, feature flags, and rollback that takes seconds. The core insight: a new agent version can be perfectly healthy on every infrastructure metric and materially worse at its job, and only a quality gate catches that.

Chapter 4 — Security for agents that can act. The threat model when the system takes real actions. Direct and indirect prompt injection, tool abuse, data exfiltration, excessive agency, supply chain risk from third-party MCP servers, and secrets. Then defense in layers, and a runnable tool-authorization layer that enforces role checks, argument policy, rate limits, human gates, and output redaction — with an audit trail.

Chapter 5 — Operating an agent in production. Observe, act, evolve. The operational levers you actually pull, managing performance and scale, cost control in detail (budgets, model routing, caching, and cost per successful task rather than cost per call), risk management, and the feedback loop that turns a production failure into tomorrow’s eval case.

Chapter 6 — A2A: agents talking to agents. Why inter-agent standardization matters once you have more than one team building agents. The A2A protocol — agent cards, the task lifecycle, message flow — from concept to running code, plus how A2A and MCP compose, and whether you need a registry.

Chapter 7 — Mini-project 10: containerize and deploy your agent. The build chapter. A multi-stage Dockerfile with a non-root user and a healthcheck, configuration and secrets done properly, a FastAPI wrapper with liveness and readiness endpoints, docker-compose for local testing, a deploy to a serverless target, and the smoke test that proves the thing that is running is the thing you built.

What you will have built

Three artifacts, all runnable.

A CI/CD pipeline that lints, unit tests, runs the eval suite as a statistical gate, builds one immutable artifact, and promotes that same artifact through staging to production.

A tool authorization layer that sits between the model’s request and your function, and denies the calls that should never happen regardless of what the model was talked into.

A deployable service — container, config, health endpoints, smoke test — that you can put in front of real traffic.

A note on scope

Two sibling repositories carry weight this part deliberately does not.

The llm-serving-inference-guide covers the serving layer in depth: GPU containers, Kubernetes, autoscaling, traffic-splitting mechanics, Triton, vLLM, and the monitoring stack underneath all of it. When this part needs infrastructure mechanics, it points there rather than reprinting them. Part 6 is about shipping and operating the agent application — the layer above the model server.

The agentic-ai-evaluation-guide covers evaluation in depth, including a long-horizon-operations track for agents that run for days. This part treats the eval harness as a component with an interface and wires it into a pipeline; go there for what to measure and how to measure it well.

How to read it

Chapters 1 and 3 through 6 are prose you can read on a train. Chapters 2 and 7 want a keyboard — a repository to put the workflow file in, and a terminal to build and run the container.

Read Chapter 4 even if security is someone else’s job at your company. It is the chapter where the abstract phrase “agents are different” turns into a concrete list of things that can go wrong, and you cannot design a tool belt safely without it.

Start with the process chapter. It is short, and it is the one that determines whether any of the machinery in the rest of the part actually gets used.