How to think about testing your agents?

There is a lot of mystique around evals. There should not be. Underneath the buzzwords, the whole thing rests on a small loop: watch real runs, name what broke, save the important cases, test before letting changes go live, and keep watching real customer use.

This guide builds those ideas one at a time. Each module assumes only what came before it, and every module returns to the same refund agent.

If you have not read Zero to Agent yet, start there. It gives you the basic mental model for what is happening in this space. This guide is for what comes after: you built an agent, the prototype works most of the time, and now you need confidence before it touches real customers.

Building a prototype that works 80% of the time is easy. That is not enough for real customer use, especially in large enterprises or sensitive healthcare, finance, and insurance workflows. If you want a simple way to test an agent you built as a fast prototype, this guide is for you.

This is not a phase you do after the "real" work. It is part of building, the way reviewing support calls is part of improving a support process. You do not wait for an incident to learn what customers are hearing. The minimum viable version is simple: review a small batch of real runs after meaningful changes, save the failures that matter, and keep the saved cases small enough that people still trust them.

A real run, often called a trace, is the full recording of one conversation: what the customer said, what the agent did, every lookup or action, what came back, and every reply. Think of it like a call recording plus the agent's work notes. Traces are where evals come from.

Zero to Agent ended on three words: watch the traces. This is where you do it. A trace is the full recording of one run: the customer's message, every lookup or action, what the agent found, and the final answer.

Do not start by writing tests. Start by reading real conversations and asking three questions:

1. What did the customer want?
2. What did the agent do?
3. Where was the first mistake?

That third question matters most. Mistakes cascade: a wrong step early forces every later step to be wrong too. The agent's final answer being wrong tells you less than the first broken step. When you are starting out, mark only the first failure in each run. Fix the cause and the later mess often clears up on its own.

Once you have read enough runs, you will notice the same problems recurring. The skill is naming them precisely. "Bad answer" tells you nothing. The actual pattern tells you what to fix.

The method is plain. Read runs, write quick notes about what broke, then group the similar notes into named patterns and count them. That is it: write notes, then group notes. Each good failure name points to an ownerable fix: policy, lookup, workflow, approval, or escalation.

Start with 10 runs. If the same 3 to 5 mistakes keep appearing, you have enough to act. As the agent matters more, read more. The goal is not to catalogue every conceivable failure. It is to find the ones that happen most.

Before you write pass/fail checks, decide what failure means. A 99% pass rate is useless if the 1% is the agent refunding the wrong customer, leaking private data, or bluffing when it should ask for help.

For an agent that stands in for a human, your goal is not a pretty average score. Your goal is trust where trust matters. Rank failures by the harm they cause: money, privacy, legal exposure, customer trust, and time wasted by your team.

Think about a finance agent that cancels subscriptions and moves money. Impressive. Until the day it confidently gives the wrong balance or moves money without being sure. Then you stop trusting all of it. For agents that replace human work, a confident wrong answer is worse than an honest "I don't know." Trust beats cleverness.

Now you turn the failures that matter into tests. Start with golden cases: a small set of must-handle customer situations your agent must always handle correctly.

A golden case is not an engineering test first. It is a customer moment where a wrong answer costs trust, money, escalation time, or legal risk.

You do not need fancy tooling. Write them down somewhere and start with the single most common request your agent must never botch. For a refund or CX agent, the list might be:

Write each case as pass or fail. Never a 1 to 5 rating. Scores like "4 out of 5" feel informative but they are a trap. The gap between a 3 and a 4 is subjective, reviewers disagree, and everyone hides indecision in the middle. Pass or fail forces a clear call. If you want more detail, break one case into several pass or fail checks.

Here is our refund case as a real eval. Plain English first:

The most expensive mistake here is testing your prompt in isolation. Once an agent is wired up to the invoice lookup, policy source, approval rules, and what the agent remembers, the behaviour lives in the whole system. Not in the prompt text.

A prompt that looks perfect on its own can fall apart the moment a real lookup returns a messy result. So good offline tests take an input, run the real agent, and check what actually happened: the answer, the lookups or actions, and what it actually changed, like creating a refund or sending an email.

For our refund agent, a real test checks four things:

Two kinds of safety check, and the difference is timing. Blocking guardrails stop risky actions before they happen. Measuring reviewers help you learn what to improve after the fact.

Offline tests catch what you thought would break. Production, meaning real customer use, shows you what actually confuses real customers. This is where evals stop being theory and become an operating rhythm.

Watch real runs. Find a failure. Reproduce it. Save it as an eval. Fix it. Ship. Keep watching.

How much machinery you need scales with your volume. At a few runs a day, just read them all. As traffic grows, you let the system tell you what deserves attention. Recurring stumbles become tracked issues. Big changes get tested on real customer traffic before rolling out to everyone. Do not reach for the heavy monitoring setup before you have read enough raw runs to know what you are looking for.

Stop here if you are new. The main loop is enough to begin. These notes are only for teams already running evals and ready for the next layer.