Agent Simulation Testing: Put a Simulated User in Front of Your Agent Before Real Traffic Does

Your agent passed 94% of the golden set, so you shipped it. Three days later a customer asked for a refund, changed their mind halfway through, gave an order number with a transposed digit, and then asked an unrelated shipping question while the agent was mid-lookup. The agent refunded the wrong order. Nothing in your test suite could have caught this, because nothing in your test suite ever talked back.

That gap has a name now. Over the past year, agent evaluation platforms converged on the same architectural answer: a simulation layer between offline assertions and live traffic, where your agent talks to a synthetic counterparty before it ever talks to a customer. The convergence happened for a simple reason. Multi-turn agents fail on trajectory, and trajectories only exist when the other side of the conversation adapts.

Single-Response Evals Grade the Wrong Unit

Most eval suites grade one response given a fixed context. That works for classification, summarization, and RAG answer quality. It quietly stops working the moment your agent maintains state across turns and calls tools whose outputs feed later decisions, because the unit of failure is no longer the response. It is the conversation.

Sierra's τ-bench made this measurable. The benchmark pits agents against a simulated human user in retail and airline support scenarios, grading the final database state rather than the transcript. Two results should bother anyone shipping on golden-set pass rates alone. First, even frontier models at the time completed under half of airline tasks. Second, and worse, the paper's pass^k metric - the probability an agent succeeds on all k independent runs of the same task - collapsed as k grew. An agent that looked passable at pass^1 dropped below 25% at pass^8 on retail tasks. Same task, same agent, wildly different outcomes across runs.

This is the trajectory problem in numbers. An agent can produce a fine response at turn three and still end up in a corrupted state by turn nine, because turn three subtly narrowed the paths available later. A frozen test case cannot express this. The moment your recorded "user turn 4" no longer matches what a real user would say in response to the agent's novel turn 3, the replay is fiction.

Industry survey data says teams feel this even when they cannot name it. LangChain's State of Agent Engineering research consistently finds output quality, ahead of cost or latency, as the top barrier keeping agents out of production. Quality here rarely means bad sentences. It means the agent did something wrong across a sequence that no single-turn check flagged.

The takeaway is uncomfortable for anyone with a green CI dashboard: a passing golden set tells you your agent handles conversations that already happened. It says almost nothing about conversations that will happen.

A Simulator Is a Test Fixture With a Policy

The fix is to replace the frozen counterparty with a reactive one. A user simulator is an LLM (or scripted state machine, for narrow cases) that plays the other side of the conversation according to a defined policy. The framing that keeps teams out of trouble is to treat it exactly like any other test fixture, which means it needs a specification, not vibes.

A useful simulator spec has five parts:

The hidden-information component deserves emphasis because it is what makes simulation different from paraphrasing your golden set. Real users do not volunteer their account number, their constraint ("I leave for the airport in an hour"), or their history unless prompted. An agent that only works when all context arrives in turn one has never been tested, only demoed.

Grading follows the same discipline. Where possible, assert on final state rather than transcript aesthetics: did the right database row change, was the right refund amount issued, did the agent call the escalation tool when the policy required it? τ-bench grades end-state for exactly this reason, and it is the cheapest reliable signal you can get. Layer LLM-judged criteria (tone, unauthorized promises, policy violations mid-conversation) on top, but validate those judges against human labels before trusting them. Structured output checks are your friend here too - if your agent emits schema-validated actions, the simulator harness can assert on them mechanically.

One implementation warning: do not power the simulator with the same model that powers your agent, at least without adversarial prompting. Same-family models are suspiciously polite to each other. They accept vague answers a real user would push back on. A different model family, or a simulator prompt that explicitly mandates terseness and non-cooperation, produces conversations that look far more like your production logs.

Seed Scenarios From Transcripts, Not Imagination

The most common way teams waste a simulation harness is by inventing personas in a brainstorming meeting. "Frustrated customer," "confused elderly user," "power user in a hurry" - these feel like coverage and test almost nothing, because they are drawn from the same mental model that built the agent. If you could imagine the failure, you would have handled it in the prompt.

The scenarios worth simulating come from error analysis on real traffic. Hamel Husain and Shreya Shankar's evals methodology is blunt about where eval time should go: reading real transcripts, annotating failures in plain language, and clustering them into recurring failure modes. Each cluster becomes a simulator scenario. The customer who supplied a malformed order ID is a scenario. The user who asked two questions in one message and got only one answered is a scenario. The person who agreed to a plan and then contradicted it two turns later is a scenario, and in our experience at OpenNash it is one of the highest-yield ones, because agents are systematically bad at revising commitments they have already verbalized.

This is also the direction commercial platforms took once they saw enough production data. Decagon's next-generation simulations generate synthetic test conversations grounded in historical conversation patterns rather than hand-written scripts, precisely because hand-written scripts encode the builder's blind spots. The pattern generalizes beyond chat: Regal's agent simulations exist because voice teams were burning hours placing manual test calls before every prompt change, and manual test callers get lazy in exactly the ways simulated ones do not.

For teams with no production traffic yet, cold-start seeds exist in adjacent data: support tickets, sales call notes, chat logs from the human process the agent is replacing. A composite scenario built from ten real tickets beats a persona invented at a whiteboard every time.

Structure the result as a coverage matrix - failure-mode clusters on one axis, behavior policies (cooperative, terse, mid-conversation reversal, wrong information supplied confidently) on the other. Ten to twenty scenarios, each run 5-10 times per release to surface run variance, gives you 50-200 conversations per release candidate. That is a reviewable volume with real statistical teeth.

Version the Simulator, Because Drift Here Is a Broken Thermometer

Here is the part almost nobody does, and it is the core of the argument: the simulator is itself an LLM system, which means it drifts, and simulator drift is indistinguishable from agent regression unless you control for it.

Concretely, suppose your simulation pass rate drops from 88% to 74% overnight. Three explanations exist. Your agent got worse. Your simulator got harder (its underlying model updated and now plays users differently). Or your grader got stricter. Only the first one is information about your product. The other two are broken instrumentation, and if you cannot tell them apart, every number your harness produces is noise wearing a lab coat.

The controls are boring and borrowed from ordinary software practice:

• Pin the simulator's model version. Aliases like "latest" have no place in a test fixture. Upgrade deliberately, never incidentally.
• Keep simulator prompts, personas, and seed transcripts in version control, reviewed like code. A one-word change to a persona prompt can shift difficulty more than a model swap.
• Maintain a calibration set. Keep 20-30 recorded real conversations with human-labeled outcomes. When any simulator or grader component changes, replay calibration and confirm agreement with the human labels before accepting new pass rates. This is the same judge-validation discipline the evals literature demands, applied to the whole fixture.
• Write a changelog entry when the simulator changes, and annotate your metrics dashboard with it. A pass-rate discontinuity should be explainable by pointing at either an agent commit or a simulator commit, never a shrug.

Teams that skip this end up in a specific bad place around month three: pass rates wobble, nobody trusts the harness, and engineers quietly go back to shipping on manual spot checks. The harness did not fail technically. It failed as an institution, because its measurements stopped being comparable over time.

Build, Buy, or Wait

The 2026 tooling market makes buying viable for the first time, but the decision splits cleanly by situation.

Buy a platform if you run high-volume customer conversations on relatively standard channels. Maxim's simulation platform covers persona-driven multi-turn simulation with integrated evals and observability, and surveys of the category (FutureAGI's 2026 roundup is a reasonable map) show the core features converging: persona libraries, scenario generation from historical data, trajectory-level grading. If your agent is a CX agent on a mainstream stack, the build-versus-buy math favors buying, and your effort should go into scenario curation, which no vendor can do for you.

Build the harness if your agent orchestrates proprietary internal tools, touches systems a SaaS simulator cannot reach, or operates in a domain where transcripts cannot leave your infrastructure. The honest engineering estimate is smaller than teams expect: a simulator loop is a few hundred lines wrapping two LLM calls and your agent's entry point, plus state assertions against a seeded test database. Open harness patterns like OpenAI's evals framework give you registry and grading scaffolding to steal. The real cost is not the loop. It is the ongoing scenario curation and calibration discipline described above, which is roughly a day per release cycle once running.

Wait if you are pre-launch on a single workflow that already has a human reviewing every agent output. Simulation buys you confidence before removing the human. Until removing the human is on the roadmap, your reviewer is a better failure detector than any synthetic user, and your energy belongs in error analysis of their corrections.

Budget note either way: simulated conversations burn tokens on both sides of the dialogue, so expect roughly double the per-conversation inference cost of production, times your run count. A full release-candidate suite typically lands in the tens of dollars. Compare that to one wrongly issued refund, or one afternoon of engineers manually role-playing customers, and the math ends quickly.

Where Simulation Sits in an Agent Delivery Pipeline

When OpenNash builds production agents, the simulator gets written during the design phase, before the agent works, for the same reason test-first developers write the failing test first. The scenario matrix forces the uncomfortable conversations early: what should happen when the user reverses a confirmed decision, what information the agent must never act on without verification, which trajectories end in human escalation rather than autonomous action. Those answers become simulator termination conditions and grading assertions, and the client owns the harness as ordinary code in their repo after handoff, so the release gate survives the engagement.

If you are running an agent today that was validated only on single-turn checks, the next step does not require a vendor or a budget line. Pull 50 recent production transcripts this week, annotate failures in plain language, cluster them, and write your first ten simulator scenarios from the clusters. If you would rather pressure-test that scenario matrix against someone who has built these gates before, bring the clusters to a working session with OpenNash and we will map them to a harness design you keep.