AI Agent Testing: Build a Sandbox Before Production

A conventional integration test asks whether the refund API returned 200. An agent test asks whether the correct customer received one refund, for the correct amount, under the correct policy, with no duplicate email, no unrelated record change, and a trace that explains the decision.

That difference is the gap between testing software plumbing and testing delegated work.

REAL, a NeurIPS 2025 benchmark, built deterministic replicas of 11 commonly used websites and 112 tasks covering information retrieval and state-changing actions. Its strongest reported baseline completed 41.07 percent of tasks. More important for enterprise builders, REAL checked action tasks by comparing exact state before and after a run, while using rubric-guided model judgments for information-retrieval answers. That split is useful: use deterministic assertions wherever the system state makes them possible, and reserve model judges for outcomes that genuinely require interpretation.

You should not learn that an agent misunderstands a refund, CRM, or invoice workflow from its first production mistake. Build a place where it can fail loudly, repeatably, and without consequences.

Agent Testing Must Verify State, Not Conversation

Agents produce trajectories. They inspect data, choose tools, fill arguments, interpret results, retry, and eventually stop. Two agents may follow different valid paths to the same outcome. One agent may write a persuasive final message after leaving the underlying system in the wrong state.

That makes three common test methods incomplete:

• Snapshotting the final text misses hidden tool and state errors.
• Matching an expected action sequence rejects valid alternative paths and may accept a path whose final state is wrong.
• Checking API status codes proves that requests were accepted, not that the business goal was met.

The test oracle should describe the desired end state and prohibited side effects. For "refund order 4812 and tell the customer," it might assert:

refund.count(order=4812) == 1
refund.amount == approved_amount
order.status == "refunded"
ledger.net_change == -approved_amount
email.recipient == order.customer_email
email.count(template="refund_confirmed") == 1
unrelated_records.changed == 0

This principle predates current agents. WebArena used self-hosted websites and programmatic checks of functional correctness, allowing more than one action path to satisfy a task. REAL extends the pattern with deterministic replicas and controlled evaluation across a broader set of sites.

Score the run at three levels:

The outcome score tells a buyer whether the work was completed. The control and execution scores tell an engineer where to fix a failure.

Build the Smallest Faithful Sandbox

A useful sandbox is not necessarily a full copy of production. Full clones are expensive, stale quickly, and often retain network paths or credentials that make "testing" less isolated than expected. Build the smallest environment that preserves the workflow's important decisions and side effects.

For an invoice agent, that may include:

• A fixture database with vendors, purchase orders, invoices, approvals, and duplicate cases.
• Stubbed document storage with clean, malformed, and adversarial files.
• A payment adapter that records intent but cannot move money.
• An email sink that captures recipients, subjects, bodies, and attachments.
• A clock you control for due dates, aging, and retry windows.
• Identity fixtures for analysts, managers, tenants, and unauthorized users.
• A reset function that restores the exact starting state before every run.

The sandbox boundary should deny production by construction. Use separate accounts, credentials, networks, projects, and destinations. A variable named TEST_MODE inside a process holding a production payments key is not isolation.

Mirror contracts, not every implementation detail. If the production CRM tool returns {account_id, status, owner}, the sandbox should return the same schema and meaningful errors. It does not need the CRM vendor's full interface. It does need to reproduce pagination, missing records, permission denial, conflict responses, timeouts, and partial failure if those affect agent behavior.

Keep test runs isolated. Playwright's browser-context guidance explains why clean-slate browser contexts prevent state from one test cascading into another. Apply the same idea to databases, queues, caches, and agent memory. Each case gets known fixtures and leaves no residue for the next.

Determinism does not mean every model response is identical. It means the environment, fixtures, time, and external outcomes are controlled enough to attribute changes. Run each high-risk case several times to measure outcome variance.

Create a Risk-Based Test Matrix

Start with real work, not benchmark trivia. Sample completed cases from production operations and interview the people who catch mistakes. Build 30 to 50 cases across the branches that matter, then grow the suite from review findings and incidents.

Use six test families:

1. Happy paths. Common, unambiguous work with complete data.
2. Business exceptions. Missing purchase orders, disputed charges, expired policies, special customer terms, or conflicting records.
3. System failures. Timeouts, rate limits, stale reads, malformed tool results, authentication expiry, and partial writes.
4. Safety and permission failures. Wrong tenant, unauthorized record, excessive amount, forbidden destination, or missing approval.
5. Adversarial content. Prompt injection in an email, false claims in an attachment, poisoned memory, and tool output that asks the agent to ignore policy.
6. Recovery cases. Retry after ambiguous failure, resume after approval, rollback after partial completion, and stop after a duplicate request.

For each action, test idempotency. Networks fail after a server commits but before the agent receives the response. A naive retry then creates a second refund, ticket, or email. Stripe documents idempotent requests as a way to retry creates or updates without performing the operation twice. Your internal write tools need an equivalent contract: stable operation IDs, duplicate detection, and a result that tells the agent the earlier action already succeeded.

Use a matrix that connects business risk to assertions:

Do not give every failure equal weight. One missed internal tag should not cancel out one unauthorized disclosure in an average. Set hard zero-tolerance gates for severe safety failures and separate targets for task completion.

Capture Side Effects and the First Failed Step

When a test fails, the team needs more than a transcript. Capture a structured trace with:

• Task and fixture version
• Prompt, model, and parameter version
• Tool catalog and schema version
• Identity and authorization scope
• Every tool proposal and normalized argument
• Tool result, retry, and latency
• Policy and approval decisions
• State diff and captured external effects
• Token use and estimated cost
• Final response and stop reason

Trace each model call and tool call as a span so engineers can locate the first incorrect decision. The first failed step is usually more useful than the final symptom. A wrong customer email may begin with bad entity resolution, not poor email generation.

Use a stable telemetry format rather than inventing a transcript for each agent framework. OpenTelemetry traces model a request as a set of causally related spans, which fits model calls, retrieval, policy checks, approvals, and tools. Add agent-specific attributes to that structure, and preserve one correlation ID from the incoming request through the final side effect. The trace should survive a model or orchestration change.

Use side-effect adapters for external systems. Email goes to a sink, payments become immutable intent records, webhooks enter a capture queue, and CRM writes land in a disposable store. Assert both what happened and what did not happen. An agent that sends the right email twice has not passed.

Keep raw sensitive data out of general logs. Record stable fixture identifiers, hashes, policy decisions, and redacted payloads. Give a restricted debugging path access to full content only when needed.

When a case fails, label the first failed layer: observation, retrieval, planning, tool selection, arguments, authorization, execution, recovery, or response. This taxonomy directs the fix. Prompt changes cannot repair a missing database constraint. Model upgrades cannot repair a sandbox that returns unrealistic errors.

Use Release Gates and a Staged Production Path

Every change to the model, prompt, tools, policy, retrieval index, or orchestration can alter behavior. Run the same suite on the candidate and current production versions, then compare:

Thresholds depend on the workflow. A draft generator can tolerate more variance than a payment agent. Write the thresholds before testing the candidate so the team cannot rationalize a favored model through the gate.

Document who may waive a gate and for how long. A one-time exception needs an owner, reason, expiry, and follow-up case in the suite. Otherwise, release pressure slowly rewrites the safety standard. The NIST AI Risk Management Framework places measurement inside a larger cycle of governing, mapping, measuring, and managing risk. In release terms, a score has meaning only when ownership and response rules exist around it.

Keep the candidate artifact immutable during evaluation. If a prompt is edited or a tool schema changes, start a new candidate run. Mixing versions inside one report makes both the pass rate and the audit history unreliable.

After sandbox approval, move through staged exposure:

1. Replay: run recorded tasks with no live systems.
2. Shadow: observe live inputs and propose actions without executing them.
3. Read-only: allow live retrieval with no writes.
4. Draft and approve: let the agent prepare writes that a human commits.
5. Limited autonomy: allow bounded, reversible action classes with sampling.
6. Broader autonomy: expand only when production evidence supports it.

Production is still part of evaluation because traffic differs from fixtures. It should reveal distribution shift and new edge cases, not basic duplicate-write bugs. For knowledge workflows, keep retrieval and answer diagnosis separate with the RAG evaluation playbook. For the operating loop after launch, see Loops and Evals and our guide to workflow-specific evals.

Give operations a capability-level stop switch and a known rollback target. If the candidate's refund behavior drifts, disable refunds while research and summarization continue.

How OpenNash Can Help

OpenNash builds agent test environments around the workflow a business actually owns. We identify the system states that matter, create representative fixtures and failure modes, wrap side effects, write deterministic outcome checks, and connect traces to the first failed step. The release report shows task success, safety, cost, latency, variance, and regressions by workflow branch.

An existing integration-test stack may only need agent traces and outcome graders. A workflow that spans several vendors or carries hard-to-reverse actions may justify a dedicated simulator. If the team cannot define the correct post-task state, pause autonomy and start with process discovery. A sandbox cannot compensate for an undefined outcome.

OpenNash can also set up the staged production path, including shadow runs, scoped credentials, approval packets, sampling, release gates, rollback, and stop controls. Ownership stays with the client team, including the fixtures and eval history that remain valuable when models change.

Book a call to design a sandbox for one workflow. Bring ten completed cases, the production tool list, and the side effect that would be hardest to undo.