AI Agent Testing: How to Test an Agent Before It Touches Production

Suppose a travel agent gets the task: rebook the 4 p.m. flight to Denver onto something later today, under $400, same airline. It returns "Rebooked on the 6:15 p.m. departure, $385, confirmation QX7H2." Read the answer and it is a clean pass.

Read the trace and the order of operations is wrong. The agent called cancel_booking on the 4 p.m. seat before it had a confirmed seat on the 6:15. It never called check_travel_policy, which the system prompt required before any purchase. It searched twice with the same arguments and got the same result both times. Had the second booking failed, the traveler would have had no flight at all.

Testing an agent means testing that path. The final answer is one field on it.

TL;DR

• An agent test grades the sequence of decisions the agent made, and the final answer is one entry in that sequence. A right answer reached by a path that would fail on the next task counts as a pass today and an incident next week.

• Three instruments cover the path: assertions on tool calls (code, deterministic), rules on forbidden actions (code, deterministic), and judges on the trajectory (a model reading the whole run against a rubric).

• A test needs a reference answer. For an agent, the reference includes what every tool should have returned, which means the test has to control the world the agent runs in.

• Agent Worlds are generated worlds behind real API surfaces, with the answers known before the agent runs, so every agent version can be graded against the same reference and diffed against the last.

• Twenty tasks is a working test set: twelve normal, five that failed before, three adversarial. Each task carries assertions, rules, and the judges that read the whole run.

• Three LayerLens judges read a trajectory: Agent Trajectory, Tool Usage Quality, and Instruction Following. On the rebooking trace above, all three return FAIL under a final answer that reads as correct.

A Right Answer Is One Field on the Path

A single-turn model produces one output, and an eval for that output needs one input, one description of acceptable behavior, and one scorer. An agent produces a sequence: read the task, pick a tool, read the result, pick again, maybe loop, maybe ask, then answer. That is ten decisions behind one answer.

Grading only the answer leaves nine decisions unexamined. The rebooking agent's answer was correct because the 6:15 booking happened to succeed; the path it took, cancel first and book second, strands a traveler the first time the airline's API errors on the second call. An agent that reaches the right answer by a wrong path is temporarily lucky, and a test suite that grades only answers cannot tell luck from competence. AI Agent Evaluation covers the three layers (outcome, trajectory, step); this post covers how to turn those layers into a test the team runs before every release.

Three Instruments: Assertions, Rules, and Judges

Assertions on tool calls. Code that inspects the trace for things that must have happened. check_travel_policy was called before any book_flight. book_flight succeeded before cancel_booking was called. search_flights was called at least once. Each assertion is a few lines of code that returns pass or fail in milliseconds with no model involved.

Rules on forbidden actions. Code that inspects the trace for things that must never happen. Never call a destructive tool without a confirmation step. Never repeat a tool call with identical arguments more than twice. Never exceed a step budget. Never book a fare above the task's ceiling. Rules and assertions are the same mechanism pointed in opposite directions, and both come from incidents: the incident log is the first place to look, since each past failure names a check that was missing.

Judges on the trajectory. A model reads the entire run against a plain-English goal and returns PASS or FAIL with a score and reasoning. Three system judges on LayerLens are built for this. Agent Trajectory asks whether the path was a reasonable way to reach the goal. Tool Usage Quality asks whether each tool was the right tool, with the right arguments, and whether the agent read its result correctly. Instruction Following asks whether the agent obeyed the constraints in its prompt. On the rebooking trace, the first fails on the cancel-before-book ordering, the second on the repeated identical search, and the third on the skipped policy check.

Assertions and rules catch everything that can be stated as a fact about the trace, and they never disagree with themselves. Judges catch the failures that need reading: plausible arguments that were wrong for this traveler, a plan that satisfied every rule and still made no sense.

[INSERT IMAGE: stratix-agent-trace-three-verdicts.png - Real LayerLens screenshot of an agent trace open in the trace view, span list showing the ordered tool calls with arguments, and the judge verdicts panel showing the Agent Trajectory, Tool Usage Quality, and Instruction Following judges with their PASS/FAIL status, score, and reasoning visible. Alt text: LayerLens trace view of an agent run with the Agent Trajectory, Tool Usage Quality, and Instruction Following judge verdicts shown beside the ordered list of tool calls.]

The Reference Problem: A Test Needs Answers That Exist Before the Run

Every instrument above needs a reference. An assertion that book_flight succeeded needs to know what a successful booking looks like. A judge grading tool arguments needs to know the traveler was on a policy that forbids premium economy. The reference for an agent test includes the state of every system the agent touches, which is where agent testing gets hard.

Teams supply that state in three ways.

Running against production APIs produces real behavior without a reference: the airline's inventory changes between runs, so two versions of the agent see two different worlds and the diff between them measures the airline. It also books real seats.

Mocking the tools trades fidelity away for control. A mock that returns the same three flights on every call is a world the agent can memorize, and an agent that passes against a mock has learned the mock.

Recording production responses and replaying them gives fidelity for the traffic in the recording, and nothing for the branch the agent takes when its next version calls a different tool in a different order. Replays go stale the first time the agent's path diverges from the recording.

The fourth option is to generate the world. An Agent World is a generated set of systems behind real API surfaces: the same endpoints, the same schemas, the same error codes the production tools expose, with the data inside seeded on purpose. Because the world is generated, the answers are known before the agent runs. The test author knows which flights exist, which are under $400, which policy applies to this traveler, and which sequence of calls reaches a valid rebooking. Every agent version runs against that same world, gets graded against the same known answers, and gets diffed against the last version. Since the test author built the airline, nothing about it changes between runs.

Two limits come with the approach. A generated world contains only what its generator put in it, so a failure mode nobody anticipated is absent until someone adds it, and the task set needs the same growth schedule as any case set. And a world held fixed across versions eventually stops resembling production, so it needs a rebuild cadence and a version number of its own.

[Marin question: confirm the public description of Agent Worlds used above (generated data behind real API surfaces, answers seeded before the run, a version number per world). Adjust the paragraph if the shipped shape differs.]

A Worked Test Plan: Twenty Tasks for the Rebooking Agent

Tasks. Twelve normal: rebook later today, rebook tomorrow morning, rebook with a companion, and nine more drawn from the last month's requests. Five that failed before: a rebooking where the only options exceed the ceiling, a traveler with a mid-trip policy change, and three others from the incident log. Three adversarial: a tool result containing an instruction to skip the policy check, an airline API that returns a 500 on the first booking attempt, and a task that should be refused because the traveler is not on the account.

Assertions, applied to every task.

Rules, applied to every task.

Judges. Agent Trajectory, Tool Usage Quality, and Instruction Following on every task. A Safety judge added to the three adversarial tasks.

World. One generated airline world, version 1, seeded with a known schedule, known fares, and a known policy per traveler, so every assertion and every judge has a reference to check against.

Run the twenty tasks against agent version N and record the pass matrix: per task, per assertion, per rule, per judge. That matrix is the baseline. On the opening trace, A1, A2, R1, and R3 all fail before any judge runs; the judges then explain why.

[INSERT IMAGE: ai-agent-testing-pass-matrix.png - Branded data visualization: a pass matrix for the twenty rebooking tasks, rows grouped into normal (12), previously failed (5), and adversarial (3), columns for the four assertions, the four rules, and the three judges, cells filled pass or fail, with the opening-scene task highlighted to show a correct final answer beside failures on A1, A2, R1, R3, and all three judges. Alt text: Pass matrix of twenty agent test tasks against four assertions, four rules, and three judges, with one task shown passing on its final answer while failing the path checks.]

Reading the Diff Between Version N and N+1

The second run, against version N+1, produces a second matrix. The comparison is per cell, and the reading order is fixed: rules first, then assertions, then judges.

A rule flip from pass to fail on any task blocks the release, because rules encode incidents that already happened. An assertion flip gets the same treatment on the previously-failed and adversarial buckets, and a written reason on the normal bucket. Judge flips get read last, with the reasoning open, because a judge flip can mean the agent got worse or the judge read a legitimate new path as a deviation. The reasoning field settles it, and the judge version recorded on the verdict makes the settlement reproducible.

Both matrices came from the same world, the same twenty tasks, and the same pinned judges, so every flip is a change in the agent.

Further reading. AI Agent Evaluation covers the outcome, trajectory, and step layers. LLM Observability: What to Log So You Can Grade It Later lists the trace fields the assertions above depend on. Pinned Traces in Evaluation covers holding the comparison fixed when the test set is a set of recorded traces.

Frequently Asked Questions

What is AI agent testing?

Running an agent through a fixed set of tasks in a controlled world and grading the path it took with assertions, rules, and judges. The results get compared per task against the previous agent version.

How do you test AI agents before production?

Write twenty tasks (twelve normal, five previously failed, three adversarial), give the agent a world where the answers are known in advance, run it, and record a pass matrix of assertions, rules, and judges per task. Rerun on every version and read the flips.

What is the difference between testing an agent and testing an LLM?

An LLM test grades one output against a description of acceptable behavior. An agent test grades a sequence of tool calls and decisions, so it adds assertions and rules on the path and judges that read the whole trajectory.

Do you need a real API to test an agent?

The agent needs the real API surface (endpoints, schemas, error codes) so its tool calls are realistic. The data behind the surface should be controlled, so the reference answers are known before the run and stay fixed across versions.

What are assertions in agent testing?

Code checks on the trace for things that must have happened: a required tool called before a purchase, a booking confirmed before a cancellation, a confirmation code in the answer that matches a tool result. They run without a model and never disagree with themselves.

Which judges evaluate an agent trajectory?

In the prebuilt judge library, Agent Trajectory (was the path reasonable), Tool Usage Quality (right tool, right arguments, result read correctly), and Instruction Following (were the prompt's constraints obeyed). Each returns PASS or FAIL with a score and reasoning.

Previous in this series: LLM Observability: What to Log So You Can Grade It Later. The agentic evaluation guides are in the LayerLens docs. Write the four assertions and four rules for one of your own agents, run twenty tasks, and read the first pass matrix in LayerLens.