An agent completes a task in a demo. The team watches it navigate the application, call a tool, and announce success. Someone asks whether it is ready for production.
That is the moment to ask for a different artefact: the release evidence.
In The Confident Wrong Answer, I looked at verifying the code an agent writes. This is the next problem: verifying a product whose behaviour depends on an agent taking several actions. A correct answer is only one part of that product.
Key takeaways
- Grade the final service state, not just the agent’s message.
- Keep critical invariants separate from average success.
- Retain every attempt, including retries and infrastructure failures.
A weighted average must never let excellent wording compensate for a duplicate order.
Start with the state the user paid for
Anthropic's guide to agent evaluations distinguishes the agent's transcript from the outcome in its environment. It also describes combining code, model, and human graders. That distinction gives quality teams a useful starting point: inspect the result independently of the agent's account of it.
Consider a fictional support agent that can create replacement orders. The user asks for one replacement for a damaged item. The agent says it has arranged it.
The release test should inspect the order service. Is there exactly one replacement? Is it attached to the correct original order? Does it contain the right item and delivery address? Was the original order's state preserved where the policy requires it?
The text can be polite and accurate while the backend has two replacements. It can also sound uncertain while the correct order exists. These deserve different findings.
Write the expected state before writing a grading prompt. If the team cannot describe the state, the feature is still underspecified.
Build a small scorecard with separate decisions
Here is the release scorecard I would start with for that replacement workflow. It is a proposed operating model, not a benchmark or an industry standard.
| Dimension | Evidence | Release decision |
|---|---|---|
| Task completion | Correct replacement exists in the order service | Compare candidate with the approved baseline |
| Critical invariants | No duplicate orders or changes to another customer's records | Any observed violation blocks promotion pending investigation |
| Recovery | Final state after a timeout or interrupted run | Require a known result or an explicit unresolved status |
| Communication | Message accurately describes the service state | Review false success claims separately |
| Operational cost | End-to-end latency and total cost per attempted task | Stay within the product's agreed budget |
A weighted average is a poor release decision here. Excellent wording must not compensate for a duplicate order. Keep the dimensions visible and give critical invariants their own gate.
Passing the invariant suite means no violations were observed in those trials. It does not prove the system can never violate the invariant. Backend constraints still need to enforce it.
Make the recovery case earn its place
The most useful scenario in this example is a write that succeeds while its response is lost.
Give the agent a tool timeout after the replacement has been created. Observe whether it checks the existing result, retries safely, creates a duplicate, or reports an unresolved outcome. A timeout before the write and a timeout after the write are separate fixtures because they leave different states.
Add an interrupted session next. Restart from the state a real resumed session would have, including any durable operation identifier. Do not quietly give the agent knowledge that production would have lost.
This is where the scorecard becomes an engineering conversation. If the agent cannot determine what happened, the missing capability may be an operation-status endpoint rather than a better prompt.
Compare versions without moving the exam
For each candidate, save a release manifest containing the model identifier, prompt revision, tool versions, policy configuration, fixture revision, and grading rules. Record provider-controlled settings that cannot be pinned as limitations.
Run the baseline and candidate against the same fixture definitions, resetting state between trials. Retain every attempt. Choose the repetition count before looking at results and show successes over total attempts for each scenario, alongside the aggregate.
Avoid reporting the best result from several retries as ordinary task success. If retries are part of the product, evaluate the whole retry policy, including its elapsed time, spend, and final state. If the evaluator failed to start, report an infrastructure failure separately rather than quietly removing it from the evidence.
Small samples can expose a failure quickly; they give much weaker evidence that a rare failure is absent. Increase repetitions for variable or consequential workflows, and describe uncertainty honestly when differences are small.
End with a decision someone can own
A useful release note might read:
Illustrative decision: hold the candidate. Ordinary replacements pass, but the lost-response scenario can create a second order. Add an idempotent operation and rerun the recovery suite before reconsidering promotion.
That is actionable. A score of 94 out of 100 is not, until someone explains what sits inside the remaining six points.
For the first week, choose one workflow, name its final state, add a failure after a successful write, and compare the current agent with one candidate. Assign an owner to the promotion decision and a rollback path for the deployed version. Expand the suite from real failures as the product grows.
The release gate has done its job when it changes a shipping decision for a reason the team can inspect.
Continue reading
- Your Agent's Tool Permissions Are Part of the Test Surface covers the actions the agent must be unable to take.
- AI-Generated Tests Need a Test of Their Own covers the evidence behind a green test suite.