How to Evaluate AI Agents With Multiple Correct Answers

A support agent can reach the right result through different lookups and explain it in different words. An evaluation that demands one transcript can reject those solutions. An evaluation that only asks whether the answer sounds helpful can miss an unauthorized action.

Start with five fields for each test case: starting facts, required outcome, permitted variation, forbidden behavior, and evidence needed to decide. A reference answer can illustrate the contract, but its phrasing should not silently become a requirement.

The distinction between a transcript and an outcome matters here. Anthropic's evaluation guidance treats the outcome as the environment's final state, separately from what the agent says happened. That supports checking an action against a controlled test environment instead of accepting a completion claim in the response. Agent evaluation concepts

Here is an original, hypothetical contract used throughout this guide. It describes an invented subscription-support service, not a real cancellation policy:

This is already more precise than “answer correctly.” It also exposes missing instrumentation: if the harness sees only final text, it cannot establish whether S actually changed.

The method below is design guidance synthesized from the linked sources. Documentation was checked on 3 October 2026. All support cases, tool names and expected verdicts are hypothetical; no agent trials or judge calibration were performed for this article.

Choose a check for each requirement, then decide which failures block acceptance. Do not select a metric first and redefine success to fit it.

A constraint can concern the trajectory without requiring an exact trajectory. “Eligibility evidence must arrive before cancellation” is a dependency; “call these three tools in this exact order” is a much stronger restriction.

Keep mandatory requirements separate from preferences. For C1, changing the wrong account is a failed case even if the response is clear and concise. A slower but correct lookup path can pass correctness while missing a separately declared performance target.

Ragas' metrics index separates tool use, retrieval, language comparison and general-purpose rubrics. That is a useful reminder to report dimensions separately. A tool-call score and an answer-quality score answer different questions. Ragas metric families

For C1, consider two hypothetical traces:

• Path A: read subscription S → receive ownership and eligibility → cancel S → report the verified result.
• Path B: list subscriptions for authenticated account A → identify S and its eligibility → cancel S → report the verified result.

Both can satisfy the contract. A single reference demanding the direct-read tool would reject Path B for an implementation choice. Conversely, a rule that merely requires a cancellation call could accept a call for the wrong subscription.

Express the shared conditions directly: evidence identifies S as eligible and owned by A; that evidence precedes the cancellation attempt; the cancellation arguments identify S; the final state matches the required outcome. The alternative lookup is allowed only because this hypothetical API returns equivalent ownership and eligibility evidence through both routes. Check that premise in your actual API.

AgentEvals offers strict, unordered, subset and superset trajectory modes, with exact tool-argument matching by default and configurable overrides. In its terminology, subset constrains actual calls to reference calls; superset allows additional actual calls around the required reference calls. Neither direction alone defines a complete task contract. AgentEvals matching options

Do not assume “strict” means every answer word is compared. In the inspected Python implementation, strict matching checks message count, roles and tool-call matches, but does not compare ordinary message text; calls within one message are matched without relying on their list order. This is revision-specific behavior, not a universal meaning of trajectory matching. Strict matcher source

These details refer to AgentEvals revision 946ad15b9f2ce, whose Python package metadata declares 0.0.9. Pin and inspect the evaluator you actually install. Keep account and subscription identifiers exact; normalize only fields whose equivalence your application defines. Package metadata

Extend C1 by changing one important premise at a time. In this hypothetical service, an ambiguous request needs clarification; an ineligible subscription cannot be cancelled; and a definite backend rejection must be reported with an escalation ticket. These rules are given to the agent as policy P1.

The test backend starts from a fresh fixture for each trial. It records attempted calls separately from committed changes and exposes final state to the evaluator through a trusted inspection interface. The agent cannot edit that inspection record.

For C4, the test API may return a generic access-denied result for an out-of-scope lookup. The permission boundary must be enforced there as well as evaluated. Checking for known foreign fixture values detects those particular leaks; it is not a complete test for every possible disclosure.

Make the suite test the grader too. Supply a deliberately correct Path B, a fluent answer with no backend change, a cancellation with the wrong identifier, and a cancellation before the eligibility result. These are proposed evaluator test inputs, not reported experimental results. The first should remain eligible to pass; the others should fail their specific contract checks.

For C2, an unchanged subscription is success. For C1, the same state is failure. The required outcome is conditional on the starting facts and user request, so one unconditional “was it cancelled?” metric cannot grade both cases correctly.

Ragas documents separate tool-call and goal-accuracy metrics. Its reference-free goal metric infers both intent and outcome from the conversation. Such a judgment can be useful evidence about that conversation; it cannot observe an unprovided backend record. The current latest page also distinguishes newer collection APIs from deprecated imports, so do not mix examples across versions. Agent metric definitions and API context

Use three case dispositions: pass, fail, and unresolved. They are proposed evaluation states, not a framework-specific API.

A failed mandatory check makes the case fail. Missing required evidence makes it unresolved unless another observed violation already establishes failure. Only a case with the required evidence and all mandatory deterministic checks satisfied reaches the final semantic acceptance decision. You may still run a judge on failed cases for diagnosis; its output does not reverse their failure.

For the support example, a judge receives the request, policy P1, relevant verified tool evidence and final response. Its rubric asks whether the explanation accurately describes the recorded outcome, states a necessary next step, and avoids unsupported promises. Require a verdict for each criterion, a short rationale and the evidence location. Allow an insufficient-evidence verdict instead of forcing a guess.

A semantic requirement can also be mandatory. If truthful explanation is required, a negative rubric result fails that requirement; disagreement or inadequate calibration keeps it unresolved. Calling a criterion “semantic” does not make it optional.

The diagram is a conceptual decision policy. It is deliberately asymmetric: a favorable model judgment cannot compensate for a known contract violation.

A judge is another component to evaluate. Ragas' alignment guide compares model verdicts with expert labels and inspects disagreements before revising the prompt. Use that pattern with labels grounded in your own task policy; agreement with a generic benchmark does not establish agreement on subscription cancellation. Judge alignment workflow

Before using semantic verdicts as an acceptance gate, complete these checks:

• Rubric agreement: establish labels for clear successes, clear failures and valid alternate paths. Resolve ambiguity in the policy before asking a model to resolve it. If suitable independent labels are unavailable, treat judging as exploratory and keep that acceptance requirement unresolved.
• False passes and false failures: count accepted responses that violate the agreed rubric separately from rejected valid responses. Inspect the relevant cases and evidence rather than tuning only for a higher overall agreement number.
• Disagreement handling: if judges disagree, preserve both decisions and inspect the cited evidence. A vote can be a diagnostic; it does not establish truth. Revise the criterion or evidence package when that is the cause.
• Presentation sensitivity: compare equivalent concise and verbose explanations; for pairwise judging, swap answer order and conceal candidate identities. A verdict that flips deserves investigation.
• Rubric injection: treat the candidate response and tool text as data. Include attempts to tell the grader how to score. Keep policy and evaluator instructions outside candidate-controlled content; this separation still needs testing.
• Held-out cases: separate cases used to tune the agent or judge from cases used to report performance. Group paraphrases and variations of the same underlying incident into the same split. When a held-out failure informs a change, record that exposure and add fresh coverage.
• Run isolation: clear fixture state and prevent access to answer keys, labels, grader files or previous trials' outputs. Share the behavioral policy the agent needs to follow; hide the case-specific answer and grading results.
• Version record: retain the agent configuration, model identifiers, policy, fixtures, evaluator code, judge prompt and judge model alongside results. Recheck agreement after changing the judge.

The MT-Bench/Chatbot Arena paper documents position, verbosity and self-enhancement biases in its studied judges. That motivates the presentation checks above; it does not supply a reliability percentage for your current model or application. Zheng et al., 2023, version 4

These are steps for a reader to carry out. This guide does not claim that a human labeled these hypothetical cases, that a judge was calibrated, or that the checklist eliminates leakage.

Evaluate both the application and the evaluator at their appropriate boundaries. LangChain distinguishes deterministic unit tests, integration tests with real network calls, and trajectory evaluations. A fake-backend contract test can isolate decision logic, while an integration check can reveal schema or environment mismatches the fake does not model. LangChain testing guidance

For a candidate change, keep the case set, policy and grading configuration stable while changing the intended application variable. Preserve previously accepted alternative paths as regressions: a new model should not be rejected merely because it discovers Path B.

Use this reporting checklist:

• Report outcome failures, constraint violations, semantic failures, unresolved cases and harness errors separately. Show counts and denominators; keep unresolved cases visible rather than dropping them from a headline pass rate.
• Inspect regressions by case family. A gain on clear requests can coexist with worse handling of ambiguity, access boundaries or backend rejection.
• When generation varies, predeclare repeated trials and retain every attempt. Report how often each requirement fails; do not repeatedly rerun a failing case until one answer passes.
• Distinguish regrading a saved trace from running the agent again. Regrading measures evaluator changes or variability; a fresh run also changes the trajectory and potentially the environment.
• Add a case for each newly understood failure and a valid variation that should remain accepted. Keep basic successes in the suite so a cautious agent cannot improve its safety count simply by refusing every request.
• Give each unresolved result a next action: repair missing evidence, fix a fixture, clarify the contract, or adjudicate a disputed rubric result. A judge timeout is an evaluation error, not evidence that the answer is correct.

For C1, the release-relevant question is whether the requested subscription was cancelled within the stated constraints and described truthfully. The exact lookup route matters only where it changes those facts. Your evaluation should preserve that freedom while making violations easy to identify.