Evaluation Set

Build a small evaluation set for the agent that covers typical scenarios, and record the change in pass rate after each modification; the evaluation set should include normal paths, boundary conditions, and deliberately created anomalies.

It is recommended to hide model selection behind a configuration value, deliver the evaluation set alongside the feature, and set a fallback path: when the small model fails validation twice in a row, escalate to a stronger model.

When choosing a model, the key question is to find the model that passes the evaluation set with acceptable accuracy and has the lowest running cost, not the smartest model.

The first version of an evaluation framework can be a spreadsheet (inputs plus expected checks) plus a script that prints scores after a run and compares them with the previous run, without needing a platform.

For each evaluation sample, someone who understands the business should write down a written definition of a "good answer", which helps the team reach agreement on the feature's goals.

The eval set should deliberately include uncomfortable cases such as ambiguous requests, out-of-scope requests, bad inputs, adversarial inputs, and high-risk cases, in order to expose real problems.

Evaluation samples should come from actual workloads (such as real customer service messages, documents, and call recordings), not fictional or clean samples.

The eval set consists of three parts: a collection of real inputs, a written definition of a "good answer" for each input, and the scores recorded after each change.

Production is the real source of the eval set: record inputs, outputs, and system decisions (within privacy rules), watch for failure signals, and turn every real failure into a new evaluation case.

Most changes to an AI feature (prompt edits, model updates, data changes, cost changes) do not show up in diff review, and the eval set should be run on every such change.

For LLM applications, examples used for prompt engineering debugging should not overlap with the final evaluation set.

The quality of building an evaluation set depends on the degree of understanding of the feature goals, and because LLM output is probabilistic, evaluation noise is unavoidable, so threshold design should accept a certain proportion of fluctuation (for example, allowing 90% of cases to pass).

It is recommended to start from a minimal viable set: ten cases that can represent "correct," including inputs that previously caused failures, and to keep adding new cases as real-world feedback comes in.

The evaluation set itself also drifts and needs periodic review and updating; otherwise the tests will gradually become ineffective.

The evaluation set should grow in sync with the expansion of the Agent's capabilities, and online failure cases should be turned into new evaluation cases.

Reliable systems in production require an eval set (20 to 50 saved test cases used to catch regressions) and prompt chains with retry guardrails, rather than relying on vibe checking or giant mega-prompts.

The method for building an evaluation set is to pick 30 to 50 real inputs from the product and pre-write the criterion for 'what counts as a good output' for each, forming a repeatable benchmark.

An evaluation set of fifty questions is large enough to cover the main query types, yet can be quickly rerun after each change (usually completed within a few minutes), and is cheap enough to be incorporated into the daily development loop.

An evaluation set is a collection of questions with standard answers or expected hit document annotations, recall measures the proportion the system actually returns out of all relevant documents that should be retrieved, and a baseline is the score of the simplest solution on the same evaluation set.

The local evaluation used a small evaluation set containing seven question-and-expected-answer pairs; in the first round of testing four passed and three failed, with failure causes including omitting known facts, skipping follow-up questions, and triggering rate limits.