Evals

The core problem with AI code generation is that models tend to produce tests that look reasonable rather than tests that capture real defects.

OpenAI将AGI定性为能在大多数经济价值任务上超越人类的AI系统

In multi-step agentic workflows, the final result is formed by a chain of intermediate decisions, and the same task may have multiple reasonable execution paths, making evaluation more difficult than traditional software testing.

Evals is a testing framework used to systematically measure the quality of model or agent outputs, analogous to unit tests and integration tests in software engineering.

A benchmark is a fixed set of questions used to score and horizontally compare models, similar to a mock exam, and is used to rank models.

AI product design requires introducing an evaluation system (Evals) to continuously measure the quality of model outputs, rather than relying on traditional functional testing.

OpenAI has publicly discussed that their model was specifically trained to efficiently use the apply_patch tool.

顶级AI公司通常设有专门的提示词工程团队，使用自动化评估框架（如OpenAI的Evals）对提示词进行量化评测

Headhunter Mia stated that what is still in vogue is agents, especially Co-worker Agents oriented toward collaboration scenarios. Coding Agents represent a persistent talent gap, and talent related to Evals and pretraining data is also beginning to receive attention.

Benchmarks answer how strong a model is, while Evals answer how an Agent performs in a specific scenario.

Once benchmark questions are made public they may be contaminated by training data, and a high score does not mean the model is equally reliable in a specific business scenario, so Evals are needed to supplement them.

The lack of reliable Evals leads to an "optimization illusion" — where metric numbers look good but actual user experience has not improved or has even regressed

Building high-quality business Evals requires domain experts to design test sets, establish human-annotated golden sets, and track model iteration effects through continuous A/B comparisons

The difference between Evals and traditional software integration testing lies in the probabilistic nature of LLM outputs, which requires replacing exact string assertions with a golden question set plus expected-behavior judgments.

Error analysis involves systematically attributing root causes after Evals expose problems, including reasons such as insufficient retrieval recall, context window overflow, incorrect tool-call formatting, or the boundaries of the model's reasoning capability.

Because agent outputs are often open-ended text, Evals typically need to combine multiple approaches such as rule-based checks, comparison against human-annotated gold answers, and LLM-as-Judge scoring.

Agent evaluations (Evals) fall into two categories: automated evaluation (e.g., LLM-as-judge, rule matching) and human evaluation

The new testing paradigm for AI systems is called 'Evals', with core metrics shifting from single pass/fail to statistical indicators such as Pass@k, consistency, and robustness.

Tests validate the deterministic parts, while Evals validate the non-deterministic parts (agent paths, tool selection, and output quality).

An Evals evaluation system allows developers to objectively measure agent performance rather than judging by intuition.