Evaluation Closed-Loop
A self-optimization mechanism in AI agent workflows in which an agent automatically assesses the quality of its output after completing a generation task, then uses the assessment as feedback to drive the next iteration. This forms a continuous cycle of generation, evaluation, feedback, and regeneration.
Timeline (last 90 days)
Establish a small-scale evaluation loop: pick 10 to 20 typical feedback emails as a test set, define the criteria for a "good answer," and run the test set after each prompt adjustment to observe changes in the pass rate.
Evaluation needs to cover multiple layers of metrics—whether the retrieved context is relevant, whether tool calls are correct, and whether the final answer is accurate—and includes both offline and online evaluation.
The evaluation closed-loop workflow lets the Agent itself complete the full cycle of generation—evaluation—feedback—regeneration, rather than having humans serve as quality gatekeepers.
All Facts (3)
Establish a small-scale evaluation loop: pick 10 to 20 typical feedback emails as a test set, define the criteria for a "good answer," and run the test set after each prompt adjustment to observe changes in the pass rate.
50%UnverifiedEvaluation needs to cover multiple layers of metrics—whether the retrieved context is relevant, whether tool calls are correct, and whether the final answer is accurate—and includes both offline and online evaluation.
50%UnverifiedThe evaluation closed-loop workflow lets the Agent itself complete the full cycle of generation—evaluation—feedback—regeneration, rather than having humans serve as quality gatekeepers.
50%