LLM Evaluation
A methodology for quantitatively scoring the output quality of AI systems, replacing the binary assertion logic of traditional software testing. It covers three evaluation targets: large models, RAG, and agents.
Timeline (last 90 days)
Agent evaluation differs from large model evaluation: large model evaluation usually gives a question and looks at the generated answer, whereas an Agent involves multi-step reasoning, tool calls and state transitions, so the object of evaluation changes from a single output to the entire execution trajectory.
Large model evaluation, RAG evaluation and agent evaluation are three problems at different levels and cannot be handled with the same set of methodologies.
All Facts (2)
Agent evaluation differs from large model evaluation: large model evaluation usually gives a question and looks at the generated answer, whereas an Agent involves multi-step reasoning, tool calls and state transitions, so the object of evaluation changes from a single output to the entire execution trajectory.
50%UnverifiedLarge model evaluation, RAG evaluation and agent evaluation are three problems at different levels and cannot be handled with the same set of methodologies.
50%