#大模型评估
7 related articles
AI Testing Transformation Guide: From Testing AI to Testing with AI — An Evaluation Playbook
From assertions to scoring, AI testing is shifting to an evaluation paradigm. This article explains the three-layer evaluation objects—large models, RAG knowledge bases, and agents—and the three integration paths of testing with AI, letting AI test, and testing AI, with practical advice on environment setup and cost.
New Direction for AI Testing Roles: An Introduction to LLM Evaluation and Agent Practice
How to get started in AI testing? From LLMs and knowledge bases to agents, a systematic overview of the three major objects of AI evaluation and a hands-on path, covering Python setup, OpenAI-compatible API essentials, and why AI testing uses scores instead of assertions.
The Reproducibility Crisis in Hosted LLM Behavioral Evaluation: Measurement Methods Matter More Than Models
A new arXiv paper reveals reproducibility challenges in hosted LLM evaluation: using Regent Chess and Gemini models, it shows replication, measurement sensitivity, and persistence can yield contradictory conclusions in a single evaluation.
AI Large Model Testing in Three Phases: The New Paradigm from Assertions to Evaluation
AI testing is shifting from assertions to evaluation. This guide covers Testing with AI, alongside AI, and of AI — plus LLM, RAG, and Agent evaluation layers, version pinning, and Promptfoo.
Hindsight Bias in Clinical Temporal Reasoning: How Future Data Skews LLM Judgment
New arXiv research reveals hindsight bias in clinical LLMs: exposure to full outcome timelines causes systematic answer shifts, while temporal masking reduces bias without hurting accuracy.
When LLM Judges Agree, Should We Actually Trust Them?
When multiple LLM judges agree, can we trust them? This article examines correlated error bias in LLM-as-a-judge evaluation and why consensus doesn't equal correctness.
