Three Stages of AI Model Testing: From LLMs to Agent Evaluation for Beginners

AI software has permeated the apps we use daily—from intelligent customer service in state-owned enterprises to various AI-native applications—and testers now face a practical question: when AI capabilities are embedded in software, can traditional functional testing still cover it? A talk on Bilibili about AI large model testing proposed a clear framework: replace "assertions" with "evaluation," and break down AI testing targets into three progressive stages.

Why AI Testing Should Be Renamed "Evaluation"

The core method of traditional software testing is assertion: given an input, the expected output is deterministic—right is right, wrong is wrong. But the output of AI systems often falls into a gray zone between right and wrong. The speaker pointed out that in this case, results need to be expressed as scores rather than simple pass/fail judgments, so the more accurate term is "AI evaluation."

Behind this terminology shift is a change in testing mindset. Traditional functional testing focuses on "whether the function works," while AI software also needs to be examined in terms of its "intelligence, capabilities, and behavioral constraints"—dimensions that are hard to assert with boolean values. The speaker emphasized that this is not wordplay, but rather because AI system outputs are probabilistic and open-ended, requiring the introduction of a scoring mechanism.

Three Relationships Between AI and Testing

The speaker reviewed three ways of combining the two discussed previously; this classification framework helps understand the current stage:

• Testing with AI: AI is an assistant, humans lead the testing process, using AI tools to help test traditional software.
• Letting AI test: AI becomes the protagonist, with the vast majority of testing work carried out by AI while humans do other work.
• Testing AI: Humans and AI no longer collaborate but stand as adversaries—the test target is AI itself.

The first two are essentially "using AI technology to test traditional software," differing only in who plays the lead role. The third is the new challenge of today: AI has shifted from tool to test target. The speaker noted that this is an inevitable step predicted earlier.

Reality-Driven: AI Is Already Everywhere

Why must we learn AI evaluation now? The speaker gave two reasons.

First, AI-native software is increasing. Claude Code, CodeX, and products from multiple domestic vendors are software built directly for AI capabilities, rather than stuffing AI features into traditional software.

Second, traditional software is also integrating AI. Apps from super state-owned enterprises like China Mobile and China Telecom already use AI features, and 2B software basically has agents or intelligent customer service behind it. The speaker stated bluntly that even testers who only want to do pure manual testing and not write code cannot isolate themselves from AI—because the software under test itself contains AI capabilities, and they must master how to test such software.

The Three-Stage Evaluation Framework: LM, RAG, Agent

The speaker summarized AI evaluation targets into three progressive levels, corresponding to three mainstream AI capabilities:

Stage One: Large Language Model (LM)

Accessing a large model grants basic AI capabilities, with chat as the typical scenario. Pinduoduo once launched a chat feature where users could chat with a large number of "people" in different roles, creating strong immersion but with a low barrier—simply integrating a large model suffices. However, such applications are limited to chatting and cannot handle specific business, because business requires extensive rules and experience that a large model alone cannot provide.

A Large Language Model (LLM) is a deep learning model trained on massive text data that generates text by predicting the next token. Its core architecture is the Transformer, with parameter counts typically ranging from billions to hundreds of billions. Open-source models like Llama, Qwen, and DeepSeek^Ad, as well as closed-source models like GPT and Claude, all belong to this category.

From a testing perspective, LLMs have several key characteristics: probabilistic output (the same input may produce different outputs), sensitivity to prompts, hallucination (generating plausible but factually incorrect content), and limited context length. These characteristics are the root cause of why traditional assertion-based testing fails, and why evaluation must introduce a scoring mechanism. When evaluating LLMs, dimensions such as accuracy, relevance, fluency, and safety are typically considered, completed through manual scoring or automated methods like LLM-as-a-Judge.

Stage Two: Knowledge Base (RAG)

RAG stands for Retrieval-Augmented Generation, which provides AI with precise rule-based grounding. The speaker gave an example of a fresh food refund: if a fish bought online arrives and one has spoiled, whether to refund and how much depends on the rules in the knowledge base—within how long after signing for fresh food to inspect, what proportion of spoilage, and at what ratio to refund. With a knowledge base, AI can handle complex after-sales service rather than merely comforting users without solving the problem.

RAG evaluation must cover three parts: retrieval quality, augmented generation quality, and whether conclusions have precise grounding. The speaker specifically pointed out an intriguing scenario: if three fish were bought and one spoiled, is a full refund reasonable? Is a one-third refund also reasonable? The key lies in what knowledge it read—this is exactly what needs to be evaluated.

Retrieval-Augmented Generation (RAG) is a technical architecture that combines an external knowledge base with a large language model. Its process typically is: after a user asks a question, the system first vectorizes the question, retrieves the most relevant document fragments from a vector database, then inserts these fragments as context into the prompt for the large model to generate the final answer.

RAG solves problems such as outdated knowledge in large models, inability to access private data, and susceptibility to hallucination, enabling AI to answer based on enterprise rules and real-time documents. When evaluating RAG, the retrieval stage (recall, precision, ranking quality) and the generation stage (faithfulness, answer relevance, fidelity to retrieved evidence) must be examined separately. Common automated evaluation frameworks include RAGAS and TruLens, which provide quantitative scores by decomposing questions and comparing evidence rather than simply judging right or wrong.

Stage Three: Agent

Having knowledge grounding is not enough—the act of refunding needs someone to execute it. The agent is responsible for doing the actual work: initiating the refund and transferring the money back to the user's account. The questions here become: did it do it, did it do it correctly, did it finish, and did it disclose anything extra during the process. The speaker emphasized that even if the knowledge grounding is fine, the correctness of the execution stage also needs evaluation.

An Agent refers to a system that uses a large language model as its decision-making core and can autonomously plan, call tools, and execute multi-step actions to accomplish goals. Unlike single-turn Q&A, an Agent has memory, tool-calling capabilities (such as APIs, databases, payment interfaces), and task decomposition abilities; common implementation patterns include ReAct (alternating reasoning and acting) and Plan-and-Execute.

In the refund scenario, the Agent needs to understand the user's request, query the order, determine the refund ratio based on the knowledge base, call the payment interface to execute the refund, and finally provide feedback to the user. The difficulty in evaluating Agents lies in the fact that their behavior is sequential and the path is not unique, so a comprehensive score is needed across dimensions such as task completion rate, action correctness, step efficiency, and safety (whether it exceeds authority or leaks information), often measured through trajectory evaluation and end-to-end task success rate.

What You Need to Get Started

The talk provided a practical minimum setup: Python, Node, and a large model interface. Installation packages are all double-click to run. Large model interfaces are currently basically not free; at work they are usually provided by the company. If you're playing on your own, DeepSeek starts at one yuan. The interface requirement is OpenAI compatibility, which mainstream domestic vendors such as Alibaba, MiniMax, Zhipu, and DeepSeek all satisfy—so you can consider this condition automatically met.

An OpenAI-compatible interface refers to an HTTP interface that follows the OpenAI API specification, including identical endpoint paths (such as /v1/chat/completions), request and response JSON structures, and authentication methods (Bearer Token). This specification has become a de facto standard, with platforms like Alibaba Cloud Bailian, Zhipu GLM, MiniMax, and DeepSeek all providing compatible endpoints.

For testers, this means mastering just one SDK (such as the official openai Python package) allows switching between models from different vendors without adapting to each one individually. To get started, you can configure the API Key as an environment variable and complete calling experiments with a small cost; in enterprise environments, the platform team usually provides a unified gateway and quota, and testers only need to request the interface address and key to start building evaluation scripts.

Key Conclusions

The core contribution of this talk is breaking down the seemingly enormous field of AI evaluation into three actionable levels: first understand large models, then progress to knowledge bases, and finally to agents. The evaluation targets are exactly these three, corresponding to the English abbreviations LM, RAG, and Agent. How to specifically test involves tools, methods, and scenario cases, which the speaker said would be covered in subsequent content.

For testing practitioners, the most memorable point is: AI software is no longer just "functionally OK"—it also needs to be evaluated in terms of intelligence, capabilities, and behavioral constraints, and these requirements often cannot be simply asserted as right or wrong but must be judged through scoring.