AI Testing Transformation Guide: From Testing AI to Testing with AI — An Evaluation Playbook

AI testing is undergoing a quiet paradigm shift. In the past, we were used to judging right from wrong with assertions. Today, when facing large models, knowledge bases, and agents, the traditional binary logic of "pass/fail" begins to fail—scores, thresholds, and human judgment have become the new language of measurement. A practical sharing session on large model evaluation on Bilibili happened to explain this shift thoroughly.

Why AI testing should be renamed "evaluation"

The core of traditional functional testing is assertion: deterministic input, predictable output, clear right or wrong. But the output of AI systems is inherently probabilistic and fuzzy—the same question may get different answers twice, and a "partially correct" answer is far more common than a "completely wrong" one.

The speaker described this state as "a transitional role between right and wrong" and pointed out that this is exactly where the term "evaluation" comes from—using scores to express degree rather than Boolean values to express conclusions. This terminological distinction is not wordplay; it determines the choice of the subsequent toolchain: what you need is scorers, evaluation sets, and statistical analysis, not a simple assertion library.

From an engineering perspective, this shift means test engineers need to supplement probabilistic thinking and dataset design skills. Assertion thinking focuses on "does this test case pass," while evaluation thinking focuses on "how does this batch of samples perform overall, and where is the confidence interval." The working methods of the two are completely different.

The three layers of AI evaluation: large models, knowledge bases, agents

The speaker summarized the evaluation objects into three progressive layers. This division is worth borrowing because it corresponds to the real growth path of AI application complexity.

Layer 1: Large Language Models

The most basic form is integrating a large model to provide chat capabilities. The speaker took the AI chat feature of an e-commerce platform as an example, pointing out that such applications have a low barrier to entry, but their capabilities are limited to "companionship chat"—they can understand emotions and offer comfort, but cannot execute specific business actions such as refunds or after-sales service.

Evaluating this layer focuses on basic language understanding and generation quality: whether answers are relevant, accurate, and free of hallucinations. This is the layer most easily quantified and the scenario most commonly covered first by evaluation tools.

Hallucination refers to content generated by large models that appears plausible but is inconsistent with facts or input. It is the most frequently mentioned defect type when evaluating language models. Its cause lies in the model generating tokens probabilistically, prioritizing language fluency over factual correctness, so it tends to "fabricate" details when lacking external evidence. Evaluating hallucination usually requires building evaluation sets with standard answers, or cross-verifying through retrieval augmentation, citation validation, and other means. For testers, identifying hallucination cannot rely solely on a single output; it is necessary to design multi-turn, multi-angle probing test cases and distinguish "factual hallucination" from "faithfulness hallucination"—the former violates world knowledge, while the latter deviates from the given context.

Layer 2: RAG Knowledge Base

When the business requires specific rules, a large model alone is not enough. The speaker gave a vivid example: after receiving frozen fresh fish from an online purchase, one fish was found thawed. Whether a refund should be given and how much depends on whether the fresh category has clear after-sales rules—for example, "thawing ratio corresponds to refund ratio." Such rules must be placed in a knowledge base so that AI can provide a grounded response.

RAG evaluation is therefore split into three parts: retrieval quality (can the correct rule be found), generation quality (can the rule be expressed correctly), and grounding consistency (does the conclusion have an exact source). The speaker specifically mentioned cases like "three fish, one bad, yet a full refund"—both a full refund and a one-third refund seem intuitively plausible; the difference lies only in which piece of knowledge the AI read. This is precisely the core scenario for evaluating the value of a knowledge base: judging the reasonableness of the output must trace back to the retrieval step.

RAG (Retrieval-Augmented Generation) is a technical architecture that combines external knowledge retrieval with large model generation: the system first retrieves snippets related to the question from a knowledge base, then passes the snippets as context to the model to generate an answer. It solves the problems of outdated knowledge and inability to access private data in large models, but also introduces new failure modes—failure to retrieve, incorrect retrieval, and ignoring retrieval results during generation. Therefore, RAG evaluation is naturally divided into two stages, retrieval and generation, and requires a "grounding consistency" metric to confirm that the final answer indeed comes from the retrieved snippets rather than model fabrication. Understanding this pipeline structure is a prerequisite for designing RAG evaluation test cases.

Layer 3: Agents

Having knowledge grounding is not enough; the refund action ultimately needs someone to execute it. The agent takes on the role of "doing things." The speaker pointed out that the evaluation dimensions of this layer are fundamentally different from the previous two: not only whether the decision is correct, but also whether the action was executed, whether it was executed completely, and whether there was any additional disclosure during the process.

In other words, agent evaluation is a composite evaluation of "decision quality + execution quality," the most difficult and closest to real business risk.

Three ways to combine testing and AI

The speaker reviewed the three integration paths discussed earlier. This framework helps understand the current stage:

• Testing with AI: AI serves as an auxiliary tool to help people test traditional software; humans are the main actors;
• Letting AI test: AI takes on the main testing work, with humans in a supporting role;
• Testing AI: AI becomes the object under test, forming a peer testing relationship between humans and AI.

The first two have been widely implemented in practice, while the third is the new battlefield today. The speaker's judgment is that as AI capabilities are embedded in various types of software—from telecom operator apps to code assistants—testers will sooner or later face the reality that "the system under test contains AI," even if they do not actively embrace AI.

Environment preparation and tooling threshold

The requirements for the practical part are quite modest: a Python and Node environment, plus an OpenAI-compatible large model API. The speaker mentioned that major domestic vendors (Alibaba, MiniMax, Zhipu, DeepSeek^Ad, etc.) are basically compatible with the OpenAI protocol, so the API conditions are usually automatically met.

Interestingly, there is the cost issue—the speaker bluntly stated that "there are basically no free large models now" and suggested DeepSeek's one-yuan minimum top-up as a low-cost solution for individual practice, while noting that in enterprise scenarios the company usually provides the API, so individuals need not worry excessively. This reminder is very pragmatic: evaluation work often requires batch API calls, and token consumption cannot be underestimated; establishing cost awareness in advance is necessary.

OpenAI-compatible API refers to HTTP interfaces that follow API specifications such as OpenAI Chat Completions, including request paths, message role structures, and response field formats. Domestic vendors generally provide compatibility layers, so the same client code only needs to replace base_url and api_key to switch models. This is of great significance for evaluation work: evaluation frameworks usually rely on a unified calling protocol to run test cases in batches, and compatible APIs make "switching models for comparison" a low-cost operation. In practice, note that the compatibility layer may not support all parameters (such as certain function calling or structured output options); before evaluation, verify interface behavior consistency with a small sample.

Final thoughts

The value of this sharing session is not in teaching a specific tool, but in clarifying the cognitive framework of AI evaluation: from assertion to scoring, from a single model to the three-layer progression of model–knowledge base–agent, and from auxiliary tool to object under test. For testers who want to enter this direction, establishing this coordinate system first and then choosing tools to go deeper is more efficient than blindly piling up technology stacks.

The mind map and outline materials mentioned by the speaker can be obtained from the teaching assistant in the comments section and can serve as an index for further learning.

One-line Summary

AI testing shifts from assertions to scoring, with evaluation objects divided into three layers: large models, knowledge bases, and agents.

Paragraph Summary

Based on a practical sharing session on large model evaluation on Bilibili, the article outlines the paradigm shift in AI testing: traditional functional testing relies on deterministic assertions, while AI outputs are probabilistic and fuzzy, so scores, thresholds, and human judgment are needed to express "partially correct." The evaluation objects are divided into a three-layer progressive structure—large language models, RAG knowledge bases, and agents—with complexity and business risk rising layer by layer, and evaluation focus respectively falling on language quality, retrieval and grounding consistency, and the composite quality of decision-making plus execution. The article also reviews the three integration paths of testing with AI, letting AI test, and testing AI, and provides pragmatic suggestions on environment preparation and cost control, emphasizing establishing a cognitive framework before choosing tools to go deeper.