AI Large Model Testing in Practice: The Mindset Shift from Assertions to Evaluation

Three Testing Paradigms in the AI Era

AI is redrawing the boundary between development and testing — testers are starting to write production code, while developers are learning how to test. One content creator who transitioned from test development to pure development, switching from Python to JavaScript along the way, shared a systematic AI testing methodology on Bilibili. At its core is a three-paradigm framework he proposed six months ago, which has since taken much clearer shape.

The first paradigm is testing with AI — using AI to assist in testing and boost efficiency. Six months ago this was the hottest topic in the testing world, but it's already become old news. The second is testing alongside AI (AI-led testing), where humans and AI each play to their strengths. Most of the work — requirements analysis, test case design, system exploration, code writing, automated regression — can be handed off to AI. This is precisely the AI-led testing model being pushed by Tencent, Alibaba, ByteDance, and a DeepSeek^Ad that has pivoted toward engineering research.

The third paradigm — and the focus of this article — is testing AI itself (AI application testing). As AI customer service from China Mobile and China Telecom, AI fortune-telling, AI video generation, and countless other applications become increasingly common, more and more software has AI capabilities baked in. This software needs to be tested too, but the mindset, workflow, and approach are entirely different from traditional software testing.

Why "Testing" Becomes "Evaluation"

Traditional software testing relies on assertions: assert the status code, assert the returned data, assert the field value. Results are either right or wrong, either pass or fail — there's no third option. This works because traditional software output is deterministic — an ID is what it is, the output is consistent and reliable.

But AI large models are probabilistic, which is why assertions no longer work. There are two lenses through which to understand this probabilistic nature:

At the architecture level, mainstream generative large models mostly follow the Transformer architecture, and Transformer output is fundamentally a probabilistic selection process. Large models are inherently probabilistic — the same question may yield different answers each time.

At the output level, even when a result is logically determinate (say, the answer is the number 2), it can be expressed in countless ways: Chinese uppercase 贰, lowercase 二, English "two", Arabic numeral 2, binary, hexadecimal, ASCII code... And that's before accounting for the fact that any minor change to model parameters or the prompt can cause dramatic shifts in output.

For all these reasons, simple equality/inequality assertions become inappropriate — the equal case might be right, but there are tens of millions of possible unequal cases. A new approach is needed: evaluation. Rather than judging "right or wrong," evaluation uses scoring to express "how good or how bad."

This logic is actually the international standard. Just as you can't simply label a person "good" or "bad," we assess intelligence with exam scores and diagnose complex conditions like cancer or depression with scoring systems. Applying score-based evaluation to AI — which mimics human behavior — is the most reasonable approach across all cultures and throughout history for handling complex domains.

Evaluation and Testing Are Not Mutually Exclusive

It's worth clarifying: the field of AI evaluation doesn't use only evaluation and abandon testing. Evaluation handles the scoring; testing sets the "passing grade" for those scores — just as the college entrance exam has admissions cutoffs, and tumor or depression diagnoses have score thresholds. Whether a product can ship still ultimately requires a clear go/no-go decision based on the evaluation score's passing threshold.

Scores are typically expressed as continuous values between 0 and 1 (0.1, 0.2, ... 0.999), but they can also be discretized: 0 means fail (0 points), 1 means perfect (pass) — effectively returning to a traditional assertion-style judgment. Both continuous scoring and traditional assertions are used in real-world projects.

---

The Transformer architecture's influence on large model probabilistic behavior deserves further elaboration. When generating each token, Transformer computes a probability distribution over all candidate words in the vocabulary, then selects from that distribution via a "sampling" strategy — rather than mechanically picking the highest-probability word. The key parameter controlling this randomness is Temperature: higher temperature means more random, diverse outputs; at temperature 0, the model always picks the highest-probability word and output becomes nearly deterministic. Other sampling parameters like Top-p and Top-k also affect output diversity. This means that even with identical model weights and the same prompt, any non-zero Temperature means successive calls may differ at the literal level. This is precisely why "version pinning" requires model parameters to be treated as part of the version under test — any parameter change shifts the output distribution and makes two evaluation runs incomparable.

---

Version Pinning: The Strict Prerequisite for AI Evaluation

Traditional testing has the concept of version numbers — which version you're testing, which version has a bug, what changed between versions. AI evaluation is more complex: because it's fuzzy and potentially volatile, it demands stricter versioning. Specifically, two versions must be pinned:

The version under test: the version of the system being tested, defined by the model in use, the parameters passed in, the prompt, and the context. Any change to any of these means the version under test has changed. For example, if you're testing DeepSeek and switch to GPT, you've changed the version — the results need to be re-interpreted.

The evaluation version: who tested it and how, including the dataset, assertions, prompts, and thresholds. Testing the same system with a different AI or a different dataset will yield different scores — just like getting different results on the Jiangsu provincial exam versus the national exam.

Only by pinning both versions and swapping one at a time can you make meaningful comparisons: the same test methodology applied to different models lets you compare model quality; the same system under test with different thresholds lets you compare threshold effectiveness. If both versions change at once, you have two completely independent evaluations that can't be compared.

An Analogy to Performance Testing

Here's a practical analogy: AI evaluation is highly similar to performance testing. Performance testing similarly doesn't focus much on right or wrong — it uncovers issues under specific conditions using a set of data, and after finding issues, you don't run just one round. You run two rounds, compare results, and submit the comparison. Anyone familiar with performance testing can map those concepts and ideas directly onto AI evaluation.

What to Evaluate: LLM, Knowledge Base, and Agent — Three Progressive Layers

Many people assume AI evaluation is just about stacking up a bunch of metrics (faithfulness, recall rate, etc.), but the key is to first identify the evaluation target, because the goals and metrics differ entirely depending on what you're evaluating. There are three common evaluation targets, and they form a progressive hierarchy:

Large Language Models (LLMs): Evaluating the correctness, completeness, safety, and stability of generated content. For companies like DeepSeek, this is a must-test item. For most companies, it's used for model selection (which is better, faster, smarter), prompt optimization (based on actual results rather than gut feeling), and verifying the effects of fine-tuning — now that Qwen and similar models have released many locally fine-tunable small models, post-fine-tuning evaluation is essential.

Knowledge Bases (RAG): The principle is to first retrieve the right reference material, then answer the question using that material. Evaluation needs to cover each stage — whether retrieval is correct, accuracy, faithfulness, and whether the final answer is correct. Recall rate and faithfulness metrics are primarily used here; faithfulness refers to whether the generated answer is supported by the retrieved reference material.

Agents: Customer service agents, data agents, and knowledge base agents are already common, with travel, legal, and medical agents on the horizon. Agent architecture involves task planning, tool use, and task execution — evaluation content and metrics are different again.

These three are progressive, not three alternatives: agents typically have knowledge bases attached, and both knowledge bases and agents rely on large models. This makes the learning path clear — first master LLM evaluation, then knowledge base evaluation, and finally the most complex agent evaluation (encompassing tool selection, task orchestration, and knowledge base usage).

What to Use: Choosing promptfoo

With a solid methodology in place, you need the right tooling. promptfoo comes highly recommended for good reasons:

It is a CLI and YAML-driven tool. CLI support means it can be invoked by AI itself. This tool for evaluating AI can in turn be called by AI — making "AI evaluating AI" a promising future scenario. It also supports human evaluation — in the current industry consensus, AI is not yet smarter than humans, so human evaluation remains the gold standard. Writing test cases and assertions in YAML aligns with the philosophy of low-code/no-code framework design.

Beyond that, it supports batch calls, Python-based custom extensions, and has strong extensibility and promising prospects. It has been acquired by OpenAI, a validation of its quality. And it covers all three evaluation targets: LLM, knowledge base, and Agent.

For different scenarios, tooling choices can be more flexible: pure academic metric research (like evaluating DeepSeek's model metric data) or user-facing online agents (evaluated through production monitoring of real-world performance) each have more suitable tools. But for most testers — where evaluation is in service of testing, and testing is about determining whether a product is ready to ship (with acceptance testing implications) — promptfoo is the right entry-level choice.

---

prompfoo's core workflow is driven by a YAML configuration file: users declare the model providers under test (e.g., OpenAI, Anthropic, local Ollama), the prompt templates, the test dataset, and the assertion rules (supporting multiple assertion types including keyword matching, regex, semantic similarity, and calling another LLM as a judge). Running promptfoo eval automatically batch-calls the models, executes assertions, and generates a visual report — viewable in the browser to compare scores across different models or prompt versions. The LLM-as-a-Judge mode is particularly widely used — having a more capable model (like GPT-4) score the output of the model under test, enabling automated evaluation of open-ended Q&A. This mode directly echoes the "AI evaluating AI" trend mentioned earlier and is the technical foundation that allows promptfoo to cover all three evaluation layers: LLM, RAG, and Agent.

---

Closing Thoughts

This material may seem dry, but it's arguably the most valuable part of the entire AI testing framework — it addresses problems at the directional level, not just the execution level. Understanding the mindset shift from assertions to evaluation, the strict requirements of version pinning, and the progressive three-layer structure of evaluation targets means that even without hands-on practice, you can hold a meaningful conversation in an interview and have a clear direction when a real task lands on your desk. The tools and specific syntax are secondary, because all practical execution is built on top of this methodology.

Background: RAG Explained

RAG (Retrieval-Augmented Generation) is currently one of the most prevalent approaches for enterprise AI deployment. Its core workflow has two steps: first, external knowledge (documents, databases, etc.) is chunked and converted into vectors stored in a vector database; when a user asks a question, the system retrieves the most relevant chunks using the same vectorization approach, then sends those chunks along with the question to the large model, which generates an answer based on them. This mechanism allows the model to answer questions about domain knowledge outside its training data, while also reducing the risk of hallucination.

RAG evaluation therefore naturally splits into two phases: the retrieval phase (whether the recalled documents are relevant and cover the information needed to answer the question — corresponding to the Recall metric) and the generation phase (whether the model's answer faithfully reflects the retrieved reference content rather than fabricating information — corresponding to the Faithfulness metric). Both phases can fail, and the causes and remediation paths are entirely different — which is exactly why RAG evaluation needs to cover each stage separately.