A New Paradigm for AI Testing: The Mindset Shift from Assertions to Evaluations

AI Is Reshaping the Boundaries of Software Testing

Over the past six months, AI's influence on software development and testing has evolved from being a "supporting tool" to becoming a "driving force." A senior test development engineer candidly shared that they started out primarily using Python, but their projects have since shifted to JavaScript — the industry's rapid transformation is forcing practitioners to keep learning continuously. The boundary between testers and developers is blurring: testers are doing development work, and developers must learn testing. This trend is irreversible.

More notably, the testing field is forming a comprehensive AI testing framework, which can be summarized into three directions: testing with AI, testing alongside AI, and testing AI itself.

The Evolution of Three AI Testing Approaches

Testing with AI means leveraging AI to assist with testing tasks and improve efficiency. This was once the hottest topic in testing, but it has gradually become a baseline skill.

Testing alongside AI emphasizes AI leading the testing process. Humans and AI each play to their strengths, with most testing tasks — including requirements analysis, test case design, system exploration, code writing, and automated regression — delegated to AI. Companies like Tencent, Alibaba, ByteDance, and DeepSeek are actively driving this model into production.

Testing AI itself is the most forward-looking direction. As more and more software integrates AI capabilities — such as AI customer service and AI video generation — how to test these "AI-powered applications" becomes an entirely new challenge. The approach, process, and methodology for this type of testing are fundamentally different from traditional software testing.

From Assertions to Evaluations: The Core Transformation in AI Testing

The core transformation in testing AI lies in a shift in terminology and mindset — from "assertions" (Test) to "evaluations" (Evaluation). This isn't the invention of any single team; it's internationally established practice.

Think of it this way: you can't simply judge whether a person is good or bad — that kind of binary judgment has enormous limitations. Assessing someone's intelligence or diagnosing a disease is done through scoring. Since AI is modeled after human behavior, evaluating it should naturally follow the same approach: scoring and assessment.

The Fundamental Difference: Determinism vs. Probabilism

This shift is grounded in deep technical reasoning.

In traditional software testing, we write assertions directly — asserting status codes, return data, and field values. Outputs are largely consistent, assertions are stable and reliable, and the verdict is simply "pass" or "fail."

But LLMs are inherently probabilistic, and this manifests at two levels:

At the architectural level: Most mainstream generative LLMs are built on the Transformer architecture, and their outputs are fundamentally probabilistic choices. The same question may yield different answers — this randomness is baked into the architecture.

At the output level: Even when the semantically correct answer is clear (say, the number "2"), it can be expressed in countless ways — Roman numerals, words, Arabic numerals, binary, hexadecimal, or various stylistic phrasings. There may be only one "correct" answer, but there are infinite ways to be wrong. Moreover, any minor change to model parameters, prompts, or context can dramatically alter the result, making simple equality assertions obsolete.

The key mechanisms behind probabilistic outputs in Transformer architectures are the temperature parameter and sampling strategies. When generating each token, the model computes a probability distribution over all candidate words in the vocabulary, then samples from that distribution to select the final output. Higher temperature increases sampling randomness; temperature set to 0 tends toward the highest-probability choice, but still cannot guarantee identical outputs across runs. Parameters like Top-P (nucleus sampling) and Top-K also affect output diversity. This is why model parameters are treated as an independent management dimension in the "system under test" — adjusting Temperature alone can flip a test result from "pass" to "fail," and must be treated as a version change within the evaluation framework.

Evaluation Doesn't Mean Abandoning Testing

To be clear, evaluation does not fully replace testing. Evaluation emphasizes scoring — measuring the quality of results on a scale; the testing stage is equivalent to setting a passing threshold for that score.

Just as college admissions have cutoff scores and tumor diagnoses have score thresholds, whether an AI product can go live ultimately still requires a definitive conclusion. That conclusion is determined by whether the evaluation score clears the passing threshold.

In practice, the AI field typically uses continuous values from 0 to 1 (e.g., 0.1, 0.9, 0.999). These can also be discretized: 0 for fail and 1 for full marks, making it compatible with traditional assertion-based testing.

Key Practices in AI Evaluation

Dual Version Management: Ensuring Comparability

Because evaluation results are inherently fuzzy and variable, version management is held to a much stricter standard than in traditional testing. In practice, two versions must be fixed:

• System Under Test (SUT) version: The version of the system being tested, defined by the model in use, model parameters, prompts, and context. Any change to any of these constitutes a version change. For example, switching from DeepSeek to GPT represents an entirely different SUT version.

• Evaluation version: The means used to conduct evaluation, including datasets, assertion rules, prompts, and pass/fail thresholds. Changes to the dataset or threshold will produce different evaluation results.

Meaningful lateral comparisons (e.g., comparing quality across different models, or the effect of different thresholds) are only valid when exactly one version is swapped at a time. If both versions change simultaneously, the results lose comparability and become an entirely independent evaluation.

Analogy to Performance Testing

AI evaluation closely mirrors performance testing: performance testing also doesn't focus on whether a single request is "right" or "wrong" — instead, it surfaces issues through a dataset under specific conditions, requiring multiple rounds of comparison to draw conclusions. Engineers with performance testing experience will find AI evaluation more intuitive to grasp.

The Three Layers of AI Evaluation and Tool Selection

Three Progressively Complex Evaluation Targets

AI evaluation isn't about stacking metrics (like faithfulness or recall rate) from the start. The first step is to identify the evaluation target — different targets require completely different goals and metrics. Common evaluation targets fall into three layers, in increasing order of complexity:

Layer 1: Large Language Model (LLM) Evaluation

Assesses the correctness, completeness, safety, and consistency of model outputs. For model vendors like DeepSeek, this is a mandatory evaluation; for other organizations, it's used for model selection, prompt optimization, and post-fine-tuning quality verification.

Layer 2: Knowledge Base (RAG) Evaluation

RAG works by first retrieving relevant information, then answering questions using that information. Evaluation focuses on each step in the pipeline — retrieval accuracy, faithfulness (whether the generated output is grounded in the specified sources), and so on. Metrics like recall rate apply primarily at this layer.

Layer 3: Agent Evaluation

Agents encompass capabilities such as task planning, tool usage, and task execution. Customer service agents and data agents are already widely deployed, with expansion into travel, legal, and medical domains on the horizon. Agent evaluation is the most complex, as it simultaneously involves tool selection, task orchestration, knowledge base usage, and LLM invocation.

These three layers aren't a "pick one" choice — they form a progressive learning path: start with LLM evaluation, move to RAG evaluation, then advance to agent evaluation, with complexity increasing at each level.

AI Evaluation Tool Recommendations

When it comes to tool selection, PromptFoo is worth highlighting. Its key advantages include:

• CLI-driven: Supports command-line and YAML configuration, making it callable by AI itself — paving the way for "AI evaluating AI" scenarios
• Supports human evaluation: Human evaluation remains the "gold standard" in AI assessment; the tool allows human-authored test cases and assertions via YAML configuration
• Highly extensible: Supports batch invocation and Python custom extensions
• Acquired by OpenAI: Its technical capabilities have been validated by a top AI organization
• Comprehensive coverage: Covers all three evaluation layers — LLM, RAG, and agent

For purely academic metric research, tools focused on metric computation may be more appropriate. For production monitoring of live agents, tools oriented toward real-world performance evaluation are a better fit. But for most test engineers, whose goal is typically to make a release-readiness judgment, tools like PromptFoo are the most practical choice.

LLM-as-a-Judge is an important paradigm shift in AI evaluation: using a judge LLM to score and assess the outputs of the model under test, replacing or supplementing human evaluation. Its advantage lies in massive parallelism at a cost far lower than human review; the challenge is that judge models carry their own biases (e.g., a tendency to favor longer answers), which must be mitigated through multi-judge voting, human spot-checking, and similar methods. PromptFoo supports configuring any LLM as a judge, allowing LLM-as-a-Judge workflows to be defined declaratively via YAML — which is the core value of its CLI-driven architecture.

Direction Matters More Than Execution

Understanding the directional thinking behind "why to test, what to test, and what to test with" is far more important than mastering specific tool operations or script writing. Tools and scripts can be picked up quickly, but a solid evaluation mindset is the true core value.

For testing practitioners navigating industry transformation, mastering the shift from assertions to evaluations, understanding the three layers of AI evaluation targets, and developing dual version management awareness — these are the core competencies for thriving in the age of AI.

Background: A Note on RAG

RAG (Retrieval-Augmented Generation) is currently the most mainstream technical approach for deploying LLM applications in enterprise settings. The core idea is: without modifying the model's weights, retrieve relevant snippets from an external knowledge base (such as enterprise documents or databases) at query time, then inject those snippets into the prompt as context, so the model "answers with the reference material in hand." This allows the introduction of knowledge beyond the model's training cutoff while effectively reducing hallucinations.

The challenge in RAG evaluation lies in its multi-stage pipeline of "retrieval → ranking → generation": the retrieval stage is measured by recall and precision; the generation stage is measured by metrics such as Faithfulness (whether the answer stays true to the retrieved source material rather than improvising). A problem at any stage degrades the final answer quality, so root cause analysis must be done per stage rather than only examining the final output.