The Year AI Takes the Lead in Testing: Building an AI Testing Workbench in Three Stages

From Assisted Driving to Autonomous Driving: A Paradigm Shift in the Testing Industry

According to a sharing by Bilibili creator Teacher Beima, the testing industry is undergoing a structural transformation similar to the evolution of cars from assisted driving to autonomous driving. He reminds practitioners to be wary of two common misconceptions: first, assuming that using AI in daily work means you are doing AI testing; second, handing over all work to AI. In his view, the real dividing line is not "whether AI is used," but rather whether AI plays a supporting role or a leading role.

He draws an analogy with autonomous driving: years ago, cruise control and lane keeping were just assisted driving, and your hands couldn't leave the steering wheel; today's robotaxis have achieved true "no one behind the wheel." The same applies to the testing industry — in the past, humans led with AI assisting, or humans and machines alternated in collaboration, and neither can be called AI-led testing. True leading-edge evolution requires AI to independently complete the entire task flow, with humans intervening and making decisions only at key nodes.

Three Core Characteristics of AI-Led Testing

While learning through trial and error in his actual teaching, Teacher Beima found that many students transitioning from functional testing have a skewed understanding of "leading." He offers three criteria: First, AI leads and humans assist, with the human's role shifting from executor to reviewer and decision-maker; Second, AI can independently complete the entire task flow, rather than repeatedly switching back and forth between human and machine; Third, humans need systematic coordination capabilities, performing process control and result review during AI execution.

These three points seem simple, but the reconstruction of working methods required for implementation is deep. In the past, you might have designed test cases yourself, executed them yourself, and used AI to help optimize; now you need AI to go from requirements analysis all the way through to case generation, execution, and result analysis, with humans stepping back to gatekeep. This role switch requires new tool support and a new capability model.

The Five-Layer Architecture of the AI Testing Workbench

A large model alone is not enough. Teacher Beima points out that DeepSeek^Ad or Doubao have a "brain" but no "hands and feet" — they cannot directly control a browser or execute API tests. Therefore, an AI testing workbench needs to be built to translate AI's thinking into concrete actions.

This workbench is divided into five layers:

The IDE layer is the window of the entire workbench, where all components are displayed together. It is used to display project structure, provide terminal execution commands, and compare the differences in each output produced by the agent. Teacher Beima particularly emphasizes the value of the comparison feature — when the agent redoes test cases, you can tell whether it is just going through the motions or actually improving.

The agent layer prioritizes the Claude Code client, but connects to DeepSeek for the model. According to him, the prompt engineering, built-in tools, and agent processing logic of the company behind Claude Code are among the top in the world. However, because that company explicitly does not provide services to China, direct registration carries the risk of account bans, so the solution is "use its client, connect a domestic model" — the model runs on DeepSeek, data stays in China, avoiding legal and political risks.

The configuration switching tool is used to solve the inconvenience of configuring Claude Code as a foreign command-line tool, and to connect DeepSeek into it.

The integrated results layer is the accumulated assets of the entire testing project, including automation code, requirements analysis documents, test case execution logs, and skills. These don't need to be created separately; they will gradually accumulate once the workbench is set up.

The model layer — Teacher Beima recommends DeepSeek's latest V4 official version, citing good results, fast speed, and low price. However, he also says that domestic newer models such as Qwen, Xiaomi, and MiniMax are all fine, with no strict restrictions.

Why do large models need "hands and feet"? A large language model (LLM) is essentially a text-to-text converter — it can only generate output based on input and cannot directly interact with the external world. To have AI actually execute testing tasks, it must be equipped with "Tool Use" capabilities — that is, enabling the model to issue structured instructions that external programs execute on its behalf, such as browser operations, API requests, and file reading/writing. This "brain + hands and feet" architecture is commonly called an Agent. The core loop of an agent is: observe the current state, plan the next action, invoke tools for execution, obtain result feedback, and continue reasoning. What Teacher Beima calls the "workbench" is essentially building a complete perception-decision-execution loop for the LLM.

Environment Preparation: Three Commands and Two Tools

Before starting, you need to confirm that Node.js, npm, and Git are installed. Teacher Beima reminds that many students who only do functional testing have never been exposed to version control or code environments — this is exactly where the course is prone to pitfalls. The check method is simple: open the command line and execute them in order; if the version number is displayed, you are ready.

The choice of Node.js version is key: you must choose the LTS long-term support version (such as v24.19), not the green latest version, because the latest version is a short-term maintenance release and may no longer be supported after a while. If you already have v24.18, that's also fine — it and v24.19 are both on the LTS line, but EOL versions like v14 are absolutely not acceptable. npm is installed automatically with Node, so there is no need to download it separately.

Git must be installed, and the version is not restricted. After installation, there is an essential step: configure the npm domestic mirror. Because npm is deployed abroad by default, downloads will be very slow without a mirror; using a domestic CDN for acceleration is standard practice, and the command can be executed repeatedly without conflicts.

Then install Claude Code via the npm command. The installation command can also be executed repeatedly, and already installed versions will be upgraded to the latest. Warnings that appear during the process can be ignored; only errors need to be handled.

The difference between Node.js LTS and Current versions is the most common pitfall for beginners. Node.js releases fall into two lines: LTS (Long Term Support) and Current (also called the latest version). LTS versions receive up to 30 months of security updates and maintenance, making them suitable for production environments; Current versions are maintained for only about 6 months and are mainly for developers to experience new features early. Therefore, when installing, be sure to choose the version marked "LTS" on the official website, not the most prominent green "Current" button on the homepage. For building a testing workbench, stability is far more important than trying new things — a Node version that is no longer maintained may cause dependencies to fail to install or produce hard-to-diagnose errors at runtime.

Practical Considerations for Tool Selection

Teacher Beima mentions that there are other agent clients on the market, such as Tencent's and some "agent server" type products, but in his view they cannot yet surpass Claude Code's agent level in production environments. If a company forbids its use, you can settle for alternatives such as CodeX, but his advice is "use the good stuff when conditions allow."

This approach is actually worth learning from: the choice of toolchain does not have to be a binary between "fully autonomous and controllable" and "use the best." Through an architecture that separates the client from the model, you can retain top-tier tool capabilities while keeping data and models within compliant boundaries. For practitioners who want to transition from traditional automated testing to AI testing, getting the environment set up and the workbench running is more valuable than waiting and agonizing.

Why can the Claude Code client switch its model to DeepSeek? This involves a key design of command-line agent tools: the client and model communicate through a standardized API protocol, rather than being hard-coded together. Although Claude Code is developed by Anthropic, the underlying model interface it calls follows general specifications, and users can point requests to other API endpoints compatible with the OpenAI format via environment variables or configuration files. This means the client's prompt engineering, tool orchestration, context management, and other capabilities can exist independently of the model. For domestic users, this "Claude Code front end, DeepSeek back end" solution avoids the compliance and account-ban risks of directly using overseas services, while retaining the top client's engineering accumulation in task planning and tool invocation.