AI-Led Testing in Practice: A Complete Guide to Building a Vibe Coding Test Workbench

From AI-Assisted to AI-Led: A Paradigm Shift in Software Testing

The way AI is applied in software testing is undergoing a fundamental transformation. According to content shared by instructor Beima in his software testing bootcamp on Bilibili, the industry's central question is no longer "should we use AI?" but rather "can AI lead the testing process?" This shift deserves serious attention from every testing professional.

Many people fall into one of two traps. The first: "I use AI at work, so that counts as AI testing." This is still firmly in the AI-assisted testing stage. The second trap swings to the opposite extreme — handing everything over to AI, from online monitoring to CI/CD regression testing. This approach carries enormous costs in both time and API tokens, and is equally problematic.

The author uses a clever analogy to illustrate the difference between AI-assisted and AI-led: the evolution of autonomous driving. Cars have had assisted driving features for years — lane keeping, cruise control — but your hands couldn't leave the wheel; you were still the primary driver. True AI-led operation is like hands-free autonomous driving, where AI handles the vast majority of the work and humans only intervene at critical moments.

Three Core Characteristics of AI-Led Testing

The author distills AI-led testing into three defining characteristics:

• AI as primary, humans as support: This is the most fundamental repositioning — humans are no longer executors but supervisors and decision-makers.
• AI can independently complete the entire task workflow: AI handles the full cycle — designing test cases, executing tests, generating reports — and only then hands results to humans for review or decisions. If humans and AI are taking turns, it's not truly AI-led.
• Requires systematic orchestration ability: Delegating development, testing, translation, and other work to AI is feasible, but it demands that the human operator possess process control and result review capabilities.

Interestingly, the author doesn't dismiss the value of traditional test automation techniques. These skills remain useful — their value has simply narrowed to the regression testing phase. In other words, the way work gets done has changed, but the underlying technical foundation still matters.

Building the AI Test Workbench: Giving the Brain a Pair of Arms

Having a smart large language model alone isn't enough. DeepSeek and other models are genuinely intelligent, but can they control a browser? Execute API tests? Not necessarily.

The author frames it vividly: building an AI test workbench is like giving the AI brain a pair of arms. Only then can the test cases the AI designs actually be executed, results observed, and the testing plan iteratively improved. The workbench is structured across five layers.

IDE: The Unified Window for All Work

The first layer is the IDE. The author emphasizes there's no strict tool requirement here — you can use your existing editor or VS Code. VS Code holds a commanding position in the IDE market and is excellent, but it's worth noting: VS Code is an IDE, not an agent.

The IDE's core roles include: displaying project structure, providing a terminal for running commands, and one particularly practical feature — comparing the diff between each iteration of AI-generated output. For example, when you ask AI to redesign a test case, you can immediately see whether it actually redesigned it or just recycled the old one. What changed? The IDE's diff view makes it immediately obvious.

CLI Agent: Why Claude Code

The centerpiece of the workbench is the CLI agent, and the author's top recommendation is Claude Code. The reasoning is straightforward: among all companies applying AI technology today, Anthropic (Claude Code's developer) leads the field globally. Its prompt engineering, built-in tools, and agent processing logic are all top-tier.

Regarding the controversy around Claude Code's "restrictions" for Chinese users, the author clarifies: Claude Code detects Chinese users and blocks their accounts because it does not offer service in China. But the author's solution cleverly sidesteps this issue: use only Claude Code's client tooling, but swap in a domestic model like DeepSeek. This way, data stays within China, there are no legal or political risks, and you don't have to worry about account bans — DeepSeek and Alibaba Cloud won't block you.

The author also mentions alternatives: Tencent's CodeBuddy, Codex, and various agent server-style tools. His assessment is that these "cannot yet surpass Claude Code" for now. That said, if your company explicitly prohibits Claude Code, these are viable fallbacks.

Anthropic is an AI safety company founded by Dario Amodei, former VP of Research at OpenAI. The Claude family of models is its flagship product. Claude Code is a command-line agent tool designed specifically for development and engineering tasks, with built-in capabilities for file read/write, terminal command execution, and code search — enabling it to autonomously complete multi-step tasks within a local project directory. Unlike conversational products such as ChatGPT, a CLI (Command Line Interface) agent's key advantage is the ability to directly operate the local file system, run shell commands, and invoke testing frameworks — forming a closed loop of "think → execute → observe → reflect." This is the technical foundation that makes AI-led end-to-end testing possible.

Model Selection and Configuration Switching

For the model layer, the author recommends DeepSeek V3 (the official release), citing good performance, fast speed, and low cost. Domestic alternatives like Qwen, Xiaomi, Zhipu, and MiniMax are all viable — any recently released version should work fine.

Since Claude Code is a foreign CLI tool that isn't easy to configure directly, you also need a configuration switching tool to connect DeepSeek to Claude Code. This is the critical link that makes the "best-in-class client + domestic model" combination work.

Accumulating Testing Assets

The final layer of the workbench is the integration of all outputs: automation code, requirements analysis documents, test execution logs, artifacts, and Skills. The author specifically notes that these assets don't need to be created manually — build the workbench properly and they accumulate organically through use, becoming invaluable assets for the entire testing project.

Hands-On Practice: Environment Setup and Tool Installation

With the theory covered, it's time for hands-on work. The author shares a valuable lesson learned: he had previously assumed all students had coding and version control backgrounds, until he discovered that one student with a purely functional testing background had never touched Git. That experience made clear why the fundamentals need to be covered thoroughly.

The Essential Three-Tool Setup

Three core tools need to be installed:

• Node and npm (npm installs alongside Node)
• Git (required, for version control)

Verification is simple: open a terminal and run the version query commands for node, npm, and git. If all three return version numbers, you're ready to proceed.

On Node versions, the author is clear: always choose the LTS (Long-Term Support) version (marked in blue on the download page), not the green "Latest" version. The latest is a short-term maintenance release that loses support relatively quickly. The recommended version is v24.19. The author candidly notes his own machine runs v24.18 (upgraded from an older version), adding that any v24.x release is fine — just don't use something like v14, which has already reached end-of-life (EOL).

Git is the world's most widely used distributed version control system, created by Linux kernel author Linus Torvalds in 2005. In an AI-led testing workflow, Git's role goes beyond code backup — it serves as a safety net for human-AI collaboration. Before asking AI to modify code or test cases, commit a version first. If the AI's changes introduce problems, you can use git diff to see exactly what changed and git checkout to instantly roll back to the last known-good state. The diff comparison feature in your IDE also relies on Git's change-tracking under the hood. For functional testers who have never used version control, think of Git as an "auto-save system with timestamps" — every AI action becomes traceable, reversible, and auditable.

Practical Details for Downloading and Installing

When downloading from the Node.js website, the author highlights a few details:

• Choose the right architecture: Mac users with M1/M2/M3/M4 chips should select the ARM build; older MacBooks (e.g., from 2015) should choose x64.
• NVM is optional: Some students mentioned using NVM (Node Version Manager) for managing multiple versions — the author agrees it's a great tool, but recommends beginners stick to the direct installer to keep things simple. "Download the blue-labeled LTS version and you'll be fine for the next 5 to 10 years."

Configuring npm Mirror Acceleration

After installing Node, there's one essential step: configuring a domestic npm mirror. Because npm's default registry is hosted overseas, direct downloads can be extremely slow — especially on mobile broadband connections. Running the mirror configuration command routes traffic through a domestic CDN, delivering a noticeable speed boost. The author emphasizes these commands can be run repeatedly without any conflicts or side effects.

npm (Node Package Manager) is Node.js's default package manager, connecting by default to registry.npmjs.org, which is hosted in the United States. Accessing this server from China is subject to network routing constraints, making slow downloads and timeouts a common problem. Configuring a domestic mirror means redirecting npm's download source to mirror servers hosted in China by providers like Alibaba Cloud or Tencent Cloud (e.g., npmmirror.com). These mirrors sync regularly with the official registry and are content-identical, but reduce access latency from several seconds to milliseconds. The configuration command is typically: npm config set registry https://registry.npmmirror.com. Once set, all subsequent npm install operations route through domestic CDN. For tools like Claude Code with many dependencies, installation time can drop from several minutes to tens of seconds.

Installing and Verifying Claude Code

With the environment ready, install Claude Code via npm. The author reminds students to use the command in conjunction with the mirror configuration set up earlier — otherwise downloading from overseas will be painfully slow. Any warn (warnings) during installation can be ignored; only error messages indicate a real problem.

After installation, verify the result. The author demonstrated his version upgrading from 2.20 to 2.27 — this simply means the latest version was installed and doesn't affect any existing configuration. He repeatedly reassures students: all commands can be safely run multiple times — if you make a mistake or miss a step, just re-run the commands in order and you'll be fine.

The final step is connecting a domestic model. Since Claude Code does not serve users in China (registration, login, and usage are all blocked), you'll need to integrate DeepSeek or another domestic model to actually power the workbench.

Capability Upgrade in the Age of Paradigm Shift

What this practical guide reveals is that AI-led testing isn't simply about "plugging AI into the pipeline" — it's a complete methodological reconstruction. It asks testing engineers to shift from executors to orchestrators, master the combination of "best-in-class client + domestic model," and develop the systemic ability to build workbenches, control processes, and review results.

For testing professionals, this transformation is both a challenge and an opportunity. Traditional automation skills won't become obsolete, but their scope of value is narrowing. The ability to command AI-led end-to-end testing will become the new dividing line that separates capability levels in the field.