AI-Led Testing in Practice: A Complete Guide to Building an Automated Testing Workbench

From AI-Assisted to AI-Led: A Paradigm Shift in Software Testing

Software testing is undergoing a profound transformation. If the past was the era of "AI-assisted" testing, we are now entering a new age of AI-led testing. The essence of this shift is like moving from "assisted driving" to "autonomous driving" — AI no longer just offers suggestions; it handles the bulk of the work, with humans stepping in only at critical decision points.

Two Common Misconceptions

Many testing professionals hold inaccurate views about AI-driven testing:

Misconception #1: Using AI as a helper tool equals AI testing. This mindset is still firmly in the "AI-assisted" camp — it's not true AI-led testing.

Misconception #2: Blindly handing everything over to AI. This includes relying on AI for online monitoring, CI/CD regression automation, and everything in between. This is equally problematic — every action AI takes costs time and tokens. Delegating everything without control is neither economical nor practical.

Three Core Characteristics of AI-Led Testing

True AI-led testing should have these qualities:

1. AI leads, humans assist — this is the most fundamental distinction from AI-assisted testing
2. AI can independently complete entire task workflows — AI executes tasks end-to-end, then hands off to humans for review or decision-making
3. Humans must develop systemic orchestration skills — maintaining process control and reviewing outputs during AI execution

It's worth noting that the automated testing skills built up over the years haven't lost their value — they'll continue to be applied primarily in regression testing scenarios. The real core of this transformation is a fundamental change in how work gets done.

Building the AI Testing Workbench: Five Core Layers

Having a "smart" AI alone isn't enough. Large language models like DeepSeek or Doubao are capable, but they can't directly control a browser or run API tests. That's why you need to build an AI testing workbench — essentially giving the AI brain a pair of arms, so it can truly execute test cases, inspect results, and refine test plans.

This workbench consists of five layers:

1. IDE: The Unified Operational Interface

The IDE serves as the display window for the entire workbench. There's no restriction on which product you use — Cursor, VS Code, or similar tools all work. Its core functions include: displaying the project, providing a terminal for executing commands, comparing diffs between each agent output, and editing and viewing files.

The diff comparison feature is especially useful. When you ask the AI to redesign test cases, a side-by-side comparison lets you determine whether it genuinely made improvements or just copied the old cases wholesale.

2. Agent Client: Claude Code Is the Top Choice

For the agent client layer, Claude Code is the recommended option. Anthropic is currently one of the world's leading companies in applied AI, and its prompt engineering, built-in tooling, and agent orchestration logic are all best-in-class.

Rumors about a Claude Code "backdoor" need clarification: what's actually described is a mechanism that identifies certain users and bans their accounts. However, since we're only using it as a client-side tool — with the actual model running on a domestic solution — there's no compliance risk involved.

If your company policy explicitly prohibits Claude Code, alternatives like Cursor or Tencent CodeBuddy are available, though they currently lag behind in overall capability.

3. Model: DeepSeek V3 (Stable Release)

For the model layer, DeepSeek V3 (stable release) is recommended for its strong performance, fast inference speed, affordable pricing, and the fact that data stays within China — eliminating compliance concerns. You can also use other domestic models such as Qwen, Xiaomi, Zhipu, or MiniMax; just opt for their latest versions.

4. Configuration Switcher

Because Claude Code is a foreign command-line tool that isn't straightforward to configure, you'll need a configuration switcher to connect DeepSeek to Claude Code.

5. Test Asset Repository

The final layer consists of the outputs generated by the workbench: automation code, requirements analysis documents, test execution logs, skill libraries, and more. These are valuable assets for any testing project. You don't need to create them manually — they accumulate organically as the workbench operates.

Environment Setup: Three Essential Tools to Install

The first step to getting hands-on is preparing your development environment. You'll need three foundational tools:

• Node.js (comes bundled with npm)
• Git (version control)

Verification is straightforward: open a terminal and run node -v, npm -v, and git --version. If each command returns a version number, the installation was successful.

A Critical Detail About Node.js Version Selection

This is the only tool with a version requirement. When downloading, pay close attention:

• Choose the LTS (Long-Term Support) version (marked in blue), not the green "Current" release (a short-term maintenance version that may soon lose support)
• The current recommendation is the v22 or v24 series — avoid outdated versions like v14 that have reached end-of-life (EOL)
• For Apple Silicon (M-series) chips, select the ARM build; for older laptops, use the x64 build

Configuring a Domestic Mirror for Faster Installs

Since npm is hosted overseas, installing dependencies can be slow. Configuring npm to use a domestic CDN mirror is an essential step. This command can be run multiple times without causing any conflicts.

Installation and Integration: Getting AI to Actually Work

Once your environment is ready, install Claude Code via the npm command. The installation command can also be run repeatedly — each run will update to the latest version. Any warn messages during installation can be ignored; only error messages require attention.

However, Claude Code can't be used straight out of the box — it doesn't serve users in China, and registration, login, and usage will all be rejected. The solution is to integrate a domestic model (such as DeepSeek) by using a configuration tool to connect it to the Claude Code client. This approach lets you leverage the capabilities of a world-class client while avoiding compliance risks entirely.

The Path to Leveling Up as a Testing Engineer

This AI-led testing methodology is, at its core, a reinvention of what it means to be a testing professional. As AI takes over the bulk of execution work, testers need to transition into the role of "AI supervisors" — shifting from writing scripts by hand to overseeing agents that write scripts, reviewing AI outputs, and managing the overall process.

For QA engineers with no programming background, closing gaps in foundational skills like Git and Node.js is the entry barrier. For experienced automation engineers, the challenge is rethinking the boundaries of human-machine collaboration. This is precisely the most important direction of skill evolution in the testing industry today.