Getting Started with Claude Code: Reshaping Software Testing Workflows with AI Agents

From LLMs to Agents: A Paradigm Shift in Testing Work

Over the past year, large language models have become an unavoidable topic for every tech professional. But the real catalyst for transforming software testing isn't LLMs themselves — it's the rapidly rising AI Agent. This article is based on a livestream by a Bilibili instructor with 13 years of IT experience specializing in software testing training. It covers Claude Code as a representative agentic tool, walking through its core concepts, environment setup, and practical value in the testing domain.

The instructor opened by noting that LLMs alone offer fairly limited assistance for testing work. With the rise of agents, however, the testing skill landscape has quietly shifted — evolving from the traditional stack of "functional testing, API testing, automation testing, and performance testing" to "functional testing, automation testing, AI testing, and performance testing." AI testing is becoming a must-have new direction, and its technical foundation is the Agent.

This evolution isn't accidental. According to Gartner's 2024 forecast, 80% of enterprises will deploy AI-assisted testing tools by 2027. More critically, as AI features — recommendation algorithms, generative content, automated decision systems — become deeply embedded in software products, the nature of what we test has structurally changed. Testing engineers now need new capabilities: evaluating the output quality of AI systems, detecting model bias and hallucinations, and designing test strategies for non-deterministic systems. While AI tools lower the barrier to traditional automation testing, they simultaneously push testing engineers to move upmarket — placing greater emphasis on test strategy design and quality architecture.

These compared to our large models

LLMs vs. Agents: The Fundamental Difference Between Brain and Limbs

To understand Claude Code, you first need to clarify its fundamental difference from LLMs like DeepSeek or Qwen.

LLMs: A Brain That Only "Thinks"

The way we typically use LLMs is to open a chat window, type a question, and wait for a response — a question-and-answer interaction. The instructor used a vivid analogy: an LLM is like the human brain. It can think, answer questions, and help you write test cases. But if you ask it to directly output those cases as an Excel file, or generate a complete Word document, it falls short.

Agents: Brain and Limbs Combined

Agents didn't emerge from nowhere. Their technical roots trace back to the AutoGPT project in early 2023 and the subsequent "ReAct" framework (Reasoning + Acting) — which enabled LLMs to alternate between "think → act → observe" cycles during reasoning, breaking through the limitations of single-turn Q&A. A complete Agent architecture includes four core modules: the LLM as the core reasoning engine, a memory module (short-term context + long-term vector database), a toolset (API calls, code executors, browser control, etc.), and a planner (task decomposition and goal tracking).

Agents are fundamentally different — they include an LLM, but go far beyond it. The instructor summarized three key capabilities that agents have over standalone LLMs:

Planning: Faced with complex tasks, an agent can independently break down steps and plan an execution path, rather than passively responding.
Memory: It records previous interactions and actions as the basis for subsequent steps, avoiding the "I forgot what we discussed earlier" problem caused by lost context.
Tool use: It can automatically invoke external tools to accomplish goals — generating files, executing scripts, controlling a browser, and more.

In short, if an LLM is the "brain," an agent is the "brain plus limbs" — it doesn't just think, it can actually get things done. This is precisely why it can be applied to real testing scenarios like Web automation scripts and test case generation.

But then I lost it in the back

Claude Code's Positioning: A Project-Level AI Agent

There are many Agent products on the market today, broadly divided into project-level and system-level categories. Claude Code is a classic project-level agent — it can connect to specific code projects, understand the project structure, and perform operations like planning, writing, and running code within it.

In the project-level code agent space, Claude Code faces fierce competition: GitHub Copilot Workspace, Cursor, Devin, and the open-source OpenHands are all competing in the same segment. Claude Code's differentiation comes from its underlying model — Anthropic's Claude 3 series supports an ultra-long context window of up to 200K tokens, making it especially strong at holistic comprehension of large codebases. Unlike IDE plugin tools such as Cursor, Claude Code runs as a command-line tool, making it naturally suited for integration into CI/CD pipelines like Jenkins or GitHub Actions — enabling fully automated test script generation and execution without relying on a graphical interface.

For testing engineers, this represents a significant efficiency multiplier. The instructor mentioned a "10x productivity improvement" target during the livestream, and frankly stated the actual gains are "even more than that." While this carries some promotional flair, given an agent's ability to automatically invoke tools and complete multi-step tasks in sequence, a magnitude leap in efficiency is entirely realistic for high-repetition tasks like test case writing and automation script generation.

You can use it for all of these

A Three-Day Onboarding Path: From Setup to Project Practice

The instructor broke the Claude Code learning path into three progressive stages — quite beginner-friendly.

Day 1: Environment Setup and Quick Start

The first day focuses on the basics: completing the Claude Code environment setup, learning commonly used slash commands, and understanding how to connect to a specific project and get familiar with basic operations. The core goal of this stage is to get the tool running and lay the foundation for practical work ahead.

Day 2: Auto-Generating Test Cases with Skills

Day two introduces the Skill concept, using agents to fully automatically generate test cases in Excel format.

A Skill is technically a combination of "Tool Use" and "prompt engineering" — essentially a predefined structured prompt template or custom script that tells the agent what output format, field specifications, and quality standards to follow for a specific task (such as generating test cases that conform to company standards). This design philosophy closely mirrors the "Chain" and "Tool" concepts in LangChain: by encapsulating reusable capability units, you avoid having to re-describe complex rules in natural language every time. For testing teams, the real value of Skills lies in making implicit testing expertise (such as boundary value coverage principles and equivalence class partitioning methods) explicit as agent-executable rule sets — enabling the AI to produce professional test artifacts that meet team standards, not just "generically reasonable" content.

Day 3: Web Automation Test Script Project Practice

The final day moves into full project practice — using Claude Code with Skills to fully automatically generate Web automation test scripts covering a variety of real testing scenarios. The entire course includes two complete projects, emphasizing hands-on practice over pure theory.

Two complete project practices in total

Three Reflections for Testing Professionals

This introductory session surfaces several trends and takeaways worth paying attention to.

AI testing is becoming a standard job requirement. The instructor explicitly stated that today's testing professionals "must find" their footing in AI testing — mastering agent tools has evolved from a "nice to have" into a "must have." The structural driver behind this: when AI features themselves become the object of testing, traditional testing methodologies face a fundamental challenge. How do you test a system whose outputs are inherently stochastic? How do you quantify the "quality" of generative AI? These questions are giving rise to entirely new testing methodologies and job requirements.

Understanding foundational concepts matters more than tool proficiency. No matter how tools evolve, the underlying logic — "LLMs are the brain, agents are brain plus limbs" — won't change. Deeply understanding the three core capabilities of planning, memory, and tool use is what will let you truly get the most out of Claude Code and any Agent product that follows.

Zero prior experience is not a barrier. The livestream audience included both complete beginners and veterans with nearly a decade of experience. The value of agent tools lies precisely in lowering the barrier to writing automation tests — what you need most is the ability to clearly describe requirements and validate results, not to hand-write every line of code from scratch.

For testing engineers who want to stay competitive in the AI era, Claude Code offers a low-barrier, high-ceiling entry point. The agent paradigm behind it also signals a broader trend: more and more phases of software engineering will continue shifting from "humans write code" to "humans direct AI agents to write code."