DeepSeek Harness Penetration Testing Mode in Practice: Capabilities and Boundaries of AI-Assisted Vulnerability Hunting

AI Enters Penetration Testing: From Manual Analysis to Intelligent Assistance

In cybersecurity, vulnerability hunting has long relied on the experience and manual analysis of security researchers. As large model capabilities advance, AI-assisted penetration testing is emerging as a noteworthy practice. A Bilibili content creator shared a vulnerability-hunting workflow that combines DeepSeek^Ad Harness's "penetration testing mode" with a dedicated SKILLS package (referred to in the video as srcHunter Skill).

The core logic of this approach is straightforward: enable DeepSeek's penetration testing mode, then layer on a content-rich skill package (Skill/Prompt library), allowing the AI to autonomously complete information gathering and vulnerability discovery with a professional knowledge foundation. The creator repeatedly emphasizes a key point — when using AI for vulnerability hunting, the quality of the SKILLS package directly determines the upper limit of what you can achieve.

Why the SKILLS Package Is the Critical Variable

According to the creator, the recommended skill package is substantial in scope: it contains 19 attack-oriented prompts, 263 payload entries, EDR bypass techniques, 2,887 real vulnerability reports from HackerOne, and 88,636 historical cases from WooYun (乌云).

This reflects an interesting product philosophy. The creator specifically notes that you can't judge an open-source project by its Star count alone — the recommended package doesn't have a high star rating, but excels in the quality and depth of its content. The skill package is organized by category, covering HackerOne (H1) reports, web-layer payloads, AI security content, and more.

When the AI draws on these resources, it's essentially being injected with structured "expert experience." The value of real HackerOne reports and WooYun cases lies in the fact that they represent verified, real-world vulnerability patterns. When the AI reads these reports, it's effectively reasoning on the shoulders of countless security researchers. As the creator puts it: "Let the AI read through these reports and put its brain to work."

From a technical standpoint, the SKILLS package is essentially an engineering implementation of Retrieval-Augmented Generation (RAG). Large language models have a training knowledge cutoff and limited coverage of fine-grained domain knowledge (such as vulnerabilities in specific versions of a particular CMS). By injecting structured real-world vulnerability reports, payload libraries, and bypass techniques into the prompt, you're effectively expanding the model's professional knowledge base at inference time. WooYun was once China's largest internet vulnerability reporting platform; before it shut down in 2016, it accumulated a vast collection of real-world cases covering everything from SQL injection to logic vulnerabilities — a dataset that remains a valuable reference for security research today. HackerOne is currently one of the world's most active bug bounty platforms, with public reports documenting vendor-verified vulnerability reproduction processes. Using these verified real-world cases as context input is far more practically valuable than relying solely on the general security knowledge the model has "memorized" during training.

The Testing Workflow: From Information Gathering to Vulnerability Localization

One setup detail worth noting: after unzipping the skill package and copying it to the corresponding Skill directory, the new skill won't appear immediately — you need to manually refresh the page to load srcHunter. These kinds of engineering details in tool-based AI setups are easy to overlook but directly affect the user experience.

Once loaded, the creator issued a command against a test target: "Help me complete a full information gathering and vulnerability discovery process." The AI then automatically executed a series of operations:

• Information gathering: DNS resolution, asset enumeration — the creator noted this skill package has "very strong" information gathering capabilities
• Vulnerability discovery: Automatically identified and listed multiple suspected vulnerability points
• Traceable output: The entire process is auditable, with more detailed asset information than the standalone penetration testing mode provides

What Was Found: A Breakdown of Vulnerability Types

The AI's output was a dense list of vulnerabilities, which the creator manually verified one by one.

File Inclusion and Information Disclosure

First was an arbitrary file inclusion vulnerability, confirmed via CMD. Next came phpinfo exposure — via Base64 decoding, the specific info file was located, revealing the server's full configuration. Apache server information exposure and downloadable backup files were also discovered. As the creator candidly noted, these information disclosure vulnerabilities "start at DV (low severity)" when submitted — they're basic findings.

Backdoors and RCE (Critical Severity)

The truly serious findings were the multiple PHP WebShell backdoors and code execution vulnerabilities identified by the AI. The demo revealed a Behinder (冰蝎) backdoor — the AI not only identified the backdoor's existence, but also decoded the connection password (admin via Base64), the corresponding key string, and the payload in use. Godzilla (哥斯拉)-type backdoors and basic one-liner webshells were also discovered; the creator tested command execution live and successfully received output.

Behinder (冰蝎) and Godzilla (哥斯拉) are the two most widely used WebShell management tools in domestic penetration testing. Behinder's core feature is AES-encrypted traffic obfuscation, which effectively bypasses signature-based traffic detection systems (WAF/IDS). Its communication key is hardcoded in the shell file by default — which is exactly why the AI was able to decode the password admin from Base64, as shown in the video. Godzilla supports multiple encryption modes and a richer set of built-in plugins with stronger AV-evasion capabilities. Both tools occupy a middle ground between memory-based shells and traditional file shells — once an attacker plants this type of backdoor on a target server, they can achieve persistent control, file management, and command execution through a graphical client. The significance of AI being able to identify these backdoors: traditional signature scanning tools rely on known patterns, whereas AI analysis grounded in semantic understanding can detect morphed or obfuscated backdoors at the code logic level, offering some degree of evasion resistance.

Database Credentials

The results also included hardcoded database password exposure. Once an attacker obtains such credentials, it typically means complete compromise at the data layer.

The creator emphasized the comparison: "If you only use the penetration testing mode on its own, the output is nowhere near this rich." The skill package meaningfully increased both the breadth and depth of vulnerability discovery.

Cost and Practical Advice: AI Is Not a Magic Key

The creator offered a notably pragmatic recommendation — and arguably the most important takeaway from this piece: DeepSeek Harness (referred to as Dispatch in the video) is expensive to run, and blindly throwing everything at it from the start is not recommended.

His actual workflow is:

1. Do a manual pass first to identify vulnerable assets
2. Only after confirming vulnerable assets exist, bring in the AI for deeper discovery

This "manual triage + AI deep dive" combination keeps costs under control while improving efficiency. According to the creator, running one target environment costs roughly 5–8 RMB — cost sensitivity is not something to ignore.

On the topic of AI "jailbreaking" (bypassing safety restrictions), the creator mentioned that DeepSeek is relatively easier to bypass than other models — often just requiring a rephrased way of expressing intent. This also exposes the real-world limitations of safety alignment in large models.

"AI jailbreaking" has a specific meaning in security contexts: using carefully crafted prompts to circumvent a model's built-in content safety policies and elicit output that would otherwise be blocked (such as attack code or instructions for creating weapons). Safety alignment quality varies significantly across models, directly tied to the investment made during the RLHF (Reinforcement Learning from Human Feedback) training phase. The creator's comment that DeepSeek is relatively "easy to bypass" partly reflects the trade-offs domestic large models make between safety alignment and model capability — overly strict filtering severely hampers legitimate security research use cases, while overly permissive policies invite abuse. There is currently no perfect solution to this tension at the industry level. Common approaches include providing enterprise API exemption channels for security professionals, or using contextual intent recognition for dynamic judgment.

The Double-Edged Sword Effect: What to Watch Out For

This workflow demonstrates the genuine potential of AI in offensive security: it can rapidly draw on a massive body of historical vulnerability knowledge, automate information gathering and vulnerability localization, and significantly lower the barrier to entry for vulnerability hunting.

But that's precisely what makes it a double-edged sword. As AI lowers the technical threshold for vulnerability hunting, the pressure on defenders only increases. For enterprises and security teams, this means assuming that attackers already have AI-assisted capabilities — and therefore doubling down on foundational practices: asset management, backdoor detection, and eliminating hardcoded credentials.

It's worth noting that the content described in this article is based on a single creator's demonstration. Claims about skill package effectiveness, AI jailbreak difficulty, and other conclusions lack third-party cross-validation, and readers should approach them critically. Any penetration testing must be conducted within authorized scope — unauthorized vulnerability hunting may constitute a criminal offense.