Escaping AI Agent Skill Hell: A Complete 4-Step Guide to Writing High-Quality Skills

In the world of AI-assisted programming, developers always seem to find themselves in a new kind of "hell." A few years ago it was "tutorial hell"—endlessly grinding through tutorials yet never assembling a complete skill set. Then came "framework hell"—a new JavaScript framework popping up every ten minutes to force you to keep chasing. And now, Matt Pocock, author of the well-known engineering skill library Matt Pocock Skills, points out that we've entered a new predicament: Skill Hell.

The paradigm shift from tutorial hell to skill hell: This evolutionary path reflects the core tensions across three distinct stages of the software development toolchain. Both tutorial hell and framework hell are essentially "information overload" problems—fundamentally, the rate at which humans can learn can't keep up with the rate at which knowledge is produced. The former manifests as fragmentation of learning paths, the latter as decision fatigue in technology selection. Skill hell, however, is a fundamentally different "quality signal deficiency" problem: the content itself is already abundant—you can find hundreds of Cursor Rules repositories and thousands of skill.md files on GitHub—but the entire ecosystem lacks a reliable evaluation mechanism to distinguish good from bad. This closely resembles the predicament of early open-source software communities that lacked package quality standards, except that the "runtime environment" for AI skills is a probabilistic language model, making quality assessment far more complex than for a code library.

What Is "Skill Hell"

Today, a massive number of AI Agent Skills can be downloaded, contributed to, and customized for free. But the problem is: you simply can't tell whether a skill is good or bad, nor do you understand how these fragments work together. People try to cram every skill they can get their hands on together all at once, only to fail to achieve the results the skills themselves promise.

What is an AI Agent Skill? An AI Agent Skill is a core abstraction layer in the modern LLM toolchain—essentially a structured Markdown instruction file injected into an AI Agent's context window (such as Cursor, Claude, Windsurf, etc.) to constrain and guide the model's behavior. This concept grew out of early System Prompt engineering and, with the popularization of product forms like Cursor Rules and Claude Projects, gradually evolved into reusable, distributable "behavior modules." Unlike function libraries in traditional software, a Skill's execution agent is a language model, and its "output" is inherently probabilistic and uncertain. This is precisely why skill quality is hard to measure by traditional software engineering standards. Notably, a Skill is essentially a kind of "meta-instruction"—it doesn't directly produce business output, but shapes the model behavior that produces business output. This makes its effectiveness evaluation markedly lagging and indirect, requiring multiple runs in real task scenarios to statistically derive reliable quality signals.

This predicament exists not only at the individual level but is equally severe at the organizational level. Companies lack a methodology for building high-quality skills, and don't know how to translate their Standard Operating Procedures (SOPs) into usable Agent capabilities. "Just one more skill, bro"—this has practically become everyone's mindset.

Matt Pocock admits he feels somewhat guilty, since he runs one of the most popular engineering skill libraries. That's why he raises the core question: What exactly are we missing? The answer is—we don't know what makes a skill excellent. The industry lacks a shared rubric that lets you look at a skill and judge what it does well and what needs improvement.

To address this, he proposes a "skill checklist" consisting of four core components: Trigger, Structure, Steering, and Pruning. Notably, he has packaged this methodology itself into a skill called "Writing Great Skills" in his repository, which users can download directly to improve their own skills.

Step 1: Trigger—Determining How a Skill Is Invoked

Skill invocation falls into two categories: user-invoked and model-invoked.

Any skill can be invoked manually—the skill file sits in your file system, and you can simply tell the Agent to read it. A model-invoked skill is different: it comes with a description that enters the Agent's context, and the Agent can decide on its own, "Based on this description, I should invoke this skill," then read the skill.md file into its context window.

This description is essentially a "context pointer" pointing to a file that stores more information. And this pointer is optional—if you remove it (set disable model invocation: true), the skill becomes visible only to the user, turning into a purely user-invoked skill.

The Trade-offs Between the Two Invocation Methods

You might think model invocation is more flexible and better. But Matt Pocock points out that every model-invoked skill you add increases the Agent's "context load"—if you have 100 model-invoked skills, that's 100 descriptions permanently residing in the context, consuming tokens with every request and adding to the burden the Agent has to reason about.

The engineering implications of context windows and token consumption: The context window is the maximum length of text a large language model can process at once, measured in tokens. A token is not simply equivalent to a character—for English, on average about 4 characters equal 1 token; for Chinese, each character typically corresponds to 1–2 tokens. Current mainstream models have expanded their context windows to 128K or even 200K tokens (e.g., Claude 3.5 Sonnet), but a longer context means higher inference latency, higher API costs, and the "Lost in the Middle" problem—research shows that models utilize information in the middle of the context significantly less than at the beginning and end, with a 2023 Stanford study revealing this gap can be as high as 20 percentage points. Therefore, the practice of trimming skill.md is essentially fine-grained engineering management of limited cognitive bandwidth, balancing both cost control and model attention quality.

User-invoked skills bring a different kind of load: cognitive load. The concept of cognitive load originates from cognitive load theory, proposed by educational psychologist John Sweller in 1988, which describes the total amount of pressure on working memory when processing information. The more user-invoked skills there are, the more the user has to keep in their head, and the higher the technical bar for the user—this mirrors the design dilemma of traditional command-line tools where "the more options, the harder to use."

Comparing the two popular skill libraries: Superpowers primarily uses model invocation, endowing Agents with "superpowers," while Matt Pocock's own skill library favors user invocation, because it minimizes the Agent's context load—at the cost of requiring deep understanding of the skills yourself. The core reason he prefers user invocation is that model invocation introduces unpredictability: even if a skill perfectly fits a task, the model may choose not to invoke it. This unpredictability forces you to introduce a skill evaluation (eval) process, which is quite troublesome; user invocation directly eliminates this class of problem.

Step 2: Structure—The Internal Layout of a Skill

Matt Pocock believes most skills consist internally of two fundamental units: Steps and Reference. Steps are the step-by-step process the skill goes through, while reference materials are the supporting information that underpins those steps.

Take his 2PRD skill as an example. It generates a product requirements document from the current context and contains three steps: find the relevant context, confirm the test seam with the user (a human-in-the-loop checkpoint), and write the PRD. To support these three steps, it comes with two reference materials: an explanation of "what is a test seam," and a PRD template.

Test Seam and Human-in-the-Loop: The concept of the test seam was first introduced by Michael Feathers in Working Effectively with Legacy Code, referring to a location in code where a test observation point can be inserted. In the context of AI Agent workflows, Matt Pocock extends it to mean "a breakpoint where a human can step in to check"—i.e., a deliberately placed human-confirmation node within a long-chain task the Agent executes automatically. This design philosophy aligns closely with the "Human-in-the-Loop (HITL)" concept in industrial automation: full automation isn't always the optimal solution, and retaining human judgment at high-risk or high-ambiguity nodes can significantly reduce the risk of automated systems going out of control. In LLM Agent practice, setting test seams reasonably is especially important, because model errors often have a "confident error" characteristic—the model outputs correct and incorrect answers in the same tone, meaning fully automated processes without human intervention carry the risk of undetectable silent failures.

Keep skill.md As Lean As Possible

Each skill consists of a description, the main skill.md file, and reference materials branching off from the main file. Keeping skill.md small has many benefits: easier maintenance, easier auditing, and lower token consumption.

The way to achieve this is to analyze the skill's "branches." If a piece of reference material is only used in one branch, it's a candidate to be moved out of the main file. Take 2PRD as an example—it has only one branch, and the PRD template is used every time, so it rightly belongs in skill.md.

But another skill, domain modeling, is different—it might update the local glossary context.md, or it might create an Architecture Decision Record (ADR), or it might do neither.

What is an Architecture Decision Record (ADR)? An ADR is a lightweight technical documentation practice, first systematized by Michael Nygard in 2011, used to record each important architecture decision in a structured document: including the decision's Context, the Options considered, the final Decision, and the Consequences of adopting it. A standard ADR format is usually about one page long, deliberately kept concise to lower the barrier to writing. In AI-assisted development workflows, ADRs have dual value: on the one hand, they serve as a decision log for human engineers, preventing team members from repeatedly debating settled issues; on the other hand, they can be injected directly into context as a "memory patch" for the AI Agent, helping the Agent understand the project's historical constraints and technical debt background, avoiding the need to re-explain things like "why we chose PostgreSQL over MongoDB" in every conversation. As AI Agents become deeply involved in long-cycle projects, ADRs are evolving from an optional best practice into essential engineering infrastructure.

domain modeling has two or three branches, so neither the ADR template nor the context.md template should be crammed into the main file—they should be placed in separate markdown files and referenced via "context pointers." The pointer only needs to say something like "if you need a template, look in this file." Matt Pocock calls this an "external reference."

Key technique: hide reference materials that belong to a single branch behind a context pointer.

Step 3: Steering—Making the Agent Actually Do What You Say

This is the component Matt Pocock considers the most important, as it solves a classic pain point: you clearly spelled it out in the skill, yet the Agent doesn't do it.

The core tool he offers is called "leading words." Leading words are words that pack a large amount of meaning into a very small space. Its mechanism is quite clever: you write a leading word into the skill text, and the Agent will repeat this word over and over in its reasoning tokens and thought process. Because it keeps reinforcing this word—and the word happens to describe the behavior you want—the Agent's output changes accordingly.

Reasoning Trace and observability: The reasoning trace, also known as Chain of Thought (CoT), refers to the intermediate reasoning steps a model produces before generating its final answer, systematized by the Google Brain team in their 2022 paper "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models." Reasoning models represented by OpenAI's o1/o3 series and Anthropic's Claude 3.7 Sonnet (Extended Thinking mode) generate a large number of internal "thinking tokens," usually visible to the user in a collapsed form. Matt Pocock's suggestion to "observe in the thinking trace whether the Agent has adopted the leading word" essentially uses the reasoning trace as a debug interface—if the model repeats the leading word during its thought process, it means the word has been activated and incorporated into the reasoning framework, thereby influencing the final output. This reasoning-trace-based observability practice is an important marker of the evolution of prompt engineering toward more rigorous "prompt debugging," and is the key dividing line between "random trial-and-error" prompt engineers and "methodology-driven" ones.

Leading Words in Practice

A typical problem is that Agents tend to "code layer by layer"—first writing the entire database layer, then the schema, then the API endpoints, and finally the frontend. They don't work like experienced developers who first build a small runnable slice, get feedback early, and then iterate and expand.

You could of course just say "don't code layer by layer, build a small slice first." But a better approach is to use a leading word: vertical slice.

The software engineering origins of vertical slice: The vertical slice is one of the core practices in agile software development, contrasting with the "horizontal layer" development model: horizontal layering requires fully implementing the database layer first, then the business logic layer, and finally the UI layer; a vertical slice, by contrast, requires each iteration to cut through all layers, delivering an end-to-end runnable minimal functional unit (akin to MVP thinking). This term has been elaborated in depth in Scrum, Extreme Programming (XP), and the domain-driven design (DDD) practices of people like Jimmy Nilsson, and holds high semantic consensus within the professional developer community. Its core value lies in early integration and early risk exposure, avoiding "integration hell" in the final stage. Matt Pocock injects this mature term into AI Agents as a "leading word," leveraging precisely the extensive prior knowledge about the term the model accumulated in its pre-training corpus (which includes massive amounts of technical blogs, Stack Overflow discussions, and open-source project documentation), enabling the model to automatically map to the correct behavior pattern without detailed explanation. This also validates a broader prompt engineering principle: using domain-specific terms already backed by rich corpus support tends to produce more stable guidance than ad-hoc coined terms.

This is a widely recognized term in the development field that triggers the Agent's prior knowledge. Simply by repeatedly using this phrase in the skill, you'll see the Agent say in its reasoning trace, "let's make it a thin vertical slice"—that's the evidence the leading word is working.

Matt Pocock emphasizes that many developers have actually long been using similar phrases to guide Agents. His advice is simply to use these leading words consistently and forcefully in skills, and observe in the thinking trace whether the Agent genuinely adopts your approach.

Splitting Skills: Forcing the Agent to Focus on the Current Step

Another lever for solving the Agent's "laziness" problem is splitting skills. A classic case is plan mode, which contains two steps: "ask clarifying questions" and "formulate a plan." But Matt Pocock found that when the Agent sees the ultimate goal is to formulate a plan, it rushes through a few questions and hurries to make the plan, never doing the clarification stage deeply enough.

His solution is to split the two steps apart: making the clarification stage a standalone skill called grill with docs, and making the planning a standalone 2PRD. This way the Agent only sees one step at a time. By hiding future goals and steps, you force the Agent to invest sufficient effort in the current step. This technique is remarkably effective in scenarios requiring deep work. The cognitive science basis behind it is the "Goal Gradient Effect"—the closer to the goal, the more the execution agent tends to accelerate rather than maintain quality; truncating goal visibility resets this effect.

Step 4: Pruning—Trimming Away Redundancy

Pruning is a set of quick-check "failure modes" to run through. The core principle is: don't have bloated mega-skills—mega-skills are often an external symptom of other problems.

Failure Mode 1: Repetition (DRY). Every piece of content should have a single source of truth. Reference materials like the PRD template or "what is a test seam" should never be defined redundantly in multiple places.

The engineering background of the DRY principle: DRY (Don't Repeat Yourself) is one of the most fundamental design principles in software engineering, formally introduced by Andrew Hunt and David Thomas in their classic 1999 book The Pragmatic Programmer, originally stated as "Every piece of knowledge must have a single, unambiguous, authoritative representation within a system." Its core tenet is that every piece of knowledge in a system should have a single, clear, authoritative representation. Violating DRY leads to "knowledge fragmentation"—when modifying a concept, you must find and update all copies simultaneously, which is prone to omission, creating "consistency debt." Applying DRY to AI skill documentation has its own special challenges: documentation maintenance is usually distributed and asynchronous, and lacks the static analysis tools found in traditional codebases (such as duplicate code detectors), making duplication harder to discover automatically. In addition, when multiple skill files are injected into the context simultaneously, duplicated content causes double token consumption, exacerbating pressure on the context window.

Failure Mode 2: Sediment. When multiple people collaborate on the same document, everyone adds content but no one dares to delete or modify others' contributions, ultimately accumulating a large amount of irrelevant "sediment." This phenomenon has deep roots at the social engineering level: the psychological resistance to modifying others' contributions (stemming from respect for contributors' efforts and concern over triggering conflict) far exceeds the resistance to adding new content, closely resembling the "Broken Window Theory" in codebases—once quality begins to decline, it decays one-directionally over time, ultimately tending toward chaos when no one intervenes. The countermeasure is to examine the structure first: determine whether new content is relevant to all branches; if not, move it to the correct branch; if it's completely irrelevant or outdated, delete it directly. Establishing an engineering culture of regular "skill audits" is more sustainable than relying on individual conscientiousness.

Failure Mode 3: No-ops. This is especially common when Agents auto-generate skills—a passage may appear to constrain behavior but deleting it doesn't actually affect the Agent's output quality. For example, telling the Agent to "write a detailed commit message," but even if you delete this sentence, the Agent will still write a good commit message. The harm of no-op instructions isn't just wasting tokens—it's that they dilute the signal density of the genuinely effective instructions in the skill document, similar to noise interfering with signal propagation. Matt Pocock reveals that his skills are so lean precisely because he continually runs "deletion tests": compressing intent into leading words and clearing out irrelevant content and sediment. A systematic "Ablation Test" method—deleting instructions one by one and comparing output quality—is the most reliable empirical means of identifying no-ops, borrowed from the standard methodology in machine learning for evaluating the contribution of each component.

Summary: A Quick Look at the 4-Step Framework

This framework completely covers the entire process of writing good AI Agent Skills:

• Trigger: Ensure the skill is invoked at the right time, weighing whether to impose context load (model invocation) or cognitive load (user invocation).
• Structure: Split into steps and reference materials, move materials belonging to a single branch out of the main file, and keep skill.md lean.
• Steering: Use leading words to compress intent into reusable keywords, and verify in the reasoning trace; when necessary, split skills to force the Agent to focus on the current stage.
• Pruning: Do one final trim, watch out for sediment and redundancy, and focus on checking for no-ops.

The best way to get started is to download the "Writing Great Skills" skill from Matt Pocock's repository and use it to review and improve your own skills, or even evaluate whether community-contributed skills are truly reliable. In an era where AI programming increasingly relies on Agent skills, mastering the ability to distinguish good from bad skills is becoming ever more critical.

Key Takeaways

Key Takeaways