[KongchangAI]
· 4 min read· 2,176 words

AI Agent Skill Development and Invocation Guide: From Functional to Effective

AI Agent Skill Development and Invocation Guide: From Functional to Effective

A complete guide to writing, invoking, and maintaining AI Agent Skills, emphasizing principle-driven design and continuous iteration.

This article systematically covers the complete practical methodology for AI Agent Skills. A Skill has three parts—master Markdown, References, and Scripts—loaded on demand via progressive disclosure to avoid context bloat. Three signals indicate when a Skill is worth building: high-frequency triggering, domain knowledge, and high error cost. Two paths exist: reverse engineering and proactive initiation, with the mindset of being the AI's PM and using Evaluation Driven Development to find sticking points. The three pitfalls—no triggering, unstable output, and overly long Skills—map to optimizing Name/Description, giving principles over narrow SOPs, and splitting into References or Scripts. Maintenance requires continuous redundancy removal and restructuring on new model releases and workflow retrospectives, with the true value being knowledge transfer to the Agent and a shared team capability interface.

From Functional to Effective: A Complete Guide to Writing and Invoking AI Agent Skills

A Skill is essentially a folder that typically contains three things: a master Skill Markdown file (with Front Matter at the top specifying the name and description, followed by the methodology), a References folder (holding reference materials that don't need to be loaded every time), and a Scripts folder (holding directly executable scripts). This structure works in tandem with the progressive disclosure mechanism: at startup, the model loads only each Skill's Name and Description (about 100 tokens). The full Skill file is loaded only when triggered by conversation, while References and Scripts are loaded on demand—anything unused takes up zero Context Window space. This is why you can mount many Skills without blowing up your context. According to a Bilibili creator's hands-on experience, he has built over a hundred Skills for various company teams over the past year. The real challenge isn't "writing one"—it's writing a Skill that triggers automatically without being explicitly requested, produces stable output, and can be maintained by humans long-term.

When Is It Worth Building a Skill

The criteria can be boiled down to three signals: high-frequency triggering (similar tasks recur with a fixed process), domain knowledge (project path conventions, company file formats, etc. that the model can't access during training), and high cost of errors (outputs must be delivered externally, and mistakes are costly). Don't agonize over whether all three apply—the cost of building a Skill is now extremely low. Rather than spending time weighing it, just start building and delete it if you don't use it.

A common example for office workers: spending an hour each week going through work logs, meeting notes, and project progress to compile them into the format your boss wants. This type of task is a textbook Skill candidate.

Before starting, if anything is unclear, please clarify with me first—don't just start working.

Progressive Disclosure is borrowed from the UI design field, referring to systems progressively revealing information only when the user needs it, avoiding cognitive overload from presenting everything at once. In LLM applications, it's used to manage the limited Context Window: the Agent loads only each Skill's metadata (Name and Description) at startup, loading the full body of a Skill only when the conversation matches that Skill's description. Reference materials in References and scripts in Scripts are further deferred until specific steps are executed. This layered loading strategy makes it possible to mount a large number of Skills without context bloat, increased inference costs, and degraded recall precision caused by preloading everything.

Two Paths to Building: Reverse Engineering and Proactive Initiation

Reverse engineering is a beginner-friendly starting point. Take weekly reports as an example: open a new Session on Wednesday morning and have the Agent connect to Google Drive to find logs, check the week's commits on GitHub, and compile everything into Markdown. Along the way, the AI may make mistakes or run into permission issues—you collaborate and adjust for an hour, eventually producing a satisfying result. At this point, don't rush to close the Session—this is the most valuable Context. Then invoke Anthropic's official Skill Creator (installable from the Claude official Marketplace, or just ask Claude to install it for you) to convert this conversation into a reusable Skill. You don't even need the plugin: just have the AI search the web for Skill-building Best Practices first, then perform the conversion.

Proactive initiation applies when you already have an idea in mind but haven't tried it yet. Tell the AI your purpose, imagined execution flow, and potential difficulties, ask it to organize them into a Skill, and ask it to clarify anything unclear with you before starting. The core mindset here: don't ask what the AI can do for you—instead think about whether you've prepared enough context, constraints, methods, and goals for the AI to successfully deliver. The process of building a Skill is essentially being the AI's PM.

There's also an advanced technique called Evaluation Driven Development: before building a Skill, have the AI attempt the task raw, with no methodology given, and see where it gets stuck. Those sticking points are the real gaps you should fill in—not rules you imagine. When optimizing, don't just say "try again"—ask: is it missing a tool, a rule, a file, or acceptance criteria? Effective iteration fills systemic gaps.

Three Common Pitfalls and Their Solutions

Pitfall 1: The Agent Doesn't Trigger the Skill at All

A Skill's Name and Description are the only basis for the Agent to decide whether to trigger it. Name it with lowercase letters and hyphenated verb-noun or noun phrases, like weekly-report-drafting or pdf-processing, specific enough for both humans and the Agent to understand at a glance. The Description has three rules: it must include both "what it does" and "when to use it"; use third person (writing "I can help you" conflicts with the system's perspective); and trigger words should be what users would naturally say, not technical terms. Reference wording: Drafts weekly status updates for managers, aggregating data from Google Drive and GitHub. Use when user mentions weekly reports, team updates, status summaries, or weekly reviews. If it still doesn't trigger, check whether the description is too technical, or conversely too broad and causing over-triggering.

Pitfall 2: Unstable Output Quality

The same Skill produces a great weekly report this week, wrong formatting next week, and a missed commit the week after. The root cause is often writing the SOP too narrowly—a narrow SOP assumes the model can't judge, but modern reasoning models are quite capable. The right approach is to give principles, not lock down steps: write clearly why you're doing it, what mindset to adopt, and what the core judgment principles are.

Taking the weekly report Skill as an example, its execution philosophy can be written as: the weekly report is written for the manager, so when organizing information, the standard isn't "what did I do" but "does the manager need to know this."

The criterion is clear: does this task have an objective standard of right and wrong? If yes (like form field validation), write strict steps; if no (like weekly report trade-offs), write a principles framework.

SOP (Standard Operating Procedure) is a written specification that codifies the execution steps of repetitive tasks, commonly found in manufacturing and customer service. In Skill writing, the SOP style lists every operation and every decision branch in detail, attempting to make the model execute strictly like an assembly-line worker. The limitation of this approach: it assumes the executor lacks judgment, whereas modern reasoning models already possess strong contextual understanding and trade-off capabilities. Overly detailed SOPs not only increase maintenance burden but also leave the model at a loss when encountering uncovered edge cases, resulting in rigid or incomplete output. Therefore, the article recommends distinguishing by task nature: write strict steps for parts with objective standards, and write principles frameworks for parts that rely on judgment.

Pitfall 3: The Skill Is Too Long

LLMs exhibit Context Rot—the more tokens loaded, the lower the recall precision. The official recommendation is to split at over 500 lines, but practitioners tend to keep it under 200 lines, since humans themselves find it hard to maintain. The splitting approach corresponds to the Skill's three-layer structure: put large amounts of reference material and expanded content used only in specific situations into References (such as hundred-line Output Format specs, company glossaries, and writing examples for when progress falls behind), and reference them on demand in the Skill with "please refer to XX in the Reference folder"; write deterministic operations (sorting, format validation, API calls, calculations) as Scripts, so the Agent's task shifts from "figuring out how to do it" to "run the script and read the result." This prevents the model from improvising, and the script code itself doesn't enter the Context—only the execution result is returned.

This is a small trick many people overlook but it's super useful.

For more professional scenarios, you can introduce a Subagent for final quality control: based on the Separation of Concerns principle, spin up an independent Subagent, give it a clean Context and a Checklist as input, let it self-validate, and return the results. This isolates the Context and prevents the main Agent from being both player and referee.

Context Rot refers to the phenomenon where, as the input context length increases, the LLM's precision in recalling and utilizing information within it actually decreases. Research shows that when the context is filled with large amounts of irrelevant or redundant content, the model tends to ignore key instructions, confuse information priorities, and even produce output that contradicts earlier content. This isn't simply "forgetting"—it's the decay of attention allocation efficiency over long sequences. For Skill writing, this means cramming all reference materials, examples, and edge cases into the main file is counterproductive. Keeping the main Skill under 200 lines and offloading non-essential content to References or converting it to Scripts is precisely to keep the main context "clean," letting the model focus its attention on the core instructions of the current task.

Subagent is an auxiliary agent launched independently of the main Agent, with its own independent context window and task instructions. In a master-slave architecture, the main Agent coordinates the process and, when it reaches a step requiring specialized checks or independent verification, hands a clean task description and acceptance checklist to the Subagent, which completes its judgment in an isolated environment and returns its conclusion. This has two benefits: first, it prevents acceptance criteria and execution process from interfering with each other in the same context; second, it achieves separation of duties—the executor doesn't grade itself, and quality control is done from an independent perspective. The Subagent's conclusion is usually returned as a structured summary, taking up little of the main context, making it suitable as a quality gate at the end of the Skill workflow.

Maintenance: The Compound Interest of Skills Comes from Iteration

The most valuable thing about Skills is the compound-interest effect—build one today, fix it once each time you use it, and a month later it's often not the same one anymore. But most people build one and never touch it again. A casual check of your own Skills will reveal: useless materials piled up in References, 200 lines ballooned to 1000, duplicate information everywhere. Unmaintained Skills perform worse and worse, and can even be the culprit behind a blown budget.

There are two maintenance triggers: when a new model is released, run your existing Skills through it and focus on deleting the "patches" added to compensate for old model weaknesses; after each workflow run, ask the Agent to do its own retrospective—inventory errors, analyze causes, and judge whether they can be incorporated into the Skill. But to prevent the Agent from inserting new information everywhere, breaking the structure, or writing content humans find hard to maintain, you must double-check before writing.

Back to the point—personally, I do three things every time I maintain.

Every maintenance session involves three fixed tasks: delete redundant information (say each thing once), restructure (read through it quickly yourself; if you can't understand it, refactor), and extract excessive examples and Corner Cases into References.

The True Meaning of Skills

A Skill isn't just about boosting personal productivity—it's the carrier for transferring personal knowledge to the Agent. What used to take a long time to teach a new employee can be packaged into a Skill that the Agent learns instantly; methodologies, judgment criteria, and pitfalls you've encountered locked in your head can now be transferred out one by one. It's also a new interface for team capability sharing—when a newcomer joins, their Agent already knows. Models will only get stronger. Writing a Skill isn't about raising the AI to your level—it's about teaching it your way of thinking. Tying back to the concept of Harness Engineering: humans steer, Agents execute. The execution philosophy matters precisely because it's what holds truly unique value.

One-line Summary

A complete guide to writing, invoking, and maintaining AI Agent Skills, emphasizing principle-driven design and continuous iteration.

Paragraph Summary

This article systematically explains the complete practical methodology for AI Agent Skills. A Skill consists of three parts—the master Markdown, References, and Scripts—loaded on demand via progressive disclosure to avoid blowing up the context. Whether a Skill is worth building can be judged by three signals: high-frequency triggering, domain knowledge, and high error cost. There are two paths to building: reverse engineering and proactive initiation, with the core mindset of acting as the AI's PM and using Evaluation Driven Development to pinpoint sticking points. The three common pitfalls—the Agent not triggering, unstable output, and overly long Skills—correspond to solutions such as optimizing the Name/Description, giving principles rather than narrow SOPs, and splitting content into References or Scripts. For maintenance, continuously delete redundancy and restructure upon new model releases and workflow retrospectives. The true meaning of a Skill lies in transferring personal knowledge to the Agent and forming a shared interface for team capabilities.

#AI Agent Skill#Skill writing guide#progressive disclosure#Context Rot#Subagent quality control#Evaluation Driven Development
Share:

Related articles