Deep Dive into AI Agent Skill Design: Engineering Practices from Anthropic and Perplexity

Two technical articles from Anthropic (Claude Code team) and Perplexity (Agent team) reveal the core philosophy and engineering practices behind AI Agent "Skill" system design. This post cross-references both articles to distill actionable design principles.

What Is a Skill? Rethinking Agent Capabilities

The most fundamental consensus across both articles is: A Skill is not a Markdown file — it's a directory.

my-skill/
├── SKILL.md          # Front matter + instruction body
├── scripts/          # Executable scripts (deterministic logic)
├── references/       # Heavy documentation (loaded on demand)
├── assets/           # Templates, Schemas
└── config.json       # First-run configuration

This reframe matters. Many developers default to thinking "writing prompts for an Agent" means writing a block of Markdown. In reality, a high-quality Skill is a miniature software package — with an entrypoint description, runtime scripts, reference materials, and configuration management.

Anthropic's Nine Internal Skill Categories

After cataloging hundreds of Skills, Anthropic identified nine clusters:

1. Library/API References — Injecting private API knowledge into the model (e.g., billing-lib, sandbox-proxy)
2. Product Validation — The category with the greatest internal quality impact (Playwright tests, tmux-driven flows)
3. Data Fetching & Analysis — Enabling self-serve data capabilities for the model (funnel-query, grafana)
4. Business Process Automation — Encoding repetitive ritual work (standup-post, weekly-recap)
5. Code Scaffolding/Templates — Eliminating boilerplate writing time (new-migration, create-app)
6. Code Quality & Review — Standardizing team conventions (adversarial-review, code-style)
7. CI/CD & Deployment — Reducing the cognitive load of releasing (babysit-pr, cherry-pick-prod)
8. Ops Runbooks — Moving from reactive firefighting to proactive diagnosis (symptom → diagnosis → report)
9. Infrastructure Operations — Coordinating cross-system operations (dependency-management)

Category 2 (Product Validation) is explicitly called out as "the most impactful Skill type internally." This suggests the core bottleneck for Agents isn't writing code — it's verifying that the code actually works.

Four Core Principles of Skill Design

The Tax Test: Only Write What the Model Gets Wrong

Perplexity introduces a clean, powerful heuristic — the Tax Test: for every sentence in a Skill, ask yourself, "Would the Agent make a mistake without this instruction?" If the answer is no, cut it.

LLMs already know how to use git, write Python, and call REST APIs. You don't need to teach them that. You only need to fill in the gaps where the model genuinely makes mistakes. Anthropic puts it even more bluntly: restating Claude's default behavior only adds to context window cost without providing any benefit.

Gotchas Are the Highest-Value Content in a Skill

Both articles independently emphasize: the most important part of any Skill is its Gotchas (traps and caveats).

A classic example: "Note: the field is named @request_id in Service A, but the same field is called trace_id in Service B."

Gotchas are the most valuable content for three reasons: they encode your team's real failure history; they represent information the model cannot infer from code; and they have an extremely high signal-to-noise ratio — every token carries critical information.

Perplexity takes this further with the Gotchas Flywheel: Agent makes a mistake → append a Gotcha → re-run Eval → verify the fix → merge. Skills are primarily append-driven — changes after the initial merge should mostly be adding Gotchas, not rewriting descriptions or expanding instructions.

Description Is a Routing Trigger, Not a Documentation Summary

A Skill's description field isn't documentation for humans — it's a routing signal that the model uses to make loading decisions.

Perplexity's best practices: start with "Load when..."; keep it under 50 words; describe user intent rather than the workflow; use words users actually say (e.g., "babysit" rather than "monitor CI pipeline").

Subtle wording differences can produce dramatically different routing outcomes and may spill over to affect the trigger priority of other Skills.

Progressive Disclosure: A Three-Layer Context Cost Model

Perplexity outlines a clear three-tier loading architecture:

• Index Layer — name + description, ~100 tokens/Skill, loaded every session
• Load Layer — full SKILL.md body, ~5,000 tokens, loaded on invocation
• Runtime Layer — scripts, references, assets, uncapped, loaded on demand

This design resolves a core tension: you want the model to be aware of as many Skills as possible (indexing is cheap), but you don't want to pay the full context cost for every Skill (loading is expensive).

When You Do (and Don't) Need a Skill

Good candidates for building a Skill:

• The model makes mistakes without specific context (internal API conventions, field mappings, deployment flows)
• Behavior must be highly consistent (code style, review standards, security checks)
• Knowledge is persistent but absent from training data (private library docs, internal tool usage)
• The task involves taste-based judgment (Perplexity's design Skill was authored by the design lead, encoding preferences around typography, color, and spacing)

Cases where you don't need a Skill:

• General workflows the model already handles well (standard git operations, common coding patterns)
• Content that duplicates the system prompt
• Underlying content that changes faster than your ability to maintain the Skill

An important warning: Perplexity cites an arXiv paper finding that "self-generated Skills provide no benefit on average." The value of a Skill comes from human domain knowledge and hard-won failure experience — not from having AI summarize "how to do something."

Skill Lifecycle Management

Eval-First: Write Evaluations Before Writing the Skill

Perplexity follows a strict Eval-First process:

1. Collect evaluation cases from real user queries, known failures, and "adjacent confusion" scenarios
2. Negative examples matter more than positive ones — focus on scenarios that should not trigger the Skill and behaviors that should not occur
3. The eval suite should cover: Skill loading precision/recall, progressive loading behavior, end-to-end task completion (with LLM Judge scoring), and cross-model consistency

Organic Distribution: Let Value Speak for Itself

Anthropic has no centralized Skill approval process internally. The path is: author uploads to sandbox → shares on Slack → builds reputation → opens a PR to the marketplace. This "prove value before formalizing" model effectively lowers the barrier to creation.

Watch Out for Action at a Distance

Perplexity specifically warns: adding a new Skill can silently degrade the performance of existing ones. This happens because descriptions compete for attention — if a new Skill's description shares vocabulary with an older one, it may capture routing priority. Every change therefore needs to be validated with an eval suite.

Practical Advice for Building Skills

Don't Railroad: Avoid writing overly prescriptive step sequences ("run git checkout → run git cherry-pick → ..."). Instead, express intent with constraints ("cherry-pick to a clean branch, preserving original intent when resolving conflicts"). Give the model information and flexibility so it can adapt to the situation at hand.

Store Scripts, Generate Code: Put deterministic logic in the scripts/ directory and let the model compose existing scripts rather than rebuilding from scratch. This prevents the model from repeatedly introducing errors every time it stitches together a SQL query or API call.

Help the Model Remember: Use a persistent data directory (e.g., ${CLAUDE_PLUGIN_DATA}) where the Skill maintains append-only logs, JSON state files, or a SQLite database — enabling genuine cross-session memory.

Use Hierarchy to Manage Complexity: Perplexity shares a telling experiment — a U.S. tax law Skill with 1,945 IRC sections actually performed worse than loading no Skill at all when flattened into a single file. Reorganizing it into a three-level directory hierarchy produced a dramatic improvement. Information overload often hurts the model more than information gaps.

Measurement and Iteration

Anthropic's approach: Use PreToolUse hooks to log Skill invocations, identifying high-frequency Skills (proving value) and low-trigger-rate Skills (needing description optimization).

Perplexity's approach: A multi-dimensional eval framework — precision (are Skills triggered when they shouldn't be?), recall (are Skills missed when they should trigger?), inhibition checks (is neighborhood isolation working?), and cross-model consistency (behavioral differences across GPT vs. Claude Opus vs. Claude Sonnet).

Side-by-Side Comparison

Dimension	Anthropic (Claude Code)	Perplexity (Computer)
Core focus	Developer tooling enhancement	Multi-domain general Agent
Distribution model	Repo-embedded + Marketplace	Runtime on-demand loading
Trigger mechanism	/skill-name or model auto-match	load_skill() + recursive dependencies
Quality assurance	Usage tracking + organic reputation	Eval-First + cross-model testing
Maintenance philosophy	"Start with a few lines, let it grow"	"Gotchas Flywheel"
Token management	Progressive disclosure via references/	Three-tier cost model
Dependency management	Not built-in, referenced by name	depends: front matter with recursive loading

An Emerging Engineering Discipline

Both articles point toward an emerging field: Agent Knowledge Engineering. It is fundamentally different from traditional Prompt Engineering:

	Prompt Engineering	Agent Skill Engineering
Granularity	Single conversation	Cross-session persistence
Focus	Output format and quality	Routing, loading, maintenance
Measurement	Output scoring	Trigger precision/recall
Maintenance	One-time authoring	Continuous flywheel iteration
Core asset	Prompt text	Directory structure + scripts + Gotchas

As Anthropic's Thariq Shihipar puts it: "The best Skills start with a few lines and a Gotcha, then grow as the model encounters new edge cases."

The AI engineers of the future won't just write prompts — they'll design, maintain, and measure "Agent knowledge assets." Skills are the standard carrier for that kind of knowledge.