AI Agent Documentation Guide: A Full Breakdown of llms.txt, AGENTS.md, SKILL.md, and MCP

Developer tool documentation is undergoing a structural shift. In the past, READMEs, API references, and quickstarts were written for humans. Now they also need to be read, understood, and executed by coding Agents. Around this need, a set of new documentation formats and protocols is rapidly taking shape: llms.txt, AGENTS.md, SKILL.md, MCP, and the still-early auth.md. Understanding what each one solves matters more than blindly adopting all of them.

Four Types of Resources, Each Handling a Different Segment

Looking at these formats together, you'll notice they correspond to four different stages an Agent encounters when using a developer product:

Resource	Primary Purpose	Location
llms.txt	Help Agents find relevant documentation	Website or docs site
llms-full.txt	Merge documentation into a single file	Website or docs site
AGENTS.md	Provide in-repo working instructions	Repository, loaded per Agent rules
SKILL.md	Define reusable workflows and supporting resources	Skill directory
MCP	Connect Agents to tools and data	MCP Server connected to the client
auth.md	Describe service registration and authentication	Service website

In addition, each major coding Agent has its own dedicated instruction file: Claude Code uses CLAUDE.md, Gemini CLI uses GEMINI.md, Cursor uses .cursor/rules/, and GitHub Copilot uses .github/copilot-instructions.md. These files differ in how they're loaded, their scope, and their priority.

llms.txt: A Documentation Map for Agents

llms.txt is a Markdown file placed on a website that uses a brief introduction plus a list of links to tell Agents "where the docs are and which ones are worth reading." Agents can follow relevant links based on the task at hand, without stuffing the entire site into context.

The original article presents a typical structure: a title, a one-line product description, then grouped links with brief descriptions. The companion llms-full.txt merges documentation into a single file. It looks convenient but can consume a lot of unnecessary context. Platforms like Mintlify can auto-generate both files.

It's important to be clear-headed about this: publishing llms.txt doesn't mean Agents will use it. What you really need to verify is whether Agents can discover the file, whether they follow the relevant links, and whether they can find the information needed to complete the task.

llms.txt was proposed by Jeremy Howard of Answer.AI in 2024, positioned similarly to a website's robots.txt or sitemap.xml, but aimed at large language models rather than search engine crawlers. It's conventionally placed at the site root path (e.g., example.com/llms.txt), uses plain Markdown format, and lets Agents grab the navigation structure of an entire site's documentation with a single lightweight request. Its companion llms-full.txt concatenates all documentation body text into a single file, suited for scenarios with ample context windows that need to ingest everything at once. Currently, llms.txt is not an official standard of any kind — adoption is entirely up to site maintainers. Anthropic, Stripe, and others have already implemented it, but there's still no unified convention on whether mainstream Agents proactively probe for this file.

AGENTS.md: Operating Instructions Within a Repository

When Agents start working inside a repository, they need to know how to install dependencies, how to run tests, and what coding conventions to follow. AGENTS.md is the dedicated place for these instructions.

Take the AGENTS.md in the Quantiles repository as an example. The file opens by clearly stating "This file is for coding assistants such as Codex, Claude Code, Cursor, GitHub Copilot, and Gemini CLI," then lists the files to read before starting a task and the rules for making changes.

Different Agents handle the loading and priority of AGENTS.md inconsistently — Codex loads repository guidance in layers, while other Agents may have their own strategies. Instructions that are too cluttered or that go stale over time can actually slow down workflows, increase task complexity, and even cause Agents to make mistakes when interacting with the product.

SKILL.md: Letting Agents Complete Multi-Step Workflows

Some tasks can't be covered by a single repository instruction. An Agent might need to install a CLI, configure credentials, execute multiple steps, and finally verify the results. Agent Skills are designed for exactly these scenarios — providing reusable instructions and supporting resources.

A typical Skill directory structure includes a SKILL.md main file, supplementary docs under references/, and helper scripts under scripts/. Among the name, description, and instructions in SKILL.md, the description is especially critical — it determines whether an Agent can judge "when to use this Skill."

Both OpenAI Codex and Anthropic's Skill implementations use "progressive disclosure": the Agent first sees a brief description, and only loads full instructions and supporting resources when needed. But this assumes the Skill is installed somewhere the Agent can discover it — just dropping a SKILL.md into a repository doesn't guarantee the Agent will find it, and even if it does, it may not accurately judge whether and how to use it.

MCP: Standardized Connections to Tools and Data

Documentation helps Agents understand a product, while MCP provides a standardized way for AI applications to connect to a product's tools and data. An MCP Server can expose three types of capabilities:

• Tools: Operations an Agent can invoke, such as creating a project or starting an evaluation
• Resources: Information an application can retrieve, such as evaluation results or execution logs
• Prompts: Reusable templates that guide specific tasks or interactions

A concrete example: if a Quantiles MCP Server existed, an Agent could create and compare evaluation results through structured tool calls, without manually assembling API requests or typing CLI commands. Of course, which capabilities an Agent can actually use depends on what its MCP client supports.

MCP and SKILL.md can work together: the Skill tells the Agent what operations to perform in what order and how to verify results, while MCP provides the tool entry points needed to execute those steps.

MCP (Model Context Protocol) was open-sourced by Anthropic in late 2024. It uses a client-server architecture: the Agent application acts as the MCP Client, the product side implements an MCP Server, and the two communicate via JSON-RPC 2.0 messages. Beyond defining the three primitives of Tools, Resources, and Prompts, the protocol also includes initialization handshakes, capability negotiation, and sampling mechanisms, enabling Agents from different vendors to connect to external capabilities in a unified way without writing custom integration code for each product. Compared to traditional Function Calling, MCP moves tool definitions and invocation logic out of the Agent and into an independent service, allowing product teams to iterate on tools independently without modifying Agent code. On authorization, the MCP spec recommends using OAuth 2.1 to protect remote Servers — which is exactly why the article emphasizes that MCP authorization and auth.md operate at different layers.

auth.md: How Agents Register and Obtain Credentials

An Agent can find documentation and understand the API, yet get stuck at "registering an account and obtaining credentials" — a real problem developers often encounter. WorkOS's proposed auth.md targets exactly this stage, and it's still in early adoption.

It lets services describe registration, authentication, and available permissions in a Markdown file, supporting two modes:

• Agent verified: A trusted Agent provider verifies the user's identity, and the service can register the Agent without additional user interaction
• User claimed: The Agent initiates registration, and the user logs in and confirms a verification code to authorize access

Both approaches ultimately result in the Agent obtaining a scoped access token. But publishing auth.md is only the first step — the service side still needs to implement authentication endpoints, token issuance, and permission checks. Be careful to distinguish auth.md from MCP authorization: MCP authorization governs access to protected MCP Servers, while auth.md governs how an Agent first registers an identity with a service.

As an emerging proposal, its compatibility and Agent support still need validation. When account ownership, permissions, or billing are involved, a human approval step may still be unavoidable.

WorkOS's auth.md targets the traditionally human-designed process of Agents autonomously completing "service registration." In existing authorization frameworks like OAuth, clients typically need to pre-register and hold a client_id/secret, whereas Agents are often ad-hoc callers with no pre-established identity — which raises the question of "how does an Agent prove it's a trusted proxy and which user it represents?" auth.md uses Markdown to describe endpoints, letting Agents read first and then follow the map to complete registration — an approach in the same spirit as llms.txt. Currently, it overlaps functionally with OAuth Dynamic Client Registration (RFC 7591) and other agent identity proposals. Whether it will become a de facto standard remains uncertain, and you should evaluate implementation cost and ecosystem compatibility before adopting it.

Which Formats to Use and How to Evaluate Results

The starting point should be "where do Agents actually get stuck when using your product," not "I need whatever everyone else has." A simple library may only need accessible documentation; a developer platform with complex workflows is more likely to need Skills, structured tools, and authentication guidance.

For evaluation, start with representative workflows — installing the product, completing the quickstart, configuring authentication, using advanced capabilities — and compare performance with and without the corresponding resources. Focus on four dimensions:

• Discovery and activation: Does the Agent find and use the relevant documentation, Skill, or tool?
• Task success and reliability: Can it complete tasks correctly and consistently across multiple trials?
• Efficiency: How much time, tokens, and tool calls are consumed?
• Errors and recovery: Where does the Agent get stuck, and can it recover without human intervention?

These resources aren't mutually exclusive, and not every product needs the full set. Adding more instructions isn't always the better answer — what matters is whether the ones you have actually help Agents complete developer tasks.

As Agents evolve rapidly, a solution that works for one Agent today may stop working tomorrow with a different Agent or after the same Agent gets upgraded. Maintain a set of representative developer tasks, re-run evaluations as products, documentation, and Agents change, and use execution traces to pinpoint problems — sometimes the fix requires instructions, and sometimes a clearer error message, a simpler installation process, or a more useful tool response is the more effective answer.

One-line Summary

An overview of new documentation formats and protocols for coding Agents — llms.txt, AGENTS.md, SKILL.md, MCP, auth.md — emphasizing evaluation over blind adoption.