[KongchangAI]
· 3 min read· 1,611 words

Data Reveals: Why /init-Generated CLAUDE.md Files Score Higher

Data Reveals: Why /init-Generated CLAUDE.md Files Score Higher

An evaluation of 8220 CLAUDE.md files shows /init-generated files score systematically higher than hand-written ones.

Reporails scored 8220 CLAUDE.md files from popular GitHub repos using ails check, finding /init-marked files have a median score of 8.4 vs 7.7 for unmarked files, leading by 0.6-0.9 points in every size bracket. The difference is structural: /init files are ~80% repository map, with 87% including Architecture and 84% Commands sections vs ~20% for others. They have fewer vague instructions and contradictory prohibitions. The only counter-indicator is more instructions buried in descriptions. The article recommends keeping the /init structure and provides self-check commands and three principles for manual rules.

8220 CLAUDE.md files from GitHub repositories with 10k+ stars were all put to the same test — the Reporails ails check scoring system. The results show a clean stratification: files generated by Claude Code's /init command outperformed other files in every file size bracket.

rss source: State of AI Instruction Quality: /init Files Score Higher

One Unremarkable Line Divides Two Groups

The method for identifying /init files is surprisingly simple: they contain this line within their first 4000 characters —

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Of all scored files, one in five carries this marker — 1641 files across 1539 repositories. These files have a median score of 8.4 (out of 10), compared to 7.7 for files without the marker. The gap extends beyond the median: 60% of /init files score 8 or above, while fewer than half of non-/init files reach that threshold.

Given that shorter files naturally accumulate fewer issues, the researchers divided the 8220 files into four equal groups by size. The conclusion held — /init files led in every size group, with an advantage of 0.6 to 0.9 points.

CLAUDE.md is a project-level instruction file that Claude Code automatically reads at startup. Placed in the repository root, its contents are injected into context as part of the system prompt. /init is a built-in slash command in Claude Code — when executed, Claude scans the codebase and automatically generates an initial CLAUDE.md, typically containing build commands, testing methods, and project structure descriptions. Since the generation process is handled by the model, the file structure is relatively consistent and includes a fixed introductory line.

Reporails is a static analysis tool for AI instruction files, with ails check being its command-line scoring command. It parses files like CLAUDE.md into structured instruction units, then scores them across several dimensions: whether specific files or commands are named, whether executable code blocks are included, whether rules contradict each other, whether instructions are vaguely worded, and so on. The maximum score is 10, with higher scores meaning the agent can more easily execute accurately.

Map First, Rules Second

Claude Code's official documentation describes /init as: "Claude analyzes your codebase and creates a file with build commands, test instructions, and project conventions." The actual output files indeed match this positioning.

About one-fifth of the content in /init files tells the agent what to do; the rest describes the repository itself: where things are located, what each command does, how the pieces connect. By comparison, other CLAUDE.md files have instruction density closer to three-tenths — double the ratio.

This difference shows up structurally. 87% of /init files include an Architecture section, and 84% include a Commands section. Among non-/init files, the proportion with either section is around one-fifth. The gap in code blocks is even starker: nearly 90% of /init files contain at least one code block, versus only 44% of other files. This gap holds steady across all size groups.

Notably, this "repository map as primary, rules as secondary" structure itself contributes part of the scoring advantage. 646 files without the /init marker but with both Architecture and Commands sections have a median of 7.9 — higher than the 7.7 for ordinary non-/init files, but lower than the 8.4 for /init files. This suggests that getting the structure right boosts scores, but /init's advantage doesn't come entirely from structure.

Large language models exhibit a "lost in the middle" phenomenon when processing long contexts: information in the middle of a prompt tends to be overlooked, while content at the beginning and end carries higher weight. Therefore, a CLAUDE.md full of rule entries isn't necessarily more effective than a file that first describes the repository structure and then lists a small set of rules.

The purpose of a repository map is to give the model spatial reference — telling it which directories correspond to which functions, which command does what. When the model knows "tests are under tests/, run with pnpm test," it can infer on its own when encountering related tasks, without needing a separate rule for every scenario. This is the structural reason why /init files lead in scores: they convert large amounts of tacit knowledge into explicit but compact descriptions, reducing the number of rules and the probability of conflicts between them.

Fewer Rules, Fewer Weak Spots

Fewer instructions mean less room for error — the data supports this intuition.

/init files show fewer "hedged instructions" (vague directives using wording like "should" or "try to") in every size bracket, and fewer cases of "never" contradicting a corresponding permission (for example, writing "NEVER push to remote" on one line while writing "you can push if asked" on another). Prohibitive instructions also make up a lower proportion of all instructions: 16.8%, compared to 21.2% for other files.

Another metric points in the same direction: about one in every four instructions in /init files fails to name a file, command, or tool, versus nearly one in three for other files.

But one data point runs the other way. Because /init files are primarily descriptive, narrative text on the same topic appears immediately adjacent to its instructions, and the checker flags "instructions buried in descriptions" — occurring 17.7 times per 100 instructions in /init files, versus 10.5 for other files. At any file size, /init files show no advantage on this metric.

A "hedged instruction" refers to a directive with uncertain tone, such as "you should probably run tests" or "try to keep functions small." This kind of wording is polite in human collaboration documents, but for an agent it means the execution boundary is blurred — the model may selectively ignore it or over-execute it. The checker flags such wording as a potential issue.

An "instruction buried in a description" refers to a rule that should stand on its own line being embedded in a narrative paragraph — for example, casually mentioning "don't modify the dist/ directory" while introducing the build process. When parsing, the model may treat it as background information rather than a binding constraint. Because /init files have more descriptive content, this situation occurs more frequently — it's the only metric where /init clearly lags. Pulling such rules out separately, with a heading or bold formatting, can eliminate this risk.

How to Make the Most of Your Own File

If you want to know where your CLAUDE.md stands, you can run:

head -5 CLAUDE.md
npx @reporails/cli check -v

The first command checks whether the file starts with that /init line; the second gives a score and lists issues with line numbers. Then use git blame CLAUDE.md to determine whether those lines came in with the initial file or were added manually later.

The /init marker line itself can be deleted — it's treated as descriptive content by the checker, and removing it has no measurable effect on the score. What's worth keeping is its structure: repository map, commands in code blocks, a small set of rules.

This pattern applies equally to AGENTS.md or GEMINI.md. The format is the same: a repository map, commands in code blocks, and a short rules list.

When you need to add rules manually, there are three principles: give rules the context /init would provide, putting commands in code blocks; for "never"-type prohibitions, state the permitted cases as clearly as the prohibited ones; keep explanatory text brief — there's already plenty in the file.

This matters especially in the data: 60% of /init files are the only instruction file in their project, versus only 30% for non-/init files. For many repositories, that repository map, commands, and small set of rules are all the information the agent gets. Claude Code's documentation adds right after describing /init: "On top of that, refine it with instructions Claude can't discover on its own." That part requires manual work from you.

One more data point worth mentioning: across all 10000 repositories, the longest main instruction file reached 129,121 words — and it also starts with that /init marker line.

AGENTS.md and GEMINI.md are similar convention files adopted by other AI coding tools. AGENTS.md is promoted by teams including OpenAI, positioned as a cross-tool universal agent instruction file; GEMINI.md is the project context file read by Gemini CLI. Their format requirements are essentially the same as CLAUDE.md: written in Markdown, placed in the repository root, automatically loaded at startup.

Different tools prefer different filenames, but the content structure can be reused. If a repository uses multiple tools simultaneously, you can maintain one primary file and point to it via symlinks or short forwarding files, avoiding rule inconsistencies across multiple files. ails check also supports scoring these files, so the same improvement methodology can be transferred over.

One-line Summary

An evaluation of 8220 CLAUDE.md files shows that Claude Code's /init-generated files score systematically higher than hand-written ones.

Paragraph Summary

Reporails used ails check to score 8220 CLAUDE.md files from GitHub repositories with 10k+ stars, finding that files with the /init marker have a median score of 8.4, higher than 7.7 for unmarked files, and lead by 0.6 to 0.9 points in every file size bracket. The difference stems from structure: about 80% of /init file content is repository map, 87% include an Architecture section and 84% include a Commands section, far exceeding the ~20% for other files; instruction density is lower, with fewer vague instructions and self-contradictory prohibitions. The only counter-indicator is that "instructions buried in descriptions" occur more frequently. The article recommends preserving the /init structure — repository map, commands in code blocks, brief rules — and provides self-check commands and three principles for manually adding rules.

#CLAUDE.md#/init#Claude Code#AI instruction quality#Reporails
Share:

Related articles