What Should CLAUDE.md Contain? The Truth About "Don't Make Mistakes" Instructions, Tested

What Should Context Files Actually Contain? A Hands-On Test of "Don't Make Mistakes"

When writing a CLAUDE.md (or AGENTS.md) for an AI coding tool, most people's first instinct is to stuff it with every rule imaginable. But a YouTube creator ran a systematic series of tests, and the conclusion may surprise you: the rules that actually matter are rarely the vague "best practices" — they're the project decisions the tool can't infer from the code.

This test covered Claude Code, Codex, and GitHub Copilot, using a small Java room-booking library as the testbed, validating rule by rule whether each one actually changed tool behavior.

Principle One: Prioritize Decisions the Tool Can't Infer

The creator's first principle is clear: prioritize decisions the tool cannot reliably infer. If a certain pattern already appears everywhere in the code, the tool will simply copy it — there's no need to remind it in the file.

The experiments confirmed this. With no context file at all, a new class was placed in the correct package path in nine out of nine runs. In other words, rules like "service classes go in the service package" offer zero benefit for the current task.

The author calls these "control group" rules — they're included to confirm the tool already gets it right, not to prove the file works. What genuinely deserves a place in the file are conventions that are important, easy to overlook, or repeatedly broken.

How AI coding tools infer context: Before generating code, tools like Claude Code and Codex first scan existing project files (package structure, naming conventions, build scripts) and continue or complete based on those patterns. This mechanism relies on "in-context learning" — the model induces patterns from examples already in the prompt rather than relying on external memory. Therefore, any convention that appears repeatedly and clearly in the codebase (package paths, naming style, dependency injection approach) can typically be reused directly by the tool without restating it in CLAUDE.md. What genuinely needs to be written manually are the decisions "not expressed in the code but must be followed" — for example, historical baggage internal to the team, external compliance requirements, or a temporary compatibility layer about to be migrated. Understanding this explains why the creator emphasizes that "control group" rules offer no benefit: they describe patterns the tool can already infer, so writing them in is just confirmation, not additional information.

Principle Two: Rules Must Be Specific Enough to Check

"Make sure it runs" is not an actionable instruction. The creator wrote the actual command mvn -q test and required the tool to report failing tests and forbade it from claiming tests passed unless it had actually run them.

But there's a hidden pitfall here: an instruction is not a permission. In the creator's setup, Claude's permission mode blocked Maven, so none of the 12 runs could actually execute the tests — though Claude honestly reported "unable to run." After permissions were relaxed, only one of three runs actually ran the tests; the other two were blocked because the command form didn't match. Codex, meanwhile, executed the tests in all 12 runs.

The file guides; tests and checks enforce.

According to the screenshots, the author mentioned cost dropping from 52 cents to about 40 cents during the comparison. This is a reminder: before writing a command, confirm the tool actually has permission to execute it.

Permission modes and command interception: Tools like Claude Code run by default in a restricted permission sandbox that intercepts commands that could modify system state or execute external programs (such as mvn, npm, docker) unless the user explicitly authorizes them. This design prevents the AI from executing dangerous operations without confirmation, but it also renders test commands written in context files "toothless." The creator's experiment, where none of 12 runs executed Maven, was precisely the result of permission-mode interception. Notably, Claude honestly reported "unable to run," indicating that when permissions are insufficient the tool reports failure rather than silently skipping. Therefore, when writing context files, beyond writing commands clearly, you must also confirm whether the tool's permission configuration (such as allowedTools in settings.json or an interactive authorization flow) allows that command. Otherwise, no matter how specific the rule, it can't actually be executed — and the tool may even be misled into thinking the tests passed.

Principle Three: Define Boundaries, Not Absolute Bans

In a previous round of videos, the tool modified the README without being asked, and Claude twice replaced existing constructors, causing dependent tests to fail. In the new video, the creator added two boundaries to the file: "Maintain compatibility; if you can't, explain why" and "Don't touch the README."

What were the results? Without the file, the README was modified in two of nine runs (once by Codex, once by Copilot); with the file, zero modifications across thirty runs. The author cautiously notes that the sample size is small, so this is only a "noteworthy observation."

As for constructors, no replacements occurred — in thirty-nine runs, with or without the file, it never happened. This means the experiment cannot prove the file prevents constructor changes; the breakage in the previous round came from a different prompt.

Testing "Don't Make Mistakes": One Rule Saves 27% Tokens, But No One Can Explain Why

The most interesting experiment is this one: a context file containing only the single line "Don't make mistakes," compared against no file at all.

All nine runs (three per tool) passed the hidden tests and convention checks — matching the performance with no file. But on Claude, token usage dropped by about 27% and cost by about 19%, beyond the tool's own variance range. On Codex and Copilot, the differences fell within noise.

The creator's own eight real rules saved 43%, but the gap between the two numbers is smaller than run-to-run variance, so he can't conclude the rules were responsible, nor that "Don't make mistakes" had any actual effect. The key difference is that "Don't make mistakes" defines no checkable behavior, whereas the README rule produced a clearly measurable difference (zero modifications vs. two).

Six Things to Remove from Your File

The creator lists six categories of content that shouldn't stay in a context file:

1. Vague advice: "Follow best practices" — if there's a real rule behind it, write that rule out.
2. Existing documentation: point to it, don't paste it. Copies go stale, and the tool ends up with two answers.
3. Requirements for the current task: the file is "how we do things here"; the prompt is "what to do today."
4. Instructions that conflict with the prompt: in testing, override rules explicitly written in the prompt won, but the hidden tests failed on all three tools — because the tests expected the project format. Keep the two consistent.
5. Outdated rules: when the project changes, the file must change too.
6. Unreviewed generated files: having the tool draft one is a fine starting point, but none of the three tools wrote "Don't touch the README" or "Preserve constructors." A generated file is a starting point, not an endpoint.

Measured Data on File Names and Length

Regarding file names, the creator didn't trust the docs — he planted a fake rule in each file name ("write pineapple on its own line at the end of every reply") to test loading. Results: Claude Code read from both CLAUDE.md and a root-level AGENTS.md (all three hits each); Codex only recognized a root-level AGENTS.md; Copilot never output the marker in 15 attempts, though its own tool listed the file under .github — meaning the file was found, but compliance couldn't be confirmed.

On length, four scenarios were compared: no file, 8 lines, 20 lines, 60 lines. For Claude, with Maven blocked, 8 lines reduced tokens by 43%, 20 lines by 27%, and 60 lines by 38% — all beyond the variance range; for Codex, changes at each length were within variance; for Copilot, with no file its own variance reached 93% of the mean, making any discernible effect impossible to isolate.

In one sentence: across the 8-to-60-line range, no consistent performance degradation was found.

The creator also hid 10 additional rules in the 60-line file (split across hard "always/never" phrasing, soft "prefer" phrasing, and explanatory phrasing), and the script check found no violations — though the small task rarely gave the tool a chance to use these rules. The only weak signal: when the rule was absent, Codex wrote wildcard imports in three of nine runs; when the rule was present, none in three runs. The sample is too small to draw a conclusion.

Loading priority and scope of context files: Different tools have their own conventions for file names and locations. Claude Code supports both CLAUDE.md and a root-level AGENTS.md, and recursively reads files of the same name in subdirectories, forming a hierarchical context; Codex currently only recognizes a root-level AGENTS.md; GitHub Copilot relies on configuration files under the .github directory (such as copilot-instructions.md), but its loading behavior was never reliably triggered in the experiments. This divergence means that when collaborating across tools, a single file may not be read by all of them. The creator's "pineapple" fake rule was precisely designed to distinguish "file found" from "file followed" — Copilot's tool listed the file under .github yet never output the marker, showing that loading and execution are two separate stages. In real projects, it's advisable to choose the right file name based on the tools your team uses, or to maintain multiple symlinks pointing to the same content, so that rules don't apply to only some tools.

Practical Recommendations

Synthesizing all the testing, the creator's advice can be condensed into six points: prioritize conventions that tests are likely to miss; write specific, checkable instructions; include test commands and ensure permission to execute them; declare compatibility and change boundaries; delete duplicate, conflicting, and outdated rules; review the output and use tests to enforce what must hold.

The most surprising finding for the creator: the tool already copies the conventions shown in the code, and a single line of "Don't make mistakes" saved nearly as much as eight carefully designed rules. This is a reminder that context files are no panacea — they supplement review, they don't replace it.

This article is based on a hands-on test video from a YouTube channel; all data comes from the original video.

One-line Summary

Testing shows context files should contain decisions the tool can't infer; a single "Don't make mistakes" line saves 27% tokens but causation is hard to prove.