Hands-On: Most Claude Code Optimization Levers Cap Out Below 2%, and Rules Stop Working After 3 Days

A developer ran a month-long tracking experiment on Claude Code's token consumption, and the conclusion was unexpected: after the initial diagnostic pass, the ceiling for almost every optimization lever came in below 2% of weighted cost. Some changes made no difference at all, and a few actually made things more expensive.

The developer had previously found that 94% of working time was spent re-reading context, so they listed every possible optimization lever and validated them one by one. The result wasn't the expected large savings, but a pile of ineffective operations—and a major error in their own measurement tooling.

What Got Fixed First Was the Ruler

Midway through the experiment, the developer noticed that in 8 days of usage statistics, total token count and API call count didn't line up. The cause was subtle: Claude Code writes one line per response block (text, tool call) in the transcript file, but the usage block is duplicated in every line. The script that summed line by line was counting each call an average of 1.9 times.

The fix was simple—group by requestId and count only once. After the correction, 8 days of data dropped from 1.003 billion tokens to 540 million, and the drag from tool results was nearly halved.

This error affected the absolute numbers in the previous article: 313 sessions became 294, and 2.356 billion tokens became 1.212 billion. But the re-read share rose from 94% to 96%, and the curve's exponent went from 1.20 to 1.19. The numbers were halved, but the structure of the argument didn't change.

The second calibration issue was the character-to-token conversion. The conventional rule is about 3.5 characters per token, but calibrating against the author's own data showed that Spanish prose is actually 2.1 characters per token. This means the actual cost of memory notes isn't the previously calculated 7%, but 10%.

Neither correction changed any conclusion, but both changed the potential payoff of every lever—and that's precisely the only basis for deciding what to optimize.

Measure State, Not History

The second error was subtler: when you change the agent's configuration (instructions, hooks, read scripts), the average over the past 45 days mixes two different operating states, describing neither accurately.

The author ran into this with memory notes. Looking at the full history, the diagnosis was clear: nearly half of the content read from notes was unnecessary accumulated history, so it clearly made sense to split each file into two. But just three days earlier, the author had changed the reading method—the agent first requests a section map with per-section costs, then fetches only the section it needs. Looking only at post-change data: 2 full-file reads versus 198 map-based or per-section reads, the waste from stepwise reading was only 1% compared to full-file reading. Splitting files would be zero-return work.

After that, every measurement script supported --desde and --hasta parameters for time slicing, and split by call rather than by session. There's also a separate regimen.py script that warns when the statistical window crosses a configuration change.

What Works Is Mechanism, Not Text

The biggest improvements this month had one thing in common: none depended on the agent remembering anything. They were all automatically running code.

Cost laddering. The problem was a 564-line memory note of about 17,000 tokens containing 29 sections. Faced with a specific question about one section, the only realistic option was to open the whole thing. So two options were added to the reader: one lists sections with their line ranges and costs (about 1,200 tokens), and another fetches just a single section (about 1,300 tokens). The core principle is see the cost first, then decide whether to pay, with each level costing roughly one-tenth of the next.

Blocking duplicate reads. Auditing a month of memory read records, 47% were reads of the same file within the same session—the text was already in context. The solution was a hook that rejects a second full read of the same file. This is the only interception that loses no information, because what it blocks has already been loaded.

Routing. A hook suggests possibly relevant memory notes on every message. Originally, hand-written trigger words matched against the message; now BM25 runs against note bodies and the first three messages of the session. To validate the effect, the author built a test bench from transcript files: for each message, which notes were actually opened to answer it. In scenarios where routing is the only entry point, the hit rate for the top four suggestions rose from 41% to 76%, and token consumption per message was halved. Counting only notes not yet in context, the hit rate rose from 43% to 68%.

This test bench also corrected a number the author had previously treated as settled. They had believed word-based routing couldn't exceed 70% accuracy because nearly a third of messages had no searchable content (like "check if everything's right"). But that was measured looking at each message in isolation. Considering the previous two messages together, many of these messages actually have content to search.

There was also a decision not to act. Fourteen notes had never been opened across 393 sessions, so deleting them seemed obvious. But checking them one by one showed that "never opened" doesn't equal "redundant": two were in the always-loaded index, precisely because they work without being opened; several others contained rules written nowhere else. They weren't opened simply because the topic never came up. Deleting them would mean losing knowledge to save tokens that cost nothing when they aren't opened.

BM25 is a classic information retrieval ranking function belonging to the "bag of words" model family, distinct from the semantic similarity approach of vector retrieval. It scores each candidate document by combining three factors: how frequently the query term appears in that document (term frequency, TF), how rare the term is across the entire corpus (inverse document frequency, IDF), and a penalty for long documents (document length normalization). BM25 needs no embedding model, no vector database, and incurs no additional fixed context overhead, making it especially suitable for high-frequency, low-latency scenarios like "quickly filtering candidate notes on every message." In the article, the author replaced hand-written trigger words with BM25 running against note bodies and the first three messages of the session, which both improved routing hit rate and avoided the hidden cost of a vector library injecting fixed text at the start of every session.

Progressive disclosure is a layered loading strategy in context engineering: rather than stuffing all information into context at once, first provide a lightweight table of contents or index, letting the model expand layer by layer as needed. claude-mem uses a typical three-tier structure—index, section map, specific content. The cost model of this design is that "each level costs roughly one-tenth of the next," so as long as a task on average only needs one or two expansions, it saves an order of magnitude compared to loading the whole thing. The author's system follows the same idea but additionally explicitly labels the token cost at each level, letting the model (and the developer) see the price before deciding whether to pay—an important enhancement to progressive disclosure.

Rules Written in Instructions Stop Working

Every API call re-reads the entire context, so an extra call costs more than almost any file. The author measured how many calls the agent makes per message and wrote an explicit rule: chain operations that don't depend on prior results into a single command, and launch independent tools together.

Ten days later, calls per message hadn't dropped—they had doubled.

That number is somewhat misleading. Messages per session dropped from 1.7 to 1.1: the author was sending longer tasks in a single message. Calls per message measures both task size and discipline, so it's an alarm, not a verdict.

What's truly comparable is the behavior the rule demanded. Shell commands containing two or more chained steps were 84-87% before the rule and 86% after—already chained anyway. Calls containing multiple tools rose to 47% within three days of the rule, then fell back to 18%, then to 10%. It worked for three days and then stopped. The same thing happened with another rule a few weeks earlier.

The author also tried the reverse mechanism: moving instructions from always-loaded files to files loaded only under specific paths (.claude/rules/ with paths:). It works, but only triggers when reading or editing with the file tool. If the agent reads via shell, the rule doesn't load. A month of data showed that in sessions involving that folder, the rule failed to load 78% of the time.

The conclusion: instructions written on paper only work for the first few days, then it's business as usual. If something matters, make it a mechanism (a blocking hook, a command that does three steps in one) rather than writing one more line of instruction.

A hook is an external script mechanism provided by Claude Code that automatically triggers on specific lifecycle events, such as before (PreToolUse) or after a tool call, at session start, or when the user submits a message. Unlike natural language rules written in instruction files, hooks are deterministic code: they either pass or block, independent of whether the model "remembers" the rule. The interception that "blocks a second full read of the same file," the routing suggestions, and the batch-read warnings in the article are all hook implementations. This is also the technical basis of the author's core argument—instructions are suggestions the model can ignore; hooks are mechanisms with a 100% execution rate. Migrating important constraints from instructions to hooks turns probabilistic execution into deterministic execution.

A Tour of the Small Levers

These are the remaining levers measured this month. The ceiling column assumes 100% execution rate—which doesn't exist. The cost column is the price of trying.

Lever	Ceiling	Cost or Result
Batch independent reads into one call	1.5-1.8%	Warning after each separate read costs 0.25%; only breaks even at 15% execution rate
Shrink tool parameters (long scripts, whole-file writes)	2.7%	This is the sum of all parameters. The "edit rather than rewrite" rule accounts for only about 0.1%
Load instructions only by path	0.2%	Doesn't load when the agent reads via shell
Trim global instruction files	0.18% saved per 10%	Abandoned
Split large memory notes	~1%	Stepwise reading already solved it
Don't search in the wrong place	0.25%	This is the floor. Should be argued from a quality angle, not a token angle
Use a folder map to locate files instead of searching	1.07%	Includes necessary searches
Abbreviate instructions	0	4.8% fewer characters, 0.6% more tokens
Remove accents and ñ	0	1% more tokens
Drop cache from 1 hour to 5 minutes	N/A	1 hour is already the default under the subscription plan

The abbreviation point deserves a bit more, since it's the most frequently asked question. The author's instructions are written in Spanish, and the tokenizer splits by word fragments, not by letters. Que, memoria, después are each one token on their own; q, mem, dsp are still one token each, and odd symbols ("c/", "x") actually add fragments. The correctly accented forms happen to be the most common in the vocabulary, so removing accents is actually more expensive. Reducing tokens comes from cutting content, not compressing it.

Splitting Sessions Doesn't Pay Off

The number-one lever in the previous article was starting a new session when changing topics. A month later, the measurements say splitting mid-task is a negative return, and sometimes the reverse is true.

The math goes like this: splitting forces you to rewrite the session prefix into cache (instructions, tool definitions, agent startup info), and writing costs 2 while reading costs 0.1. What you save is the cumulative redundancy of re-reading on every subsequent call. Splitting only pays off when that redundancy times the remaining number of calls exceeds about 19 times the prefix size. At the author's session lengths, that almost never holds.

Simulating on transcript files: "split when redundancy exceeds 25,000 tokens" gives -2.7%; a threshold of 50,000 gives -0.6%; above 75,000, no sessions get split at all. Even an "oracle" scheme that knows the future and pays no redirection cost only reaches 2.9%, and only across 4 sessions.

What actually pays off is the reverse operation: when a session is still light (within about 20,000 tokens above baseline, same folder, less than an hour apart), continue the next task in the same session. Measured +2.2% on the first day, and +1.7% to +1.9% after excluding sessions that actually ran in parallel. On the second day, only +0.5%.

This isn't a contradiction—the state changed. The first article measured a period of very long sessions of hundreds of turns, where splitting saved a third. Habits took effect: in the currently measured state, 43 of 54 sessions had only a single message, with a median of 13 calls. With sessions this short, the marginal benefit is on the other side.

The cost asymmetry between cache writes and reads is the key to understanding the economics of session splitting. In Anthropic's prompt caching mechanism, writing a piece of text to cache for the first time costs about 1.25 times a standard input token (recorded as 2 in the article), while a cache-hit read costs only 0.1 times the standard price (recorded as 0.1 in the article). This means once the cache prefix is written, it's reused nearly free on every subsequent call; conversely, creating a new session requires rewriting the prefix—system instructions, tool definitions, agent startup info—at 2x billing. Splitting saves the cumulative redundant re-reads in the old session, but the price is paying a write fee for the entire prefix. Only when "redundant tokens × remaining calls > prefix size × 19" do the saved read fees cover the write fee—which explains why splitting almost always loses money in short sessions.

Comparing with the Pioneers

Midway through the month, the author reviewed the most-cited memory handling projects: Anthropic's context engineering guide and memory tools, Manus's stable prefix approach, claude-mem's three-tier progressive disclosure, Letta's idle consolidation, and Mem0 and Zep.

Surprisingly, the system the author had built by measuring item by item was already on par with them. Suggested routing, the section map, and single-section reads are essentially claude-mem's progressive disclosure, just with the cost shown at every level.

The author decided not to install any of them, and not to introduce a vector database: they all add fixed text at the start of every session, which is exactly where you pay the most, and word-based routing already achieves a 75% hit rate.

The Measurement Method Is Itself the Product

The most important thing left from this month is the order of operations: before applying a lever, measure its ceiling using your own transcript files, assuming 100% execution rate. If the ceiling is small, it's not even worth writing the text to explain it. If the ceiling is large, prioritize mechanism over instruction. After applying, measure only from the moment of change.

The scripts are open-sourced in the context-economy repository on GitHub, and there are now six. They read the .jsonl files Claude Code leaves on disk and send data nowhere.

The author concludes: this data belongs to one person and their way of working. But the core message doesn't depend on that—after the initial diagnostic pass, the remaining gains are both small and scattered, and the only way to avoid spending a month optimizing a 0.2% lever is to know beforehand how much each lever can deliver.

One-line Summary

A month of tracking shows: after fixing the measurement tools, the remaining Claude Code optimization levers all cap out below 2%.

Paragraph Summary

A developer ran a month-long systematic tracking of Claude Code's token consumption, with an unexpected conclusion: after the initial diagnostic pass, almost every optimization lever caps out below 2% of weighted cost. The first thing fixed along the way was the measurement tooling itself—the usage block in transcript files was overcounted by 1.9x, and after correction the total halved but the conclusion structure was unchanged; the character-to-token conversion was also recalibrated against real data. The improvements that actually worked were automatic mechanisms rather than text instructions: cost-laddered reading, a hook blocking duplicate reads, and BM25-based routing. Meanwhile, rules written in instructions only worked for the first three days, after which behavior reverted to baseline. Splitting sessions was even a negative return in short-session scenarios. In the end, the author treats the method itself as the product: first measure each lever's ceiling using transcript files, then decide whether to act.