AI Has No Memory: Understanding the Core Logic of Token Costs and Context Management

AI has no memory and re-sends the full context every step; cost is roughly steps multiplied by information package size.
The article uses a vivid "photocopier" metaphor to reveal how LLMs work: AI has no memory, and the "remembering" you feel is the tool re-sending all content from the start of the conversation to the model each time. This determines the essence of cost—cost ≈ steps × information package size—not any single message. In Agent mode, each step re-copies an ever-expanding package, and one request can trigger hundreds of steps and tens of millions of tokens. Worse, larger packages reduce accuracy, and compaction loses details. The article recommends monitoring usage via Cursor's context meter (or Claude Code's /context) and acting once it exceeds 50%; good habits all reduce steps, shrink the package, or lower per-step cost.
Many people develop an illusion when using AI chat tools: it seems to remember what was said ten minutes ago. But the truth is, AI has no memory at all. The "memory" you perceive is actually the tool re-sending the entire content from the start of the conversation to the present with every interaction. Understanding this is the foundation for understanding Token costs, context windows, and all AI usage techniques.
AI's "Amnesia" and the Photocopier Model
Imagine AI as a brilliant consultant who suffers from complete amnesia. Every time you ask a question, he forgets everything before it. To get him to answer correctly, you have to hand him the entire case file again—including your instructions, the documents already read, and every reply he has given.

This is like a photocopier: for the first question, you copy 10 pages; for the tenth question, the file has grown to 100 pages, and you copy all 100 pages; for the thirtieth question, 300 pages, and it is still copied in full. The cost does not come from the first message, but from this ever-expanding information package that is repeatedly duplicated.
Token is the basic unit of text processing for large language models, and also the basic unit of billing. A token roughly corresponds to an English word or word root, or 1–2 Chinese characters, with the specific segmentation determined by the model's tokenizer. The model itself does not understand characters; it splits input text into token sequences, converts them into vectors for computation, and generates output token by token. Therefore, the "information package size" mentioned here is actually the total number of tokens sent to the model (input tokens).
Context Window is the maximum number of tokens a model can receive in a single pass, and different models have different upper limits, commonly hundreds of thousands of tokens. It is a hard limit: once a conversation or Agent's information package approaches this limit, the tool must truncate or summarize historical content to continue—this is precisely the root cause of the "compression" mechanism discussed later. Only by understanding the context window can you understand why the file in the "photocopier" cannot grow infinitely thick.
The Agent Mode Amplifies the Problem: More Steps, Higher Costs
In Agent mode, this problem is dramatically amplified. AI asks itself: reading a file is one step, running a command is another step, re-reading the result is yet another step. Each step sends the complete information package into the photocopier again, plus whatever it has just read. A single request you send may trigger 20, 50, or even over 100 copies.
According to a real case on the Cursor forum, a developer sent just one message asking to upgrade React across the entire project. The Agent executed 155 steps, each re-sending an ever-growing information package, eventually reaching 180,000 tokens per single step and over 11 million cumulative input tokens—all from just one message.
Therefore, the entire course has only one core formula: Cost ≈ Steps × Information Package Size.

All good usage habits essentially do one of three things: reduce the number of steps, shrink the information package, or lower the cost per step.
The Larger the Information Package, the Less Accurate AI Becomes
Beyond cost, there is an equally important issue: the thicker the information package, the lower the Agent's accuracy. When AI speeds up, it struggles to identify the truly critical parts from massive content. An instruction at the beginning of a conversation may be buried under hundreds of pages.

When the information package becomes too large, the tool automatically summarizes to free up space—this is compaction. It is practical, but summarization inevitably loses details. So a lightweight information package is not only cheaper but also produces better results.
How exactly does compaction work? When input tokens approach the context window limit, the tool calls the model itself to generate a summary of earlier conversation history, replacing the original content with this summary to free up space for new interactions. This process is usually transparent to the user, but it comes at two costs: first, the summary is generated by the model and may omit or distort key constraints (such as a specific version number or a particular requirement); second, the summarization itself requires an additional model call, which counts toward cost.
It is worth noting that compaction often happens when the information package is already very large, and a large information package has already reduced the model's accuracy—in other words, compaction is a "remedy" rather than an "optimization." Proactively controlling information package size (e.g., starting a new session in time, clearing irrelevant files) is more reliable than relying on automatic compaction.
How to Check Your Context Usage
The good news is that this information package is visible. In Cursor, there is a small ring next to the input box—that is the context meter. Click it, and you can clearly see what is filling it: base instructions, rules, available tools, connectors, and conversation content. You will find that even after sending just one message, the information package may already be quite full—some content was filled in before you even started working.

In Claude Code, the corresponding command is /context; in anti-gravity, it is located in the settings.
A simple rule recommended by the Cursor team is: take action once usage exceeds 50%. Above this threshold, every step is more expensive and AI is less accurate. The course gives an example: Leah checks her meter and finds usage has reached 85%, while the information package still contains Monday's bug, even though she has spent an hour working on documentation. She is paying for this package with every message, and it is making her document answers vaguer.
Agent mode differs from ordinary conversation in a key way: the model can autonomously call tools (read files, execute commands, search, edit code, etc.) and decide the next action based on the results returned by the tools, forming a "think—act—observe" loop. Each round of the loop (i.e., each "step") requires re-sending the complete conversation history along with new tool results to the model.
This means the number of Agent steps is not directly controlled by the user, but is jointly determined by task complexity, tool feedback quality, and the model's planning ability. A vague instruction may cause the Agent to repeatedly try and fail, causing the step count to explode; while clear instructions, explicit file scope, and verifiable goals can significantly reduce the number of steps. This is precisely the concrete lever for the optimization direction of "reducing steps" discussed later.
Core Conclusions
AI has no memory; the entire information package is re-sent at every step; cost equals steps multiplied by information package size; and the meter tells you your current state. Master these three insights, and you can make smarter choices in daily use—fewer steps, smaller information packages, higher accuracy.
One-line Summary
AI has no memory; every step re-sends the full context, and cost is roughly steps multiplied by information package size.
Paragraph Summary
The article uses a vivid "photocopier" metaphor to reveal the truth about how large language models work: AI has no memory, and the "remembering" you perceive in a conversation is actually the tool re-sending all content from the start of the conversation to the present to the model with every interaction. This mechanism determines the essence of cost—cost ≈ steps × information package size—rather than coming from any single message. In Agent mode, each step such as reading files or running commands re-copies an ever-expanding information package; one request may trigger hundreds of steps and accumulate tens of millions of tokens. More seriously, the larger the information package, the harder it is for the model to identify key instructions, and accuracy declines accordingly; although tools use compaction mechanisms to free up space, summaries inevitably lose details. The article suggests monitoring usage through Cursor's context meter (or Claude Code's /context command) and taking action once usage exceeds 50%; all good usage habits ultimately boil down to reducing steps, shrinking the information package, or lowering the cost per step.
Related articles

The 15-Year Programmer Who's Been Unemployed for Two Years: The Tech Worker's Dilemma in the AI Era
A US senior software engineer with 15 years of experience has been unemployed for two years, exhausted his savings, and sought help on Hacker News. This article analyzes the structural causes of long-term unemployment among tech workers, strategies for addressing resume gaps, and the redefinition of experience value in the AI era.

Soundcore Liberty 5 Pro Drops to All-Time Low: The Go-To True Wireless Earbuds for Commutes and Calls
Soundcore Liberty 5 Pro drops to an all-time low of $129.99 (from $169.99). These Anker true wireless earbuds excel at call quality and are on sale at Amazon, Best Buy, and Walmart in four colors—ideal for commutes and online meetings.

After Anthropic's IPO, How Far Can Safety-First Go?
The Verge podcast explores Anthropic's post-IPO core tension: can a public company stay safety-first? How public market pressure reshapes AI lab safety decisions.