Explaining Grok Bot's Memory Mechanism: Why Does It Remember the Project but Still Re-check Materials?

Introduction: A Question That Confuses Many

Have you ever run into this situation: you've already told the AI assistant your project's deadline, but in the next conversation it says "unconfirmed"?

In the ninth episode of his video series, Bilibili creator "馬上花生波特記" raised a core question — Grok Bot remembers the project, so why does it still need to re-read materials? Behind this apparent contradiction lies a key distinction: how technical work is done versus knowing the current facts are two different requirements.

In other words, the Bot remembers "how to do it," but it may not know "what the current situation is."

The Three Layers of Memory: Long-term Preferences, Current Input, and Changing Facts

In the video, the creator showed a three-layer diagram (part of course preparation, not a product database) that divides the information the Bot needs to handle into three categories:

Long-term Preferences

Tell the Bot which sections the report should include and what format to follow. For example, "always keep the source and to-be-confirmed columns" — this is a stable rule, suitable for long-term memory.

Current Input

Explains what is being done right now. For example, "this time we're writing an analysis report about XX" — it's the context of the current task.

Changing Facts

Information that needs support from current materials or source systems. Data like deadlines, test status, and scheduling changes at any time and must be obtained from the latest materials, not from memory.

The value of this three-layer model is that it clarifies which information should be "remembered" and which must be "re-checked." Treating changing facts as permanent rules to remember is the root cause of many wrong answers.

Here, "memory" does not refer to knowledge solidified in the AI model's parameters, but rather to the combination of context state maintained at runtime by the dialogue system and externally retrievable information. Large language models themselves are stateless — each request is computed independently, and so-called "remembering" usually relies on two mechanisms: first, concatenating historical dialogue into the current prompt (context window); second, retrieval-augmented generation (RAG) via external storage (such as vector databases or file systems). The former has limited capacity and disappears when the session ends, while the latter requires explicit reading and writing. Therefore, "the Bot remembers the project" often just means that the current session's context happens to carry the project information; once a new conversation starts or the context is truncated, it returns to the "unconfirmed" state. Understanding this explains why stable rules can be written into system prompts or long-term memory, while changing facts must be re-retrieved from external materials each time — it's not that the Bot is "lazy," but that the architecture determines it.

A Specific Experiment: Changing from October 20 to October 22

The video used self-made version materials for a demonstration. The first material stated the task deadline as October 20, and the second clearly changed it to October 22, declaring it supersedes the old version.

At this point, if you directly ask the Bot "when is it due," it may answer with the old date from memory. The correct approach is: require it to re-read the attachment and list version differences and sources. Only changes supported by the new file can be adopted, and if no material is found, mark it "unconfirmed."

The replicable message template the creator gave is:

"Please continue using the report's source and to-be-confirmed columns, re-read this attachment, treat the new version as the current fact, and list the evidence that replaces the previous deadline. If you can't read it, don't fill in conclusions from old memory."

Note that two points are emphasized here: first, the returned content needs version and source, so people can tell whether it re-checked or just rephrased the old answer; second, the same file name and task title cannot prove that the new content has been read.

"Version differences and sources" are the core means of dealing with information timeliness. In document collaboration, different versions of the same file name may have completely different content, and file name or modification time alone cannot determine which is the authoritative version. Asking the Bot to list version differences essentially forces it to execute a retrieve-compare-cite chain: first locate the latest attachment, then compare old and new content item by item, and finally mark the source location of each conclusion. This has two benefits: first, it is auditable — people can trace back to the original materials through the sources; second, it is falsifiable — if the Bot did not actually read the file, it cannot provide specific difference items and locations, only vague statements. This is consistent with the idea of "citation tracing" in RAG systems — having the model output with evidence rather than generating plausible-looking answers purely from parameter memory.

Three Things That Must Be Checked in Real Tasks

The priority in the course case (new version replaces old version) cannot be directly copied as a universal guarantee. In real tasks, the creator suggests checking:

1. Who confirmed it — authority and responsible person
2. What is the scope — which parts are affected by the change
3. Where is the evidence — is the source locatable

If two materials conflict, first explain the differences and timing, then handle it according to the authoritative source agreed upon in the task. If there is no agreement, ask directly, because a newer file may also be a draft, and modification time alone should not decide for you.

The Actual Capability Boundaries of the Current Personal Bot

The video specifically clarified several easily misunderstood points:

• The current personal Bot has no old description box or independent settings page; responsibilities and long-term preferences need to be explained in chat
• Names and tags mainly help with identification and are not equivalent to a complete memory system
• XAI's "call for help" page still has a description entry (note: this refers to X platform's settings entry)
• Current course editing and work assistance have been updated, but this cannot be used to infer that the team Bot's description settings must also have been removed

The creator emphasized: having the Bot summarize the current role agreement is feasible and can help find misunderstandings, but this is only a verifiable explanation, not a complete memory export, and it cannot guarantee that all underlying records have been listed.

The difference between a "verifiable explanation" and a "complete memory export" involves AI systems' explainability and visibility of internal state. When the Bot summarizes the current role agreement, it is generating a natural language summary based on the current context and accessible configuration; this summary may be accurate, or it may omit or fabricate things. A "complete memory export" means the system can list all persistently stored rules, preferences, and facts with no omissions. Currently, most personal AI assistants do not provide such an interface, and users cannot directly view the underlying memory store. Therefore, treating the Bot's self-description as acceptance evidence is dangerous — it may just be a paraphrase of the prompt, not a query of the real storage. The reliable approach is indirect verification through behavioral testing: design a task that depends on specific memory and see whether the Bot executes as expected, rather than asking it "what do you remember."

How to Verify After Correction: Behavioral Testing Is More Reliable Than "I Remembered"

Just replying "I remembered" cannot replace behavioral verification. The creator suggests:

• After correction, use a new task to check again
• When switching to another Bot, explain responsibilities, current files, and unconfirmed items. Files can be shared, but a new role cannot know which version you previously adopted or what acceptance criteria you used just because a file exists
• Don't let multiple Bots overwrite the same file at the same time. Use one primary version and a clear handoff message stating who takes the next step, what content is changed, and which decisions are still waiting for you

If you find it used the old deadline, point out the specific invalid item, why it is invalid, and the new source, and require revision of the same result, rather than just saying "the memory is wrong" and letting it guess which sentence you don't accept.

Three Criteria for the Second Round of Acceptance

At the end of the video, a concise acceptance framework is given — look at three points:

1. Stable sections are still there — were long-term preferences followed
2. The deadline indeed comes from new material — were changing facts re-checked
3. If the old date is no longer used, explain why — is the source traceable

If only the date changed but the source still points to the old version, this result still fails acceptance.

Add one more boundary test: give an input missing a deadline and see whether the Bot honestly marks it "to be confirmed" rather than automatically filling in the 22nd. Being able to report gaps is also an important capability in the coordination between memory and task context.

"Being able to report gaps" is an important indicator of an AI assistant's reliability, often called honesty or calibration in the AI safety field. A model that only pursues "answering every question" tends to fill gaps with plausible guesses when facing missing information, which is precisely the source of hallucinations. A well-designed system should clearly output "unconfirmed" or "needs supplementary materials" when information is insufficient, rather than automatically filling in a number. This behavioral test is effective because it simultaneously examines two capabilities: first, whether the model is aware that it lacks key information (metacognition); second, whether it is allowed or encouraged to express uncertainty. In task acceptance, giving an input missing a deadline sets a "trap" — if the Bot honestly marks it as to be confirmed, it shows that its retrieval and judgment chain is honest; if it fills in the 22nd, it shows that it is still relying on old memory or making unsupported inferences.

Core Conclusion: Remember the Way of Working, Re-check Changing Facts

The creator summarized clearly at the end of the video: remember the way of working, re-check changing facts. This is not about having you introduce yourself from scratch every time, but about separately explaining rules that need to be stable and data that needs to be updated, so that subsequent work has a reliable basis.

The next episode will save an already-accepted method as a skill, then switch inputs and retest — when saving, keep the method and judgment rules, and don't secretly turn a meeting's name and date into a permanent answer. This direction is worth continued attention.