One Diary Gives AI Persistent Memory: A Semantic Retrieval Solution Tested

Title

One Diary Gives AI Persistent Memory: A Semantic Retrieval Solution Tested

Meta Description

A Bilibili creator open-sources an AI persistent memory solution: a lean persona stays in context, while dynamic info is written to a diary and retrieved semantically on demand. Hybrid vector and keyword retrieval, with three real-world pitfalls of the BGE model and how to solve them. Install on GitHub with a single command.

Article Content

AI assistants forgetting everything right after a conversation is a headache for every power user. A creator on Bilibili combined a "diary + semantic search" approach to restore the broken memory of an AI running on DeepSeek^Ad Harness—not by stuffing the entire conversation history into context, but by looking things up on demand. The solution is open-sourced on GitHub (MIT license) and can be installed with a single command.

Why "Writing the Rules into a File" Doesn't Work

The initial approach was straightforward: write the AI's persona, rules, and notes all into a single text file and read it in full every conversation. But as it grew, the file ballooned to thousands of characters, requiring a reload every time and consuming a staggering number of tokens.

What's more troublesome is the retrieval problem. The creator said bluntly in the video: "The real trouble isn't that it's long—it's that new entries can't find themselves." The longer the file, the harder it is for newly added content to be accurately matched, and the AI still "forgets" key information it just recorded.

The essence of the problem isn't memory capacity but the retrieval mechanism. Humans don't recall all memories simultaneously; they remember relevant fragments when needed—and AI should work the same way.

Layered Memory: Persona Stays in the Open, Dynamic Info Goes into the Diary

The new idea is to split memory into two layers. The first layer is a lean "persona" of a few hundred characters, always kept in context to keep the AI's basic behavior stable. The second layer is a "diary" recording what was done today, what pitfalls were hit, what was promised—this part doesn't enter context and is only retrieved when needed.

The benefits of this architecture are obvious: the context window isn't crowded out, yet historical information remains searchable. The key is that retrieval must "just ask for the meaning"—no need to remember the exact words, only to understand the semantics.

Hybrid Retrieval: Semantic and Keyword Paths in Parallel

The creator gave a telling example: when asked "the person who won the championship playing StarCraft," the query doesn't contain the three characters "Li Peinan" at all, yet the system still surfaced the relevant entry—because it compares meaning, not words.

Technically, this solution computes both vector scores and keyword scores, merging and ranking the results from both paths. Vector scores handle semantic similarity, while keyword scores catch exact matches of proper nouns. The creator's assessment is apt: "Using either alone isn't enough; together they hold steady."

The Complementarity of Vector and Keyword Retrieval

Vector retrieval maps text into dense vectors in a high-dimensional space and measures semantic closeness via cosine similarity, so it can understand the implicit association between "championship" and "Li Peinan." But an inherent weakness of dense vectors is insensitivity to exact matches of proper nouns, numbers, code identifiers, and the like, because the semantic information of such symbols is sparse. Keyword retrieval (such as BM25) scores based on term frequency and inverse document frequency, precisely hitting literal overlaps but unable to handle synonymous rewrites. Hybrid retrieval fuses the two scores with weighting (common approaches include weighted summation after score normalization, or Reciprocal Rank Fusion), preserving semantic generalization while catching exact-match scenarios—this is also the mainstream retrieval paradigm of current RAG systems.

Three Real-World Pitfall Records

In the video, the creator shared three key lessons from the implementation process, which are valuable references for developers wanting to reproduce the solution.

BGE Model Queries Must Include a Prefix

When using the BGE embedding model for retrieval, a specific prefix must be added on the query side. The consequence of omitting it is quite serious: a relevant sentence scored 0.44, while an irrelevant sentence scored 0.46—the ranking is completely reversed. This is determined by how the BGE model was trained; queries and documents need to use different encoding formats.

The Asymmetric Retrieval Design of the BGE Model

BGE (BAAI General Embedding) is a series of Chinese and English embedding models open-sourced by the Beijing Academy of Artificial Intelligence, performing prominently on retrieval benchmarks such as MTEB. It adopts asymmetric retrieval training: queries and passages use different encoding templates, and the query side needs a directive prefix like "Generate a representation for this sentence to retrieve relevant articles:" This is because the positive and negative sample pairs during training were constructed in this format, and the model learned to project queries under this prefix into a semantic region that better matches documents. If the query side omits the prefix, the encoding distribution becomes inconsistent with the training distribution, and similarity scoring becomes distorted, producing the inversion where relevant sentences score lower than irrelevant ones. Some later versions (such as bge-m3) simplified this, but classic versions like bge-large-zh still require compliance.

The 512 Token Input Limit Trap

BGE's input limit is 512 tokens, roughly one token per Chinese character. The creator split text at 700 characters, and it "collapsed" outright. Overly long input leads to truncation or errors, so the splitting granularity must stay within the model's limit.

Context Windows of Embedding Models and Splitting Strategies

Mainstream embedding models are based on BERT-like encoder architectures, whose positional encoding determines the maximum input length—the 512 token limit of classic BGE versions comes from this. After BPE tokenization, Chinese is roughly one character per token, so 512 tokens is approximately 512 Chinese characters; leaving some margin, splitting at around 500 characters is safer. Overly long text gets truncated (only the first 512 tokens are encoded) or throws an error, causing the latter half of the information to be lost. In practice, sliding window splitting with overlap is commonly used (e.g., chunk size 400, overlap 50) to avoid semantics being severed at split boundaries; for scenarios where global information must be preserved, one needs to switch to embedding models that support long contexts or adopt a strategy of chunking followed by aggregated summarization.

Small Models Can't Bridge Long-Distance Semantics

With a small embedding model, "search for can plant on night shift" and "search for how to arrange schedule" go off track—the semantic distance is too far, and the small model can't establish the association. Switching to a larger embedding model solved it. This shows that in semantic retrieval scenarios, the lower bound of model capacity is higher than one might imagine.

Who Should Use This Solution

This diary system is already open-sourced on GitHub under the MIT license, installs with a single command, and comes with instructions on how to paste the rules into your own agents file. It doesn't dictate "what to record and when to record it"—that's each Agent's own strategy. The tool is responsible for just one thing: making forgotten things retrievable again.

For long-running AI assistants, Agent applications that need to maintain context across sessions, and developers constrained by context window limits, this lightweight solution offers a pragmatic starting point. It's not universal memory but a controllable "external index"—persistent, searchable memory at minimal cost.

One-line Summary

An open-source solution uses layered memory and hybrid retrieval to help AI assistants recover memories across sessions.

Paragraph Summary

The article introduces an AI memory solution open-sourced by a Bilibili creator, solving the "forget after chatting" problem of an AI assistant on DeepSeek Harness. The core idea is to abandon stuffing all rules into a single file and instead use layered memory: a lean persona stays resident in context, while dynamic information is written to a diary and retrieved on demand. Retrieval uses a hybrid approach that merges and ranks vector scores and keyword scores, understanding semantic queries like "the person who won the championship playing StarCraft" while also precisely hitting proper nouns. The author also records three real-world pitfalls: BGE queries must include a prefix or the ranking inverts; the 512 token input limit requires controlling split granularity; and small embedding models can't bridge long-distance semantics, requiring a larger model. The solution is open-sourced under the MIT license and can be installed with a single command.