[KongchangAI]
· 2 min read· 1,352 words

Open-Source Memory Plugin Passes Three Rounds of External Testing: All 39 Tests Passed

Open-Source Memory Plugin Passes Three Rounds of External Testing: All 39 Tests Passed

Open-source memory plugin deepseekharness solves AI long-conversation forgetting with a three-tier structure, weight-based eviction, and persona locking.

deepseekharness is an open-source AI memory plugin that divides memory into stable, to-do, and archive zones, mimicking human layered note-taking. When the stable zone fills to 100 entries, it evicts by weight rather than hard-deleting, and core persona settings are permanently retained to prevent character drift. For reliability, corrupted files are not silently treated as empty, four processes can write concurrently without data loss, and three snapshots are retained for rollback. It injects 1,000 snapshots in 76 ms with 0.5 MB disk usage, has zero dependencies, and all numbers are reproducible via scripts. Three rounds of external testing covered 39 tests with over 20 accumulated issues, showing iteration from real feedback. Suitable for long-term companion AI and cross-session assistants.

Title

Open-Source Memory Plugin Passes Three Rounds of External Testing: All 39 Tests Passed

Meta Description

Open-source memory plugin deepseekharness uses a three-tier memory structure. When the stable zone fills to 100 entries, it evicts by weight rather than hard-deleting, and persona settings are permanently locked. It supports concurrent writes from four processes and rollback across three snapshots. Three rounds of external testing covered 39 tests: injecting 1,000 snapshots took 76 ms and 0.5 MB of disk, and every number can be reproduced with scripts. Zero dependencies; runs with a single command.

Article Content

The AI memory problem has long been a pain point in long-running conversations: the longer the chat, the more likely the model forgets early settings or loses key information outside the context window. Recently, an open-source memory plugin called deepseekharness sparked discussion on Bilibili. Its pitch is straightforward: split memory into three tiers, mimic the human cognitive habit of "long-term impressions, to-dos, and archived history," and submit to three consecutive rounds of external testing.

Three-Tier Structure: Layered Note-Taking Like a Human

According to the UP host, this plugin has only a three-tier memory structure. The stable zone stores long-term settings, such as persona and persistently valid background information; the to-do zone manages temporary tasks with deadlines; and the archive zone stores conversation summaries, compressing expired content into retrievable summaries.

Three-tier memory structure diagram

The benefit of this layering is clear separation of responsibilities: long-term information does not mix with temporary tasks, and archived content does not crowd out active context. For AI applications that need long-term companionship or sustained project progress, this is closer to real usage scenarios than cramming everything into a single vector database.

Context window is the upper limit on the number of tokens a large language model can process in a single inference, including input prompts and output generation. All conversation history, system settings, and retrieved memories must fit into this window; anything beyond it is truncated or cannot be processed. This is why models "forget" early content in long conversations—not because the model actively forgets, but because early information has slid out of the window. The core idea of tiered memory is to place only the currently most relevant active information into the context window, compress long-term settings into brief summaries, and archive expired content to external storage, thereby maintaining conversation coherence for as long as possible within a limited window.

Full Without Hard Deletion: Weight-Based Eviction and Persona Locking

Many memory solutions directly overwrite or delete the oldest data when capacity is reached, often at the cost of accidentally harming important settings. deepseekharness takes a different approach: when the stable zone fills to 100 entries, it does not hard-delete but evicts the oldest entry by weight.

Stable zone capacity and eviction strategy

More critically, the persona never moves—regardless of how weights are calculated, core persona entries are never evicted. This solves the long-standing problem of character-setting drift in long conversations. The UP host mentioned that three rounds of external testing covered 39 tests in total, with an accumulated issue list of more than 20 items, indicating that this mechanism was iterated from real feedback rather than designed in one go.

Weight-based eviction is a cache replacement strategy; common variants include LRU (least recently used), LFU (least frequently used), and importance-based weighted scoring. In a memory plugin, each memory can be assigned a weight score that considers factors such as last access time, reference frequency, and explicit importance markers. When capacity is full, the entry with the lowest weight is evicted rather than simply deleting the oldest data. Compared with hard deletion, weight-based eviction preserves "old but critical" information (such as user names and core preferences) and cleans up only low-value content. Persona locking adds a hard constraint on top of weight calculation: no matter how low the score, entries marked as core never participate in eviction, preventing character settings from gradually drifting in long conversations.

Data Safety: No Silent Failures and Concurrent Writes

The reliability of memory plugins is often overlooked. deepseekharness emphasizes two details: corrupted files are not silently treated as empty—that is, a read failure does not pretend there is no data; and four processes writing concurrently do not lose data, with three snapshots retained and rollback supported.

Data reliability and concurrent writes

For users integrating a memory plugin into automated workflows, these two points matter more than the number of features. Silent failures can make AI "lose memory" without any warning, and data loss from concurrent writes is almost inevitable in multi-task scenarios. Retaining three snapshots provides a fallback rollback path.

Silent failure means a program does not throw an exception or log an error when reading or writing data fails; instead, it returns an empty result or default value, leading the caller to mistakenly believe the operation succeeded. In memory plugin scenarios, silent failure is extremely dangerous: if reading a memory file fails and an empty list is returned, the AI behaves as if it never had any memory, while the user receives no error prompt and simply thinks the model "suddenly lost its memory." The correct approach is to explicitly throw an error or return a result marked with an error so that upper-layer logic can decide on a degradation strategy. Concurrent writes involve data consistency when multiple processes modify the same memory file simultaneously. Common solutions include file locks, atomic writes (write to a temporary file first, then rename), and copy-on-write; retaining multiple snapshots with rollback support is a fallback means of recovering data after concurrent conflicts or write corruption.

Performance Numbers: Reproducible to Be Credible

The performance figures given by the UP host are: injecting 1,000 snapshots takes 76 ms, and disk usage is 0.5 MB. More noteworthy is the second half—all numbers can be reproduced with scripts, with zero dependencies, and after cloning, it runs with a single command.

Performance and reproducibility

Performance claims for memory plugins are easy to inflate because test conditions vary widely. Open-sourcing the reproduction scripts alongside the code effectively hands verification power to the community, which is more persuasive than simply reporting a millisecond figure. Zero dependencies also lowers the barrier to entry—no need to set up a whole environment first.

Who It's For

If you are building a long-term companion AI, an assistant that needs to maintain settings across sessions, or you simply want your chatbot not to "forget things" after a few exchanges, this kind of tiered memory plugin is worth trying. Its design approach is not complicated, but it accounts for easily overlooked details such as eviction strategy, persona protection, and data reliability. The repository address is in the video description and comments, and parameters can be modified as needed.

It should be noted that the information in this article mainly comes from a single Bilibili UP host's introduction video. The specific content of the 39 tests and details about the external testers have not been elaborated in the article. For actual results, it is recommended to rely on your own cloned reproduction.

One-line Summary

Open-source memory plugin deepseekharness uses a three-tier structure, weight-based eviction, and persona locking to solve AI forgetting in long conversations.

Paragraph Summary

deepseekharness is an open-source AI memory plugin that divides memory into three tiers: stable zone, to-do zone, and archive zone, mimicking human layered note-taking habits. When the stable zone fills to 100 entries, it evicts the oldest entry by weight rather than hard-deleting, and core persona settings are permanently retained to prevent character drift. For data reliability, corrupted files are not silently treated as empty, four processes can write concurrently without data loss, and three snapshots are retained for rollback. In terms of performance, injecting 1,000 snapshots takes 76 ms and 0.5 MB of disk, with zero dependencies and all numbers reproducible via scripts. Three rounds of external testing covered 39 tests in total, with an accumulated issue list of more than 20 items, indicating the mechanism has been iterated through real feedback. It is suitable for long-term companion AI, assistants that maintain settings across sessions, and similar scenarios.

#open-source memory plugin#deepseekharness#AI memory#three-tier memory structure#weight-based eviction#persona locking#concurrent writes#snapshot rollback#long conversation memory#Bilibili
Share:

Related articles