A College Student Built a Cross-Model Persistent Memory System with Ollama: Decoupled Architecture + AES-256 Encryption

A College Student's Side Project: Giving Local LLMs Persistent Memory

Running large models locally is nothing new — Ollama has made it easy for anyone to run open-source models on their own machine. Ollama is an open-source local LLM runtime that wraps complex model inference environments into a clean, Docker-like CLI, allowing everyday users to run mainstream open-source models like Llama, Mistral, and Qwen locally without a deep machine learning background. With the maturation of the GGUF quantization format, an ordinary Mac with 16GB of RAM can now run 7B–13B parameter models smoothly, giving rise to a vibrant local AI app development ecosystem. But after using these tools seriously, a persistent pain point emerges: context doesn't carry over between models. New models drop every week, and we habitually jump from 8B to 32B, from one version to the next — and every time we switch, memory resets to zero.

A developer who identifies as a college student shared his solution on Reddit. Tired of doing everything in the terminal, he built a full-featured Mac application around Ollama. At its core is a "semi-persistent, self-healing" memory system, complete with an AES-256 encrypted vault and a "Jarvis-style" voice mode. According to him, the project totals approximately 39,848 lines of code.

Read-Only Adapter: Zero Interference With Your Models

The app's integration with Ollama is worth highlighting — it functions as a read-only adapter. It automatically detects models you've already installed, and can even identify model files sitting on disk when the Ollama daemon isn't running.

Critically, it never pulls, copies, or modifies anything — your models always remain untouched. Users can switch between models in real time mid-conversation using a switcher. For an out-of-the-box experience, the app bundles a qwen3 4b model, though the author emphasizes the real intent is to point it at your own model library.

The Memory System: Fully Decoupled from the Chat Model

This is the soul of the entire project, and the reason the author built it in the first place.

The conventional approach is to stuff the entire conversation history into the prompt — which wastes context window space and breaks down entirely when you switch models. Compared to a "long-context" solution that dumps all history directly into the context window, even models supporting 128K tokens experience noticeably slower inference and skyrocketing VRAM usage when the window fills up during local inference. This app takes a fundamentally different approach: memory is completely decoupled from the chat model.

Standalone Embedder + Local Vector Index

The app uses a separate, lightweight embedding model — all-MiniLM-L6-v2 — dedicated solely to converting "worth-keeping information" into vectors and storing them in a local index. This index is not bound to any chat model.

all-MiniLM-L6-v2 is a sentence embedding model developed by Microsoft and widely used on Hugging Face. It compresses text of arbitrary length into 384-dimensional dense vectors, weighs only around 80MB, and runs inference extremely fast — making it ideal for local devices. It's based on a distilled variant of BERT called MiniLM, which uses knowledge distillation to dramatically reduce computational overhead while retaining strong semantic understanding.

When you submit a query, the system performs a semantic search over the index rather than dumping the entire history into the prompt. The core principle of semantic search involves converting both the user's query and stored memory entries into vectors, then finding the most semantically similar content via cosine similarity — even if the wording differs. The system also maintains a running profile about you that gets passed to whichever model is currently loaded.

This design is essentially applying the RAG (Retrieval-Augmented Generation) paradigm to conversational memory management. The classic RAG pipeline vectorizes documents into a database and retrieves the most relevant fragments at query time to splice into the prompt — fundamentally breaking through the physical limits of the context window. This means when you swap your 8B for a 32B model, memory carries over naturally — because the index never moved. The design elegantly separates the "memory" layer from the "inference" layer, fundamentally solving the context-loss problem that plagues model switching.

Self-Healing: Keeping Memory From "Rotting"

The author specifically notes that this memory system actively cleans itself to prevent rot over time. Philosophically, this borrows from cognitive science models of memory: human memory is not a static archive but a dynamic reconstruction process. The Ebbinghaus forgetting curve describes how information decays exponentially without reinforcement, while new information actively overwrites or revises the old. On the engineering side, handling "facts that change over time" (e.g., "the user used to live in Beijing, now lives in Shanghai") is a core challenge in knowledge graph design — simple overwriting loses historical context, while ignoring contradictions produces inconsistent responses.

The specific mechanisms in this system include:

• Approximate deduplication: Similar memory entries are merged;
• Contradiction handling: When a new fact conflicts with an old one, the system keeps the new fact but archives the old one along with its history, rather than simply overwriting it;
• Decay mechanism: Information that hasn't been accessed for a long time automatically decays and gets archived, simulating the forgetting curve;
• No hard deletes: Everything can be browsed, pruned, or recalled.

This "repair and heal" design philosophy makes local memory behave more like human memory's dynamic nature — preserving historical context without being dragged down by stale information.

Privacy and Interaction: Encrypted Vault and Local Voice Mode

AES-256 Encrypted Vault

For sensitive data like API keys, passwords, and private notes, the app provides an AES-256 encrypted vault. AES-256 is a symmetric encryption standard established by the National Institute of Standards and Technology (NIST). Its 256-bit key length is considered unbreakable by brute force for the foreseeable future, and it's widely used in military, financial, and enterprise data protection.

Introducing this design into a local AI context addresses a subtle but real security concern: a locally running LLM is itself a "black box." Even if data never leaves your machine, sensitive information exposed directly in the model's context window could theoretically be induced to leak through carefully crafted prompts. The key point is that the model itself cannot read the vault's contents — it can only be unlocked with the user's passphrase. Keeping sensitive information outside the model's accessible scope is an effective isolation measure against "prompt injection"-style attacks.

Jarvis-Style Hands-Free Voice Mode

The author primarily uses the app as a coding assistant. The built-in "Jarvis-style" voice mode enables hands-free conversation while you work, with responses delivered sentence by sentence via speech. When you need quiet, you can switch directly to text input.

On the privacy front, the microphone only activates when the window is open, and no data is streamed externally. For privacy-conscious users, the promise that "data never leaves your machine" is a significant selling point.

Current Status and Future Plans

The app currently supports Mac (Apple Silicon) only. The author acknowledges that a large portion of users are on Windows and is actively working on a Windows version. He even called on Windows users to comment on his post to help prioritize development.

Technical Choices and Product Thinking

From a technical standpoint, this project doesn't rely on anything exotic — all-MiniLM-L6-v2 is a mature, lightweight embedding model, and semantic retrieval with a vector index is a classic RAG combination. What's truly worth noting is the completeness of the productization and the precision with which the problem is defined:

• Decoupling "memory" from "model" directly targets the core pain point of local multi-model switching;
• The memory self-healing mechanism reflects deep thinking about long-term user experience;
• The read-only adapter and local-first privacy design respect what local AI users value most: a sense of control.

For a solo project approaching 40,000 lines of code, this represents genuinely solid engineering practice. It also reinforces a growing trend: the value of the local AI ecosystem is increasingly found in the "surrounding capabilities" built around models — memory, privacy, interaction — rather than in the models themselves. As rapid model iteration becomes the norm, a stable "foundation" that lets memory persist seamlessly may be exactly what everyday users actually need.