MCP Memory: A Lightweight AI Agent Memory Solution Built with SQLite

Why Agents Need "Memory"

As large language models (LLMs) evolve from single-turn Q&A to multi-turn, long-term agent applications, a core pain point has become increasingly apparent—models have no persistent memory. Once a conversation ends, the context vanishes. To enable AI agents to remember user preferences, interaction history, and key facts, the industry has developed various "memory layer" solutions, most of which rely on vector databases and expensive embedding computations.

LLMs are fundamentally stateless functions: given an input token sequence, they output the next token based on a probability distribution. A model's "memory" depends entirely on what's in the context window. While context windows have expanded from GPT-3's 4K tokens to Claude's 200K and even Gemini's million-token capacity, this remains "working memory" rather than "long-term memory." Once a session ends, this information is no longer available. More importantly, longer context windows mean higher inference costs (the attention mechanism's computational complexity scales quadratically with sequence length), making it uneconomical to stuff all historical information into the context. This is why agents need external memory systems—retrieving relevant information at the right moment and injecting it into the context, rather than carrying the entire history at all times.

Recently, an open-source project called MCP Memory caught attention on Hacker News. It takes a different approach, combining Google's OKF (Object Knowledge Format) with SQLite's FTS5 full-text search capabilities to provide a fast, lightweight agent memory solution that requires no heavy infrastructure.

MCP Memory Technical Architecture

MCP Protocol: A Standardized Context Interface

MCP (Model Context Protocol) is a recently spotlighted open standard designed to establish a unified communication interface between AI models and external tools or data sources. By packaging memory functionality as an MCP Server, any MCP-compatible client (such as Claude Desktop, various IDE plugins, or Agent frameworks) can plug in persistent memory capabilities without redundant development for each application.

MCP was proposed and open-sourced by Anthropic in late 2024, with design inspiration similar to how the Language Server Protocol (LSP) unified the IDE ecosystem. Before MCP, every AI application needed custom integration code to connect with external tools (file systems, databases, APIs, etc.). MCP defines a JSON-RPC communication standard with three core primitives: Tools, Resources, and Prompts. An MCP Server provides capabilities, while an MCP Client is the AI application consuming them. This decoupling means a memory service only needs to be implemented once to be callable by all MCP-compatible clients—from Claude Desktop to Cursor IDE to custom Agent frameworks. As of early 2025, the MCP ecosystem already has thousands of community-developed Servers covering scenarios from GitHub operations to database queries.

This is one of the project's most valuable design choices: rather than reinventing the wheel, it embraces an emerging industry standard, ensuring strong ecosystem compatibility.

Why SQLite FTS5 Instead of a Vector Database

Current mainstream agent memory solutions (such as Mem0, Zep, etc.) generally adopt vector embeddings + semantic retrieval. While this approach captures semantic similarity, it comes with notable costs:

• Computational overhead: Every write and query requires calling an embedding model, incurring API fees or local compute consumption
• Infrastructure complexity: Requires deploying and maintaining specialized vector databases (such as Pinecone, Qdrant, Chroma)
• Latency: The embedding computation and vector retrieval pipeline is relatively long

To illustrate with specific costs: OpenAI's text-embedding-3-small costs about $0.02 per million tokens, which seems cheap. But agents in continuous interaction need to call the embedding API for every memory write and every query. Assuming an active agent generates 100 memory writes and 500 retrievals per day, with an average of 200 tokens per operation, monthly embedding costs would be approximately $1.08. This is manageable for a single user, but scaling to 100,000 users brings monthly costs to $108,000. Additionally, there are hosting fees for the vector database itself: Pinecone's standard plan starts at $70/month, and Weaviate Cloud starts at $25/month. By comparison, the SQLite approach costs virtually nothing—one file, no network calls, no hosted services, everything completed locally.

MCP Memory goes against the grain by adopting SQLite's built-in FTS5 full-text search engine. FTS5 (Full-Text Search version 5) is SQLite's built-in full-text indexing extension module, available since SQLite 3.9.0. Under the hood, it uses an inverted index data structure: mapping each term in a document to a list of documents containing that term, enabling O(1)-level keyword lookup. FTS5 uses the BM25 ranking algorithm by default—a classic relevance scoring formula in information retrieval that considers term frequency (TF), inverse document frequency (IDF), and document length normalization. Compared to earlier FTS3/FTS4, FTS5 offers significant improvements in performance and API design, supporting custom tokenizers, column-level weighting, prefix queries, and phrase matching. In practical benchmarks, FTS5 retrieval latency for million-document collections is typically in the single-digit millisecond range.

Its core advantages are:

• Zero external dependencies: The entire memory store is a single SQLite file that can be distributed with the application
• Extremely fast retrieval: Keyword searches return locally in milliseconds
• No embedding costs: Completely independent of embedding models, eliminating API calls and compute overhead

The Role of Google OKF in Knowledge Organization

OKF (Object Knowledge Format) is a structured knowledge representation format proposed by Google, designed to organize unstructured information into entity-centric knowledge units. Each knowledge object contains a type identifier, attribute key-value pairs, and relationship links—similar to node representation in knowledge graphs but more lightweight.

In the context of MCP Memory, OKF's value lies in structuring a memory like "the user prefers Python" from free-form text into {type: 'preference', subject: 'user', attribute: 'programming_language', value: 'Python'}. This structuring not only improves retrieval precision (enabling exact attribute filtering) but also facilitates memory updates and deduplication—when user preferences change, the system can precisely locate and update the corresponding attribute rather than appending redundant text.

Compared to simply storing memories as chunks of free-form text, structured knowledge representation enables more precise retrieval and clearer factual relationships. This combination of "structured knowledge + full-text search" is what distinguishes MCP Memory from pure vector-based approaches.

Advantages and Trade-offs of the SQLite Memory Approach

The Practical Value of a Lightweight Solution

For many real-world application scenarios, developers don't need complex semantic vector retrieval. Much of agent memory is fundamentally factual and keyword-oriented—such as "what's the user's name," "the project discussed last time," or "preferred programming language." In these scenarios, FTS5 keyword-based retrieval is perfectly sufficient, with faster response times and virtually zero cost.

This pragmatic design philosophy aligns well with the developer community's widespread demand for "de-heavification" of tools: not every AI application needs a full vector database stack.

Inherent Limitations of Full-Text Search

Abandoning vector embeddings also means giving up semantic understanding. FTS5 relies on literal matching—when query terms differ from stored content but are semantically similar (e.g., searching "what car do they like" when the stored memory says "loves Tesla"), keyword retrieval may miss relevant results. This is an unavoidable shortcoming of pure full-text approaches.

From a technical perspective, BM25 (Best Matching 25) is a representative probabilistic information retrieval model that originated in the 1990s and remains a foundational ranking component of search engines today. It calculates relevance scores between queries and documents by analyzing term frequency and document frequency—essentially a "bag of words" model that only considers word occurrence frequency without understanding word order or semantics. By contrast, vector semantic retrieval uses neural networks to encode text as high-dimensional dense vectors (typically 256-1536 dimensions), computing semantic distance via cosine similarity or dot product. Semantic retrieval can understand that "automobile" and "car" are synonyms, while BM25 cannot. However, BM25 is more reliable for matching precise terms (such as product names or code variable names) and doesn't suffer from the "semantic drift" problem common in vector retrieval—returning semantically similar but actually irrelevant results.

The ideal approach may be hybrid search—combining FTS5's precise keyword matching with lightweight semantic retrieval to balance speed and recall. Hybrid search has become a best practice in production search systems, with common implementation methods including: Reciprocal Rank Fusion (RRF), which merges ranking lists from both retrieval methods via formula; weighted linear combination, which normalizes BM25 scores and vector similarity scores before applying weighted sums; or two-stage retrieval, which first uses keywords for fast candidate recall then applies a semantic model for re-ranking. In the SQLite ecosystem, extensions like sqlite-vec already support maintaining both FTS5 indexes and vector indexes in the same database. If MCP Memory introduces hybrid search in the future, it could use small local embedding models (such as all-MiniLM-L6-v2, only 80MB) for queries requiring semantic understanding while maintaining its lightweight architecture, achieving a balance between cost and effectiveness.

Implications for AI Agent Developers

The emergence of MCP Memory reflects a noteworthy trend in AI engineering: after the vector database hype, developers are beginning to reassess the value of traditional database technologies. Battle-tested embedded databases like SQLite, with their zero configuration, high performance, and excellent portability, are finding new relevance in the AI era.

For developers building AI agents, this project offers several takeaways:

1. Assess real needs first: Does your memory retrieval actually require semantic understanding? If most queries are keyword or fact-based, a lightweight solution may be superior
2. Embrace standard protocols: Packaging functionality as an MCP Server greatly improves reusability and ecosystem compatibility
3. Maintain cost awareness: Embedding model API costs become significant at scale—local full-text search is an economical alternative or complement

Conclusion

MCP Memory doesn't aim to replace all vector memory solutions. Instead, it provides a fast, light, and cost-effective option for specific scenarios. It demonstrates that in today's increasingly complex AI infrastructure landscape, returning to simple, mature engineering solutions can often yield unexpected efficiency gains. As the MCP ecosystem continues to expand, these "small but beautiful" tools that focus on a single capability and follow open standards may become essential building blocks for constructing reliable AI agents.