ContextWeaver: Building Local Code Intelligence Retrieval for LLMs with MCP + Tree-sitter

Project Overview: Helping LLMs Truly Understand Your Codebase

Using large language models to write code is nothing new, but enabling an LLM to truly "understand" your local codebase remains a core pain point for developers. Full-text search produces too much noise, manually copying and pasting code snippets is inefficient, and cramming an entire project into a context window is simply unrealistic.

ContextWeaver is an open-source tool built specifically to solve this problem. Based on the MCP (Model Context Protocol), it combines Tree-sitter syntax parsing with vector search technology to provide LLMs with precise local codebase context retrieval and intelligent weaving capabilities.

The project is developed in TypeScript and has already garnered 268 Stars on GitHub, with community interest rapidly growing.

Core Technical Architecture Breakdown

ContextWeaver's tech stack consists of three key components, each with its own responsibility yet tightly integrated.

MCP Protocol: A Standardized Context Communication Layer

MCP (Model Context Protocol) is an open protocol officially open-sourced by Anthropic in November 2024, designed to standardize communication between LLMs and external data sources and tools. Its design philosophy draws heavily from a successful precedent in software engineering — LSP (Language Server Protocol). Proposed by Microsoft, LSP standardized communication between editors and language tools, completely solving the ecosystem fragmentation problem of "every editor needing separate plugins for every language." MCP aims to replicate this success in the AI tools space: using a single protocol to connect all AI clients with external data sources, avoiding redundant work. Currently, major AI development tools including Claude Desktop, Cursor, and Zed have all announced MCP support, and the protocol ecosystem is rapidly taking shape.

By choosing MCP as its underlying protocol, ContextWeaver gains a direct benefit — it can seamlessly integrate with all AI clients that support MCP, without being tied to any specific IDE or model.

The core advantage of this architectural design lies in decoupling. The code indexing and retrieval logic runs as an independent MCP Server, and any MCP-compatible client can call it directly, offering strong versatility and extensibility.

Tree-sitter Syntax Parsing: From Text Splitting to Structure Awareness

Unlike simple text splitting, ContextWeaver uses Tree-sitter to perform syntax-level parsing of code. Tree-sitter was originally developed by GitHub engineer Max Brunsfeld and has since become the core syntax highlighting engine for modern editors like Neovim and Helix, while also being used by GitHub itself for code search and navigation. It is an incremental parsing library capable of building precise Abstract Syntax Trees (ASTs) for source code — the core data structure of compiler front-ends that transforms source code into a tree-based semantic representation, clearly describing the type, hierarchy, and relationships of every element in the code. Tree-sitter's "incremental" nature means that after code modifications, there's no need to re-parse the entire file — only the subtrees corresponding to the changed portions need updating. This delivers extremely high performance and is the key reason it can run in real-time within editors. Tree-sitter currently supports dozens of mainstream programming languages including Python, JavaScript, Go, and Rust.

This means that when ContextWeaver indexes code, it doesn't mechanically split by line count or character count. Instead, it can identify semantic units such as function definitions, class declarations, and module imports. This structure-aware splitting approach delivers two key advantages:

• Context completeness: Code snippets provided to the LLM are semantically complete, avoiding the awkward situation of truncated functions
• Retrieval precision: Syntax structure-based indexing more accurately matches developers' query intent

Vector Search: Semantic-Level Code Matching

After Tree-sitter completes the structural segmentation of code, ContextWeaver vectorizes (embeds) these code snippets and stores them in a vector database. The essence of vectorization is using an embedding model to convert text into high-dimensional numerical vectors, so that semantically similar content is closer together in vector space. The system measures semantic distance using algorithms like cosine similarity, enabling search that "understands meaning" rather than "matches characters." This technology is a core component of RAG (Retrieval-Augmented Generation) architecture — proposed by Meta AI in 2020, RAG dynamically retrieves external knowledge bases during LLM inference to compensate for limitations in model parametric knowledge and timeliness. It has become the mainstream architectural paradigm for enterprise AI applications. ContextWeaver adapts the RAG approach specifically for code scenarios, and significantly improves retrieval quality through Tree-sitter's structured segmentation.

Compared to traditional keyword search, the advantage of vector search lies in understanding the semantics behind queries. For example: when you search for "user authentication logic," even if the code doesn't contain the word "authentication" but includes related functions like verifyToken and checkPermission, vector search can still match them accurately.

ContextWeaver Workflow in Detail

ContextWeaver's complete workflow is divided into three phases:

1. Indexing phase: Scans the local codebase, uses Tree-sitter to parse code structure, vectorizes semantically complete code blocks and stores them in the index
2. Retrieval phase: Receives query requests from MCP clients, finds the most relevant code snippets through vector search
3. Weaving phase: Organizes retrieved code snippets into structured context and returns them to the LLM via the MCP protocol

The concept of "weaving" is precisely where the project name ContextWeaver comes from. It doesn't simply return a bunch of scattered code snippets; instead, it intelligently organizes related code dispersed across different files, providing the LLM with coherent, complete contextual information. This approach significantly improves the quality of LLM-generated code and answers.

Typical Use Cases and Practical Value

AI-Assisted Development for Large Codebases

For large projects with hundreds of thousands of lines of code, an LLM's context window is far from sufficient to accommodate all the code. Even models with the largest current context windows (such as Gemini 1.5 Pro with its million-token support) still fall short when facing real enterprise codebases, and ultra-long contexts bring significant inference latency and cost increases. ContextWeaver, through intelligent retrieval, provides only the code snippets most relevant to the current task, giving the LLM maximum information density within a limited context window.

Code Review and Refactoring Assistance

During code review or refactoring, developers need to quickly understand a module's dependency relationships and call chains. ContextWeaver's syntax-aware capability enables precise location of relevant code, helping the LLM provide more targeted refactoring suggestions.

Rapid Onboarding for New Team Members

Developers who are new to a team can ask the LLM questions in natural language, and ContextWeaver will automatically retrieve relevant implementations from the codebase. This helps new members quickly understand project architecture and business logic, dramatically shortening the onboarding period.

Technical Ecosystem Positioning: How It Differs from Copilot and RAG

In the current landscape of AI programming tools, ContextWeaver occupies a unique position:

Compared to cloud-based solutions like GitHub Copilot, ContextWeaver runs entirely locally — code never needs to be uploaded to third-party servers. For enterprises that prioritize code security and data privacy, this is crucial — especially in industries like finance, healthcare, and defense where data compliance requirements are stringent and local deployment is often a hard prerequisite.

Compared to general-purpose RAG frameworks, ContextWeaver is deeply optimized specifically for code scenarios. General RAG solutions (such as LlamaIndex or LangChain's document retrieval pipelines) typically can only mechanically split code by fixed character counts or paragraphs, easily breaking the integrity of semantic units like functions and classes. The introduction of Tree-sitter gives ContextWeaver a clear advantage in code comprehension precision — it achieves true function-level precise segmentation with semantic preservation, something general RAG solutions struggle to match.

As the MCP protocol ecosystem continues to grow, tools like ContextWeaver built on standardized protocols will have increasingly broad application space. It represents an important direction in AI-assisted programming: giving LLMs structured, semantic code awareness capabilities, rather than blindly guessing at code logic.

Summary: AI Programming Infrastructure Worth Watching

ContextWeaver provides a practical solution for AI-assisted development of local codebases through its technical combination of MCP protocol + Tree-sitter + vector search. Its open-source nature and standardized protocol design give it excellent extensibility and a solid foundation for community collaboration.

If you're exploring AI programming workflows, or frustrated by LLMs' inability to effectively understand large codebases, ContextWeaver is a tool worth trying.

Key Takeaways

• ContextWeaver is built on the MCP protocol, seamlessly integrating with MCP-supporting AI clients like Claude Desktop and Cursor
• Uses Tree-sitter for syntax-level code parsing, ensuring semantically complete code snippets are provided to the LLM
• Implements semantic-level code retrieval through vector search, more precise than traditional keyword search
• Runs entirely locally with no need to upload code to third-party servers, suitable for security-conscious enterprise scenarios
• Developed in TypeScript with 268 Stars, representing the structured context direction in AI-assisted programming tools