Developing a Local AI Assistant with Agents: Achieving Token Freedom and Privacy-First Deployment

Local agent deployment is emerging as a new trend in AI applications. A Bilibili creator shared his full journey of using agents to develop agent software called "Mowen AI." The software mimics the interaction logic of mainstream products like Doubao and WorkBuddy, and adds features unique to local deployment, such as VRAM management, model switching, and offline conversation. More importantly, the creator reveals how to use existing agent tools (like DeepSeek and WorkBuddy) in a "boss-employee" model to develop software—even without knowing how to code, you can bring ideas to life. This article takes a deep dive into Mowen AI's core features, development insights, and the advantages and limitations of locally deployed agents.

Mowen AI: Feature Highlights of a Local Agent

Mowen AI's interface resembles Doubao and WorkBuddy^Ad, lowering the learning curve for users. It supports multi-conversation management and project categorization. Project mode provides better memory retention, preventing context loss in long conversations. The creator particularly emphasized the archive feature: users can hide completed conversations and restore them when needed—archived conversations return to their original position and won't get mixed up with projects.

Model management is at the core of local deployment. Mowen AI comes with multiple built-in models, each labeled with VRAM requirements and recommended use cases. The models the creator configured mainly target 16GB and 24GB VRAM. Among them, the DeepSeek^Ad R1 model is smaller and runs smoothly but doesn't support vision; the Qwen 3.8 9B and 27B series have vision capabilities but consume more VRAM. Users can flexibly choose based on their hardware conditions, or even import their own model files.

The software also provides VRAM release and reload functions, making it easy for users to quickly free up resources when running other GPU tasks. On the voice side, Mowen AI integrates auto-read-aloud and voice input, and supports voice cloning—users simply record 10 seconds of audio and attach a text transcript to add a custom voice. The creator compared four TTS models and recommended the second one for its balance between naturalness and resource consumption.

Quantized models: "Quantization" in local deployment refers to reducing model size and VRAM usage by lowering the numerical precision of model weights (e.g., compressing from 16-bit floating point to 8-bit or 4-bit integers) while maintaining performance as much as possible. Common quantization formats include GGUF, GPTQ, AWQ, etc., with different quantization levels (like Q4, Q8) trading off between precision and resource consumption. Most models labeled in Mowen AI are quantized versions, which is why they can run on 16GB or 24GB VRAM. Users importing custom models should also pay attention to compatibility between quantization format and inference framework.

VRAM management: VRAM (Video RAM) is the high-speed memory on a GPU used to store model weights, intermediate activation values, and KV cache. When running large models locally, insufficient VRAM can cause inference speed to plummet or even crash. The VRAM release function frees up space by offloading model weights or clearing cache, while reloading restores the model to VRAM. Proper VRAM management allows users to flexibly switch when running other GPU tasks (like gaming or rendering), but frequent loading and unloading increases latency—it's advisable to weigh based on task duration.

Development Insights: Building Software with Agents—Ideas Matter More Than Code

The creator described the development process as a "boss and employee" collaboration: the user is the boss, the agent is the programmer employee, and you just need to clearly express requirements for the agent to execute. He used DeepSeek and WorkBuddy for development. The first version was completed quickly, but subsequent testing and optimization took up most of the time. During the process, the agent sometimes couldn't understand complex descriptions. The creator suggests switching to clearer instructions or directly providing design mockups (like PS files) for the AI to implement.

For long-term projects, the creator strongly recommends using "project" mode rather than regular conversations, because projects better maintain memory and avoid information loss from context compression. He mentioned that when developing the background execution feature, WorkBuddy failed to solve the problem multiple times, and switching to DeepSeek finally completed it. This reflects capability differences between models on complex tasks.

Project mode and context memory: Regular conversation mode compresses or loses early information in long conversations due to context window limitations, causing the agent to "forget" requirements. Project mode maintains memory consistency by independently storing project-related conversation history, files, and instructions, and prioritizing retrieval of project context in each interaction. For long-cycle tasks like software development, project mode avoids repeating requirements and reduces the risk of the agent drifting from goals. However, project mode still requires reasonable information organization to avoid context exceeding the window.

Advantages and Limitations of Locally Deployed Agents

The biggest appeal of local deployment is "token freedom"—no need to consume credits or pay, and the ability to run uncensored models with unrestricted responses. Privacy is also a key advantage: all data stays local, avoiding cloud leak risks. However, the creator candidly admits that local model capabilities lag noticeably behind online models, and complex tasks (like software development) are still better suited to online agents. Local deployment is more suitable for simple tasks, privacy-sensitive scenarios, or experimental projects.

Additionally, local deployment has high hardware requirements—insufficient VRAM can cause slow operation or even freezing. The creator suggests choosing models based on VRAM size and making good use of the VRAM release function. Speech recognition models may miss punctuation when someone speaks quickly, but overall recognition quality is acceptable.

Uncensored models: Refers to model versions that haven't undergone content safety alignment or filtering, typically fine-tuned by the community on base models or released with original weights directly. Such models' responses aren't restricted by platform policies, but may generate harmful or inaccurate content—users bear the risk themselves. Local deployment allows running such models, which is another appeal beyond "token freedom," but also requires users to have discernment and filtering capabilities. Online services generally have built-in censorship mechanisms due to compliance requirements.

Conclusion: The Present and Future of Local Agents

Mowen AI demonstrates the possibilities of local agents: with existing agent tools, ordinary people can develop fully functional AI applications. Although local models haven't yet reached the intelligence level of online models, their advantages in privacy, cost, and freedom cannot be overlooked. As hardware performance improves and models are optimized, local deployment is expected to become the choice of more people. For developers who want to try, the creator's advice is: clarify requirements, iterate patiently, and make good use of project mode to manage long-cycle tasks.

One-line Summary

A Bilibili creator developed local AI software Mowen AI using agents, sharing features, development insights, and the pros and cons of local deployment.

Paragraph Summary

The article describes how a Bilibili creator used agent tools like DeepSeek and WorkBuddy in a "boss-employee" model to develop locally deployed agent software "Mowen AI." Mowen AI mimics mainstream products like Doubao and features multi-conversation management, project mode, archiving, model switching, VRAM management, TTS, and voice cloning. Development insights emphasize clarifying requirements, providing design mockups, and using project mode to maintain memory, while noting capability differences between models. Local deployment's advantages include token freedom, the ability to run uncensored models, and privacy protection, but model capabilities are weaker than online versions—suitable for simple tasks or privacy-sensitive scenarios. The conclusion looks ahead to the future of local agents.