Fully Local AI Agent Platform Deployment: What Each Component Requires

Running an AI agent platform entirely on local models with data never leaving your internal network is technically feasible — but far more complex than most people expect. A developer from the Vstorm team shared their hands-on experience on Reddit, based on their open-source platform AgenticOS (Apache-2.0 license), breaking down every component required to achieve "zero third-party involvement." Their core message is straightforward: simply slapping the "self-hosted" label on something doesn't guarantee data stays local. Every component is an independent decision.

What Components You Need for Full Localization

To build an agent system where data never leaves your network, you need to verify each of the following critical components one by one. A gap in any single area can silently send data to external services.

Conversational Model

The core chat model can be configured via an Ollama config file (no API key needed, pointing to your own hosted endpoint), or routed through a self-hosted LiteLLM proxy to a local model. vLLM and LM Studio connect via OpenAI-compatible endpoints. The author emphasizes a commonly misunderstood point: a configuration counts as "self-hosted" because its endpoint is local — not because it lacks an API key. In other words, having no key does not equal being local.

Embeddings and Document Parsing

Embeddings also require a self-hosted Ollama service, registered as a local service and selected per knowledge base collection. The documentation stresses that the embedding model is a "permanent choice" — chunked data and search queries are sent exclusively to the designated host, nowhere else.

For document parsing, the default is PyMuPDF, running alongside LiteParse and Tesseract OCR inside the worker. Note that LlamaParse is a cloud-based solution requiring an API key — if you want full localization, it must be disabled.

Embeddings play a critical role in RAG (Retrieval-Augmented Generation) architectures by converting text into vectors. When a user uploads a document or asks a question, the system calls the embedding model to transform text into high-dimensional vectors, which are stored in a vector database like pgvector. Retrieval then works by matching semantic similarity. This is why the embedding model is a "permanently binding" decision — once a knowledge base has been vectorized with a particular model, switching models means all historical data must be re-embedded, otherwise distance calculations between old and new vectors become meaningless. If the embedding service points to a cloud API, every document upload and query sends text content to an external party — often the most overlooked data leakage path.

Tracing and Tool Calls

Handling traces is simple: don't set a LOGFIRE_TOKEN, and don't configure any tracing token on agents or in the environment — run logs stay local.

Tools represent the biggest "data exfiltration risk." The author explicitly advises against connecting web search, web scraping, browser automation, mem0 memory, or any MCP servers — because every one of these features makes outbound calls. This means full localization comes at the cost of some capabilities, requiring a deliberate trade-off between privacy and functionality.

MCP (Model Context Protocol) is an open protocol proposed by Anthropic, designed to standardize how AI models interact with external tools and data sources — essentially a unified "plugin interface" for AI agents. MCP servers can expose capabilities like file systems, databases, and network services for models to call. The problem is that public MCP server registries are hosted externally, and each MCP server may make network requests to its own remote endpoints during execution. So even if your conversational model and embedding model are fully local, an agent connected to external MCP servers will still send data outside your network during tool calls. This is why the article recommends using --no-mcp to skip mirroring the registry.

Two Hidden Pitfalls of Local Models

When using local models, two easily overlooked but highly impactful issues deserve attention.

The first is budget controls breaking down. Platform costs are attributed to the API keys used during execution, and keyless local models don't log any spending. This creates a blind spot: conventional dollar-based budget limits have no effect on local models running out of control. The only real mechanism to stop a tool from entering an infinite loop is each agent's max_steps limit — the maximum number of model requests per run. The author repeatedly stresses: always set this value.

The second is that agent capability is entirely dependent on the model's tool-calling ability. Planning, document search, and sandbox operations all happen through tool calls. The author candidly admits that the team currently has no benchmark data for local models to share, and therefore won't claim any model "works great." This honesty actually highlights the biggest uncertainty in local deployments today.

The max_steps (maximum steps) limit is a fundamental safety mechanism in agent systems to prevent "runaway loops." In multi-step tool-calling scenarios, agents repeatedly invoke the model to plan the next action — if a tool call consistently returns errors or the model gets stuck in a flawed reasoning loop, an agent without a step limit will keep running and consuming resources. With paid APIs, the bill itself acts as a natural external constraint. But local models have near-zero marginal cost, so the price of runaway execution shifts to compute resources and time — and becomes completely invisible. In production, max_steps typically needs to be calibrated to the complexity of specific tasks: too low and complex tasks won't complete; too high and the protection becomes meaningless.

Deployment Environment and Current State

Setting up this system requires Docker Compose 2.24 or later, two published images (supporting amd64 and arm64 architectures), plus Postgres with pgvector, Redis, and Prefect — all launched from the same compose file.

The installer includes a --check flag for testing prerequisites only. Since the installer defaults to offering managed provider options, achieving full localization requires selecting "decide later" (--provider none) and then manually adding Ollama or vLLM configurations in the console. It also defaults to mirroring the public MCP server registry, so you'll need --no-mcp to skip that download.

The author is also candid about the project's limitations: it's still at version 0.0.x, supports single-host only, has no Kubernetes manifests, and lacks a reranker. There's also a HIPAA-oriented configuration that parses each model endpoint's hostname to determine whether it's local — so an address like https://ollama.vendor.example does not qualify as local. The author explicitly clarifies this is a configuration check, not a compliance certification.

Open Questions for the Community

The author closes with a question of genuine practical value: Which local models are you currently using that reliably handle multi-step tool calls? At what parameter scale, and on what hardware?

This question hits exactly on the core pain point of local agent deployments. For privacy-sensitive domains like healthcare, finance, and government, the need for fully local deployment is real. But the reliability of local models for multi-step tool calling remains an open question. The platform layer can ensure data doesn't leave your network — but an agent's ultimate performance is still bounded by the capabilities of the underlying model.

The project repository and documentation are open source, with full instructions for local deployment available in their data protection documentation. For teams looking to build truly "zero external dependency" agent systems, this component checklist offers a pragmatic starting point — it doesn't oversell outcomes, but instead puts every potential data leakage point squarely on the table.