Can Local LLMs Really Run Offline? A Deep Dive into Bees' Network Requests

Local LLMs are often marketed as "your data never leaves your machine"—but how true is that promise? After the Bees team released a promotional video titled "Nothing leaves your computer," one developer decided to go through the Bees source code line by line, cataloging every network request the app makes at runtime to verify whether that claim holds up.

According to the bees.bot blog, when users run models locally, prompts, files, Agent work products, and memories do stay on the machine—the model runs in llama-server, listening only on 127.0.0.1, which is unreachable from outside. Bees also has no built-in crash reporting, analytics, or tracking code.

But the author readily admits: by default, four categories of requests are still sent out. One of them surprised even him. He chose to disclose these details openly rather than let users discover them on their own in a packet capture.

The Four Categories of Default Requests

This is the article's core finding. Even if a user never signs in, Bees still sends the following requests to external servers on startup:

Config query: On every launch, Bees hits app.bees.bot/api/config to ask about sign-in configuration. The request itself carries no account info or token—just the usual IP address and user agent. This behavior can't be disabled in-app; the only way to change the target address is by setting the BEES_ACCOUNT_API_URL environment variable.

Update check: On startup and every 6 hours thereafter, Bees pulls bees-update.json from GitHub. The request carries no user data and won't auto-download anything—unless the user clicks "Update and restart." There's currently no off switch.

FreeLLMAPI model list sync: This is the one the author didn't anticipate. Bees ships with a small built-in server to support the free cloud AI option, and it runs on every startup—even if the user only uses local models. Ten seconds after launch, and then every 12 hours, it hits api.freellmapi.co to fetch the latest free model list. The request only sends the version number of the existing list, no user data. The only way to turn it off right now is setting CATALOG_SYNC_DISABLED=1, and the team says they'll fix it so the service only starts "when the free AI option is selected."

First-time memory initialization: Local memory is enabled by default. On first launch, Bees downloads Python 3.12 (from GitHub), memory-related packages (from PyPI), and an embedding model all-MiniLM-L6-v2 (from Hugging Face) via uv. The code has an explicit comment: "Never forward memories to a hosted model." User memory content doesn't appear in any of these requests.

One detail: the official help page currently only mentions the sign-in check and model downloads, omitting the FreeLLMAPI list sync and memory initialization. The team has promised to fix that page.

What is 127.0.0.1 (the loopback address)?

The article repeatedly mentions that the model listens only on 127.0.0.1, which is the technical foundation for understanding the "data doesn't leave your machine" promise. 127.0.0.1 is the loopback address in IPv4, commonly known as localhost, used to refer to the machine itself. A service bound to this address only accepts connections from the same machine—other devices on the LAN, routers, and the public internet can't reach it. By contrast, if a service listens on 0.0.0.0, it accepts connections from all network interfaces, meaning any device on the same network could connect. So when llama-server is bound only to 127.0.0.1, the prompts, context, and outputs used for model inference genuinely don't leak out through the network interface. That said, this only guarantees that the "model service itself" isn't exposed externally; other modules in the app can still proactively initiate outbound requests—which is exactly the focus of the four default request categories discussed later. The loopback address is a fact constraint at the network layer, while "whether the app connects out" is an application-layer behavioral question. The two shouldn't be conflated.

The Relationship Between Embedding Models and Local Memory

The article mentions that first-time memory initialization downloads all-MiniLM-L6-v2, which is an embedding model. An embedding model's job is to convert text into a fixed-dimensional vector of numbers, so that semantically similar sentences end up closer together in vector space. The local memory feature relies on it for two things: first, encoding the user's historical content into vectors stored in a local database; second, encoding new queries into vectors and retrieving relevant old memories via similarity search. all-MiniLM-L6-v2 is a very small open-source sentence embedding model, about 80MB, commonly chosen as the default by local apps because it runs fast and takes up little resources on consumer hardware. Since the embedding process runs entirely on the local machine, neither the original memory text nor the vectors leave the device—which is exactly what the code comment "Never forward memories to a hosted model" refers to. One distinction to note: downloading the model weights is a one-time network action, while using the model for encoding is pure local computation. The two are fundamentally different.

On-Demand Requests: Networking Only When There's an Action

Unlike the default requests, the following network behaviors require the user or Agent to actively trigger them, and each one only sends the data necessary to perform that operation.

Model download: When enabling the Qwen3 4B model, Bees downloads about 2.5 GB from Hugging Face and verifies it with a fixed checksum. Users can also paste their own GGUF link.

Web search and reading: Every Agent has built-in search and page-reading tools, with no toggle. Search sends queries simultaneously to search.brave.com, html.duckduckgo.com, lite.duckduckgo.com, and bing.com; news search goes through Google News RSS. Only the search keywords leave the machine, and only when the Agent decides to search.

Ask Bees planning: When requesting a plan, Bees may query the MCP public registry registry.modelcontextprotocol.io and read skill packs from GitHub. The Skills page uses the same calls.

Apps page: Pulls the app list from bees-bot.github.io when opened.

Cloud models and team sync: When choosing an API key provider, prompts go to the corresponding service; after signing into a Bees Cloud team, it syncs every 15 seconds. On his own Mac, the author observed about 20 long-lived connections to app.bees.bot.

What is the GGUF format?

The article mentions a model download of about 2.5GB and the ability to "paste your own GGUF link." GGUF (GPT-Generated Unified Format) is currently the most mainstream model file format in the local LLM ecosystem, designed and promoted by the llama.cpp project. It packages model weights, tokenizer configuration, and metadata (such as architecture, quantization type, and context length) into a single file, making distribution and loading easy. GGUF's core advantage is its support for multiple quantization schemes—by reducing weight precision (say, from 16-bit down to 4-bit), it dramatically compresses file size, letting models that would otherwise need tens of GB of VRAM run on an ordinary laptop. That's why a 4B-parameter model might only be about 2.5GB after quantization. llama-server is the inference service provided by llama.cpp, and it's also the underlying engine Bees uses when running models locally. Understanding GGUF helps explain why users can swap out model files on their own: as long as the format is compatible, the same local inference stack can load models from different sources and of different sizes, with no reliance on the cloud.

Items Confirmed "Off" in the Code Review

The author didn't just list "what's sent out"—he also checked which suspicious-looking features are actually off:

The Agent runtime has telemetry built in, but Bees passes DSH_TELEMETRY_DISABLED=1 on startup, and the author traced the code to confirm the setting takes effect. Even if telemetry were on, it only uploads data after the user sends feedback.

The application window's Content Security Policy (CSP) only allows itself and 127.0.0.1, so no remote fonts, scripts, or images can load. Help links open in the local browser when clicked.

The memory service has tracing code, but it only exports if an endpoint is set—and Bees doesn't set one. The device ID file is used only in team sign-in scenarios.

What Hasn't Been Verified Yet

The author honestly draws the boundaries: this list comes entirely from reading source code, and hasn't been verified via packet capture on a fresh installation. The Bees on his Mac is already signed into a team, so it can't represent the clean state of a first run.

There are also two questions he couldn't resolve from the code: the memory reranker points to Hugging Face, but he couldn't find where its downloads are stored; and a library in the memory service might pull a price table from GitHub when loading, though the path Bees uses theoretically shouldn't trigger it. The author says the next step is a clean install, running only local models, monitoring network traffic for the first 15 minutes, and publishing the results.

Three Fixes Promised by the Team

The author is also on the project side, and he's given a clear list of improvements: FreeLLMAPI list sync shouldn't run when the user has never selected a free cloud model; the update check should have a toggle; and the help page needs to add documentation about memory initialization.

Bees is free and open-source software, supporting Mac, Windows, and Linux, with code hosted on GitHub. The author encourages users to file issues when they find missed requests—this "self-audit before disclosure" approach itself offers a reference sample for privacy transparency in local AI tools.

For local LLM users concerned about data privacy, the value of this article is that it doesn't broadly promise "everything is local." Instead, it lays out in clear categories: what's sent by default, what's sent on demand, what's confirmed off, and what hasn't been checked yet. That level of granularity is more trustworthy than a tagline.

One-line Summary

The developer audited the Bees source code, verifying the boundaries and exceptions of the local LLM's "nothing leaves your computer" promise.

Paragraph Summary

A developer read through the Bees source code line by line to verify whether its "Nothing leaves your computer" claim holds up. The conclusion: when running locally, prompts, files, Agent content, and memories do stay on the machine, the model listens only on the loopback address, and there's no built-in telemetry. But four categories of outbound requests are still sent by default: config queries, update checks, FreeLLMAPI model list sync, and first-time memory initialization downloads—with FreeLLMAPI sync triggering even if the user never uses the free cloud model. The article also distinguishes on-demand requests (model downloads, web search, cloud model sync, etc.) from features confirmed off (telemetry, CSP remote loading, etc.), and honestly admits it hasn't been verified via packet capture on a fresh environment. The team has promised three fixes.