DeepSeek Orb Is Now Open Source: Turning Harness Into a Floating Ball That Takes Over Your Computer

From Official Harness to a Floating Ball: A Desktop-Focused Mod

DeepSeek^Ad V4.1 brings two key changes: native dynamic visual capabilities that allow the model to understand and process real-time screen images, and extremely fast token generation — around 300 tokens/sec in some scenarios. Together, these two improvements fundamentally change the viability of AI-controlled computer operation. When generation speed is fast enough, having the model take over the mouse and keyboard to click buttons, copy and paste, and so on can actually be faster than doing it by hand.

Building on this insight, a Bilibili creator took the latest officially open-sourced DeepSeek Harness and modded it into an agent product called DeepSeek Orb. The demo at the start of the video — having the model paint its own portrait with a brush tool, handing it an online quiz to complete, even making it play a mini-game — all ran on DeepSeek Orb.

DeepSeek Harness is the official open-source agent framework from DeepSeek. It provides a standardized tool-use interface that allows the model to invoke external capabilities — such as taking screenshots, executing code, and manipulating the file system — in a structured way. The core design philosophy of Harness is to decouple "model reasoning" from "tool execution": the model outputs structured action instructions, while Harness handles the actual execution and feeds the results back to the model, forming an observe-decide-act loop. For developers, Harness acts as an extensible scaffold — you can quickly build capable agent applications simply by plugging in new tools or modifying the interaction layer. DeepSeek Orb adopts this same core loop and layers a floating-ball UI and dual-track scheduling logic on top of it.

Main Window + Floating Ball: Two Interfaces, Two Roles

DeepSeek Orb is structurally divided into two parts: the main window and the floating ball window.

The main window is essentially DeepSeek Harness itself — functionally identical to the official version, because it uses the official implementation directly. The genuinely new piece is the floating ball window, which is a "computer control" mode within Harness that shares the same screen and the same mouse and keyboard as the user.

The creator demonstrated a typical workflow: summon the floating ball, issue a natural language command, have it search for a specific phone on JD.com, capture the product info in a screenshot, and then send it to a contact via WeChat for confirmation. The model handled the search, screenshot, and cross-app messaging entirely on its own — the user only needed to describe the task in a single sentence.

Dual-Track Agent Architecture: Keep the Foreground Free from Long Tasks

The most noteworthy design in DeepSeek Orb is its dual-track agent architecture, consisting of a floating ball agent and a background agent.

This design addresses a real pain point: if a complex task requires prolonged use of the mouse and keyboard, the user can't do anything else in the meantime. To solve this, when the floating ball receives a complex task that requires a long execution chain — such as writing code, building a presentation, or conducting product research — the floating ball agent hands the task off to a standard-mode agent running in the background.

This way, the foreground agent is never blocked by long-running tasks. The user can monitor the background task's progress while still issuing new instructions to the floating ball at any time. The creator mentioned that the background agent typically runs with a model like Zhipu GLM-4.5 Flash.

Live Test: Writing a Mini-Game While Watching a TV Show

In the demo, the creator asked the background agent to build a simple web-based mini-game similar to "Sheep-a-Sheep," with the task running in the background inside the main window. While waiting, he issued a new instruction through the floating ball, asking the foreground to open the first episode of a TV series on Tencent Video — the two tasks ran in parallel without interfering with each other.

Once the background task finished, the floating ball immediately received a completion notification from the background agent and opened the result. The mini-game was playable right away, and functionality was confirmed. This "background working, foreground on standby" collaboration is the dual-track architecture's value proposition in action.

Zhipu GLM-4.5 Flash is a lightweight large language model from Zhipu AI, designed for low latency and low cost. It's well-suited to serve as the "execution layer" in agent tasks — handling large volumes of repetitive inference and tool calls once the task objective is clear, without requiring complex multi-turn dialogue or creative generation. In a dual-track architecture, the background agent typically faces a sequence of clearly decomposed subtasks, where the key requirements are stability, speed, and cost-efficiency — exactly where Flash-series models excel. By contrast, the foreground floating ball agent needs to interpret ambiguous natural language from the user and decompose tasks accordingly, which demands stronger semantic understanding and is better paired with a more capable flagship model. The two complement each other, striking a balance between performance and cost.

Open Source and How to Get It

The project, named DeepSeek Harness Orb, is now open-sourced on GitHub and supports both Windows and Mac. Users can download the installer directly from GitHub or find the link in the video description.

From a product design perspective, DeepSeek Orb is a textbook example of "model capability upgrades driving interaction innovation": it is precisely V4.1's native vision and high-throughput speed that transformed GUI-level automation from "barely usable" to "faster than human hands." The combination of a floating ball UI and a dual-track architecture offers a compelling reference design for a desktop agent that fits naturally into everyday workflows.

One caveat worth noting: allowing AI to take over the mouse and keyboard and operate across applications — especially for actions like shopping or sending messages — raises real questions about permissions and security boundaries. For sensitive operations, it's still advisable to keep a human confirmation step in the loop.