DeepSeek Harness Hands-On Review: A New Benchmark for Plugin-Based Agentic Coding

DeepSeek recently launched its V4 Pro model alongside something even more noteworthy for developers — DeepSeek Harness, a fully homegrown agentic coding system. Agentic coding refers to an AI paradigm where the system doesn't just generate code in one shot, but acts like an autonomous developer — planning tasks, executing step by step, calling tools, reading and writing files, running tests, and iterating based on results. Unlike traditional code completion, agentic coding emphasizes end-to-end task completion — from understanding requirements to delivering a fully runnable project. Since 2024, products like Claude Code, Cursor Agent, and Codex have launched in succession, signaling this paradigm's shift from experimental to mainstream. Compared to model iterations alone, this developer preview of the Harness framework demonstrates significant ambition in both design philosophy and real-world experience. This article is based on an in-depth hands-on review by a tech blogger, covering the system's core highlights and actual performance.

DeepSeek V4 Pro Model Upgrade: Price-Performance Remains the Killer Advantage

DeepSeek positions the V4 series as a "major agentic upgrade." Agentic coding has been a fiercely contested battleground among domestic large models, and V4 Pro is no exception. It also introduces flexible reasoning effort options for the Flash version and adds support for the OpenAI Responses API.

Reasoning effort is an important parameter recently introduced in reasoning models, allowing users to specify how much computational resources the model should invest in thinking during a call. High reasoning effort means the model performs longer chain-of-thought reasoning, suitable for complex math or programming problems; low reasoning effort provides quick answers for simple queries. The essence of this design is giving users flexible control over the tradeoff between latency, cost, and quality, avoiding a one-size-fits-all compute budget for every request. DeepSeek's choice to support the OpenAI Responses API — a next-generation interface standard supporting tool calling, multi-turn conversation state management, and streaming output for agentic features — means existing applications and frameworks built on the OpenAI ecosystem can switch to a DeepSeek backend at relatively low cost. This is a classic ecosystem leverage strategy.

Based on independent benchmarks, V4 Pro ranks eighth overall, trailing Qwen3.8 Max (released just days prior), but it's important to note its parameter count is far smaller than models like Kimi K3 or Qwen3.8. The blogger assessed it as more "aligned with Kimi K3" rather than competing at the GPT-5 level.

Notably, pricing has increased 2x to 4x. But even so, it remains extremely competitively priced among frontier-tier models, sitting at the optimal frontier of cost-performance tradeoff. This also reflects the reality that major labs universally face compute constraints — DeepSeek is no exception.

Harness's Core Philosophy: Everything Is a Plugin

DeepSeek Harness is still in developer preview, and the team openly acknowledges rough edges. But its design philosophy is remarkably forward-thinking — everything is treated as a plugin, including tools, skills, and even rendering capabilities.

Highly Composable Plugin Architecture

This plugin-based design means developers can freely mix, match, replace, and extend different capabilities. Plugin architectures have a long history in software engineering, from Eclipse IDE to VS Code. In the AI agent domain, being plugin-based means each capability (file operations, terminal execution, web search, code analysis) is encapsulated as an independent module, registered and invoked through a unified interface. The greatest advantage is composability — developers can flexibly assemble capability stacks based on task requirements, even embedding third-party agents as sub-modules to form multi-agent collaboration topologies.

More interestingly, you can even call other coding agents (like Claude Code) as plugins, enabling multi-agent systems built on this framework. DeepSeek elaborates on this "composability-oriented programming" philosophy in an accompanying paper.

The installation experience is smooth. The blogger cloned the repository and installed from source, then launched a local web application (running on port 3018 by default). The web app features a polished interface — enter your DeepSeek API key on first login and you're ready to go.

Flexible Work Modes and Permission System

After setting up a workspace (essentially a project folder), users can choose from multiple work modes: Full mode with file editing, web search, and complete coding capabilities, Code mode, Mechanical mode, and Creator mode with customizable profiles and agents.

Permission management offers four tiers: read-only, workspace write, and full access — where "full access" is equivalent to Claude Code's dangerous skip-confirmation mode. The entire framework uses the MIT license — one of the most permissive open-source licenses, allowing anyone to freely use, modify, and distribute code including for commercial purposes, with the only requirement being preservation of the original copyright notice. Compared to competitors using AGPL or custom licenses, MIT has virtually no compliance burden in commercial scenarios, making it very easy to integrate other models (including locally deployed versions). All configuration is managed through a single YAML file, and the web search plugin auto-enables once a DeepSeek key is provided.

Coding Test: 20 Million Tokens to Complete an ISS Real-Time Tracker

To evaluate Harness's real capabilities, the blogger set up a test task: create a real-time tracker for the International Space Station, polling an API endpoint every 5 seconds for live data.

Impressive Speed and Cache Hit Rates

Both Flash and Pro versions ran remarkably fast. The interface footer displays detailed runtime information in real time — tokens per second, cache hit rate, number of turns, elapsed time — which deserves special praise, as many similar tools deliberately hide this data.

In testing, token generation speed reached 112 to 130 tokens per second, and most impressively, the cache hit rate was 95% to 100%. In LLM inference, KV Cache (key-value cache) stores previously computed attention key-value pairs. When prefix content remains unchanged across multi-turn conversations, caching avoids redundant computation, dramatically reducing latency and cost. A 95%-100% cache hit rate means Harness maintains prefix stability with extreme care when constructing prompts — requiring extensive engineering optimization in system prompts, context management, and tool output concatenation order. For agentic coding scenarios involving dozens of interaction rounds, high cache hit rates directly determine response speed and final costs. The blogger stated this is one of the best cache designs he's seen recently — DeepSeek clearly invested significant effort here.

Output Quality Exceeds Expectations

The entire task took approximately 35 minutes, went through two iterations (requiring manual intervention to fix a minor issue), and consumed a staggering 20 million tokens with roughly 240,000 output tokens. While token consumption was admittedly high, the output quality thoroughly impressed the blogger.

The space station tracker not only tracked position accurately (nearly perfectly matching real-time data), but also delivered impressive visuals. The system selected appropriate shaders to render Earth through multiple attempts, and even the Sun's position relative to Earth was correct — testing occurred during US daytime, and the output faithfully reflected this. Such details had previously been a weakness of DeepSeek models, but Harness clearly improved this.

DeepSeek Flash vs Pro: How to Choose for Coding Tasks

An interesting finding was that the blogger preferred the Flash version over Pro for coding tasks. While Pro represents a major upgrade over its predecessor with extremely competitive pricing overall, Flash delivers better ROI — significantly cheaper with coding performance that's no worse. For agentic coding tasks, the blogger strongly recommends trying Flash first.

A Notable Development Detail

The blogger mentioned that DeepSeek Harness itself was developed with substantial assistance from Codex, with at least 20% of commits and pull requests coming from Codex workflows. This explains why its interface and interaction style closely resemble Codex — the experience of progressing through a to-do list item by item is virtually identical. This phenomenon itself reflects an interesting trend in current AI tool development: AI agents are being used to build next-generation AI agents, forming a bootstrapping development loop.

Current Limitations and Future Outlook

Despite the overall satisfying experience, Harness reveals a persistent issue with DeepSeek's open models: low token efficiency. Token efficiency refers to the number of tokens consumed to complete a given task. Open-source models and some domestic models are generally weaker than Claude and GPT series in this regard, due to differences in code proportion and quality in training data, redundant output from instruction-following capabilities, and lack of effective early-stopping mechanisms in agent loops. Consuming 20 million tokens to complete a medium-complexity project means that while the unit price is low, the total volume is significant — this is why efficiency optimization remains important even after DeepSeek's price increase. The blogger hopes future iterations will improve token efficiency.

Additionally, multi-model support isn't yet available — once added, it will dramatically expand this framework's potential. Since Harness is a self-deployable web application, it can theoretically be deployed to the cloud for anytime, anywhere access to your coding agent — a highly attractive direction. The blogger also mentioned that V4 Flash can run locally on their own DGX cluster with solid results. DGX is NVIDIA's high-performance computing system designed specifically for AI training and inference, equipped with multiple top-tier GPUs per unit, capable of running large-scale language models locally without relying on cloud APIs — particularly important for enterprise scenarios with strict data privacy requirements.

Overall, DeepSeek Harness sets a new benchmark for agentic coding tools with its plugin-based, composable design philosophy, excellent cache performance, and transparent runtime statistics. While still in early preview, it's already enough to generate excitement for its official release.