Same Model, Three AI Agents: DeepSeek Harness vs. ZCode vs. Hermes — Which One Wins?

Same Model, Three AI Agents: DeepSeek Harness vs. ZCode vs. Hermes — Which One Wins?

With AI Agent tools multiplying by the day, one question often goes unasked: if you use the same model and the same prompt across different Agent frameworks, how much will the outputs actually differ? A Bilibili creator recently ran a hands-on test and got a clear answer — the differences are bigger than you'd expect.

The test used GLM 4.5 Flash (referred to in the video as "GIM 5.3 Flash," likely a GLM-series Flash model) as the underlying model, paired with an identical prompt, and ran it across three Agent environments: DeepSeek Harness (called "Harmonies" in the video), ZCode (the model's native environment), and Hermes. The evaluation criterion was refreshingly simple — "Create the most beautiful file you can imagine, one that will make people gasp when they see it."

Test Setup: A Controlled Agent Showdown

What makes this test valuable is its controlled-variable design. The model stays the same, the prompt stays the same — the only variable is the Agent framework wrapping the model. Any differences in output can therefore be attributed almost entirely to how each framework breaks down the task, its code generation strategy, and the design of its execution pipeline.

The prompt itself is deliberately open-ended — asking for "the most beautiful file" is really a test of how well each Agent interprets an ambiguous requirement, exercises creative autonomy, and translates the abstract concept of "beauty" into runnable code.

For developers, open-ended tasks like this reveal differences in an Agent's "way of thinking" far more effectively than structured programming tasks. Given the exact same instruction, one Agent might conservatively generate a basic page, while another proactively adds music, animations, and interactive transitions to enrich the experience.

What is an Agent Framework? An Agent framework is an execution layer built on top of a large language model. It takes a user's natural-language instruction and breaks it down into a sequence of executable subtasks, while orchestrating capabilities like code generation, tool calls, and error retries. When the same underlying model runs inside different frameworks, it's essentially being guided by different "cognitive scaffolding" — the framework determines what the model is asked to do first, how the context window is allocated, and how failures are handled. This is why, in a controlled-variable test, the choice of framework can directly impact output quality: it doesn't change the model's weights, but it does change how and how often the model is invoked.

ZCode (Native Environment): Leanest Code, Not the Worst Result

ZCode, as the "native" environment for the GLM Flash model, completed the task in 32 minutes and 41 seconds and generated 893 lines of code — the shortest of the three.

The output was a scrollable page with background music, which the creator described as sounding "closer to white noise." Despite generating the least code, it didn't come in last. This challenges a common misconception: line count does not equal output quality. Shorter code can mean a more concise implementation, not missing functionality.

ZCode landed in second place overall in the creator's ranking — a solid, reliable performer. For scenarios where code maintainability matters and redundant logic is unwanted, this "just enough" approach is actually an advantage.

DeepSeek Harness: Faster and Better

DeepSeek Harness ("Harmonies" in the video) was the clear winner of this test. It completed the task in just 28 minutes — faster than ZCode — and generated 1,313 lines of code in "standard mode," some 400–500 lines more than ZCode.

More importantly, the result looked impressive. The creator described the page as "pretty stunning" — beautiful enough to use as a computer lock screen — and it supported interactive switching between views. With the triple advantage of faster execution + richer code + more visually striking output, DeepSeek Harness took first place in this comparison.

This result is worth thinking about: the higher line count here corresponded to richer visual output, not meaningless bloat. This suggests that the Harness framework may do a better job of "unlocking" the same underlying model — giving it more room to express itself. The same GLM Flash model, in different Agents, seems to operate with a fundamentally different creative ceiling.

A note on "Harness": In AI evaluation contexts, a "harness" typically refers to a test-driven execution framework originally designed to standardize model performance assessment. As adopted by DeepSeek for Agent use, the framework excels at structured task decomposition — breaking vague instructions into multiple verifiable sub-goals, then generating and validating code incrementally. This "generate-then-verify" approach may explain why Harness produced more code yet ran faster than ZCode: each step is more directional, reducing the likelihood of large-scale backtracking or rewrites, and allowing richer visual layers to accumulate in the final output.

Hermes: Slowest and Last

Hermes turned in the weakest performance of the three. The creator admitted they "stared at it for a while and couldn't figure out how long it took," but estimated its runtime was the longest of all three — noticeably exceeding both ZCode and DeepSeek Harness.

Considering the final output quality, Hermes ranked third in this test. Spending more time without producing better results is a telling sign: execution efficiency and output quality in an Agent framework aren't always linearly related, and the soundness of the framework's design matters enormously.

What Does This Comparison Tell Us?

This informal but illuminating test yields a few conclusions worth keeping in mind:

The impact of the Agent framework is massively underestimated. The same model running in different Agents produced noticeably different results across both speed and final output quality. Picking the right Agent framework may deliver more immediate gains than upgrading to a better model.

Line count is not a quality metric. ZCode generated the least code but didn't finish last; Hermes took the longest but ranked at the bottom. Agents should be judged on final deliverable quality, not on intermediate process metrics.

Open-ended tasks are an excellent way to test an Agent's creative ceiling. Vague prompts like "create the most beautiful file" are effective at differentiating Agents based on their capacity for autonomous expression and experience enhancement.

The creator's final ranking: DeepSeek Harness > ZCode > Hermes. Of course, this was a single-run, single-model, single-prompt subjective evaluation with a limited sample size — treat the results as directional rather than definitive. More reliable conclusions would require repeated testing across a wider range of task types and model combinations.

For developers currently evaluating their tooling, the core takeaway from this comparison is simple: before investing in a more expensive model, try switching your Agent framework first.