DeepSeek V4 Pro vs. Codex: A Hands-On Comparison of AI-Recreated Don't Starve

An Informal but Fascinating Real-World Test

In the early hours of August 13, DeepSeek quietly released its V4 Pro model. A Bilibili content creator, unable to sleep, decided to put it to the test with an intriguing challenge — having AI recreate the classic survival game Don't Starve from scratch.

To make the comparison more compelling, the creator simultaneously tasked OpenAI's Codex (using version 5.6 with maximum reasoning intensity) with the exact same challenge. It's worth noting upfront that since DeepSeek's official Agent tools haven't been released yet, DeepSeek V4 Pro had to run through a third-party tool called Zcode, while Codex used its own official environment.

Some technical background is helpful here. DeepSeek V4 Pro is a next-generation large language model released by DeepSeek in August 2025, continuing the company's strong presence in the open-source community. DeepSeek had previously demonstrated GPT-4o-level competitiveness in code generation and mathematical reasoning with its V3 and R1 series models. Codex, on the other hand, is OpenAI's dedicated programming Agent product launched in 2025, built on the codex-1 model. It's not just a language model — it's a complete engineering environment that includes a code execution sandbox, file system access, terminal operations, and test runners. This distinction is crucial: Codex isn't merely a language model but a tightly coupled "model + engineering environment" system.

This introduces an unavoidable confounding variable: the two models were running in different environments. The creator acknowledged at the beginning of the video that this was a "purely subjective and not particularly rigorous" evaluation, meant to be taken with a grain of salt. Even so, the details that emerged during testing are well worth analyzing in depth.

DeepSeek V4 Pro's Performance: Stunning Frontend, Broken Gameplay

A Well-Structured Planning Process and Self-Testing Mechanism

With nothing more than the six-character Chinese prompt meaning "recreate the game Don't Starve," DeepSeek V4 Pro spent 43 minutes completing the entire project on Zcode. Its workflow was commendably systematic:

• Before starting, it proactively asked three key questions: tech stack preference, gameplay scope, and visual style;
• It explicitly stated that all assets would be originally drawn, not copied from official resources;
• It delivered a complete plan including file structure, gameplay features, and implementation steps, estimating 2,000+ lines of code built in phases.

The most surprising aspect was its self-verification capability. DeepSeek opened the game page in a browser for testing and even took screenshots to verify rendering. However, DeepSeek lacks visual understanding — it can't interpret images — so it fell back on an alternative approach: reading the game's internal state and canvas pixel data to assess rendering correctness.

This self-verification strategy represents an important research direction in AI programming. Traditional AI code generation is one-directional — the model outputs code, humans test it. In Agent mode, AI needs the closed-loop ability to "write code → run → observe results → fix." DeepSeek's approach was quite clever: as a text-only model unable to "see" screenshots like multimodal models can, it adopted a workaround — indirectly assessing rendering correctness by reading HTML Canvas pixel data and internal game state variables. While logically sound, this method has obvious blind spots: it can verify the "data layer" (e.g., whether the hunger value is decreasing) but struggles to verify the "interaction layer" (e.g., whether mouse clicks trigger gathering actions) — and the latter is the core of the gaming experience.

It even "played" the game itself to verify mechanics like "hunger decreases over time," claiming all 38 tests passed.

Actual Experience: Nothing Left but Walking and Getting Hit

However, reality fell far short of expectations. When the creator actually played the game, problems piled up:

• The frontend interface was genuinely decent — the homepage looked passable with a minimalist hand-drawn paper-craft style;
• But core gameplay was almost entirely broken — couldn't gather resources, attack (F key) didn't work, no pause function, no crafting;
• The player character was essentially limited to "moving" and "getting beaten up";
• At nightfall, fire was needed for light, but with no way to make fire, the character could only watch helplessly as they took "time-based damage" until death.

DeepSeek acknowledged this in its verification report: due to limitations of the testing browser environment, it hadn't actually tested "canvas click interactions" and classified this as an "environment issue, not a game issue." This foreshadowed the deeper analysis to come.

This "environment issue" exposed the fundamental limitations of Zcode as a third-party Agent tool. Compared to a purpose-built official Agent environment like Codex, third-party tools face inherent disadvantages across multiple dimensions: first, toolchain compatibility — official environments can optimize specifically for the model's output format, context window management, and function-calling mechanisms; second, test feedback loops — Codex can run code directly in a sandbox, capture errors, and auto-fix, while Zcode's browser interaction testing capabilities are clearly insufficient; finally, context management strategies — different tools handle memory retention for long tasks and subtask decomposition in different ways. These differences directly impacted the quality of the final output.

Codex's Performance: Less Polished UI, but Solid Gameplay

Two Controlled Experiments

The creator set up two projects for Codex:

Project One: Fed Codex the planning document generated by DeepSeek and had it follow the specifications. Completed in 19 minutes, with no obvious errors found during testing.

Project Two: Same as DeepSeek — just the six-character prompt, complete creative freedom. Completed in 20 minutes.

Game Features Were Largely Complete

While the creator admitted Codex's frontend interface wasn't as visually appealing as DeepSeek's, the gameplay completeness formed a stark contrast:

• The game started with basic resources like grass, twigs, berries, and rocks;
• It supported a pause function with keyboard shortcut instructions in the upper left corner;
• E key for interaction and picking up items worked, spacebar attack was functional;
• C key crafting supported making a stone axe, and weapons actually increased damage;
• Before nightfall, players could craft torches for light — and with materials on hand, could even "instantly craft torches."

These were precisely the features completely missing from the DeepSeek version. The creator joked at one point: "No wait, it's not DeepSeek's fault — it must be Zcode's fault."

Detailed Documentation Significantly Improved AI Output Quality

An interesting finding was that the project built from detailed documentation was noticeably better than the one from just the six-character prompt. And that detailed document was written by DeepSeek. This indirectly confirms that DeepSeek V4 Pro's planning and solution design capabilities are genuinely strong — the problems lay more in execution and self-verification.

This finding powerfully validates a core principle in Prompt Engineering — the quality of a large language model's output is highly correlated with the richness of its input. The six characters meaning "recreate the game Don't Starve" carry a wealth of implicit knowledge for human developers (game mechanics, interaction patterns, art style, etc.), but for AI, all of this needs to be explicitly spelled out. DeepSeek's generated planning document effectively performed a "tacit-to-explicit knowledge conversion" — expanding six characters into a complete specification covering file structure, gameplay details, and tech stack choices. When this document was fed to Codex, the latter's output quality was significantly better than its free-form version. This shows that in AI programming scenarios, "requirements analysis" and "solution design" capabilities are equally critical competitive advantages — not just code implementation ability.

That said, Codex's second project also had its flaws. For instance, the day/night time indicator was inaccurate — the interface displayed daytime when it should have been night. After modifications it improved somewhat, such as being able to switch held items, but the time indicator issue was never fully resolved.

The Core Question: Model Capability or Tool Environment Differences?

This is the most thought-provoking question from this test. The creator's preliminary conclusion was:

DeepSeek V4 Pro + Zcode's Don't Starve recreation was "absolutely terrible," but it's very likely not the model's fault — rather, Zcode was unable to fully leverage DeepSeek V4 Pro's capabilities.

Key evidence supporting this conclusion:

1. DeepSeek's planning capabilities are solid — the planning document it wrote enabled Codex to produce a better result than Codex's own free-form attempt;
2. Click-to-gather functionality could never be reproduced, even after having DeepSeek modify the code — and DeepSeek itself explicitly identified the testing environment (browser clicks) as the root cause;
3. The creator admitted to having limited experience with Zcode, having only used it for local tasks before and never for a complete project, with little familiarity with Zcode's operational mechanics.

In other words, this comparison had a fundamental fairness issue regarding execution environments. Codex used an officially deep-optimized Agent environment, while DeepSeek was forced to use a third-party tool — their engineering capabilities and test feedback loops were simply not on a level playing field. The coupling of model capability and engineering environment made any "which is better" conclusion ambiguous.

This also exposes a widespread methodological dilemma in current AI programming evaluations: how to decouple "model capability" from "engineering environment capability." Academic code evaluation benchmarks like HumanEval, MBPP, and SWE-bench typically test code generation in standardized environments where variables can be well-controlled. But in real-world Agent scenarios, model performance is highly dependent on toolchain support — including code execution environments, error capture mechanisms, file system operations, and network access capabilities. A more scientific evaluation approach should include: comparing different models in the same engineering environment (controlling for environment variables), comparing different toolchains with the same model (controlling for model variables), and scoring planning ability and execution ability separately.

Conclusion: AI Programming Ability = Model × Engineering Environment

More than a rigorous model evaluation, this late-night test serves as a vivid case study of "AI Programming Ability = Model × Engineering Environment."

For those following AI programming, several takeaways are worth remembering:

• A powerful base model without a matching Agent toolchain and test feedback mechanism may see its actual output dramatically diminished. This also explains why major AI companies have recently been investing heavily in building their own programming Agent platforms — the model is only half the infrastructure; engineering capability is the other half.
• Detailed requirements documents significantly improve AI output quality; ultra-minimal prompts like a six-character instruction rarely yield ideal results. In practice, spending 10 minutes writing a clear requirements document may save several times the effort of repeatedly asking AI to fix bugs.
• When evaluating AI programming capabilities, always pay attention to the critical variable of runtime environment. Different Agent tools can have a massive impact on the same model's performance, and discussing model capability without considering the environment can lead to misleading conclusions.

The creator also mentioned plans to re-test on Zcode with more tuning, or to wait for DeepSeek's official Agent release for a fairer rematch. Only then might DeepSeek V4 Pro's true capabilities be more accurately represented. Until then, the verdict on this "Don't Starve recreation showdown" is far from settled.