[KongchangAI]
· 3 min read· 1,836 words

Qwen 4 Max Preview Hands-On: Million-Token Context and Benchmark Controversy

Qwen 4 Max Preview Hands-On: Million-Token Context and Benchmark Controversy

Qwen 4 Max Preview is cheap and dense in native apps, but benchmarks and 3D navigation expose weaknesses.

Alibaba released Qwen 4 Max Preview with a 1M token context, up to 393K output tokens, input costs of roughly $2-3 under a token package, and compatibility with OpenAI and Anthropic APIs. With no official metrics table, community tests become the main reference: GameFeel RE score of 10.3, trailing the Qwen 3.8 series and even unreleased variants; ARI Bench accuracy of 23%, far behind frontier models. Yet in its native ecosystem it completed complex procedural generation in a single 83,173-token pass, and a 30-minute task versus a competitor's 70 minutes and $6.30—a clear cost advantage. Under external frameworks it showed slow responses, parameter-following failures, and logical hallucinations, and 3D vehicle navigation failed to converge after 125 minutes. In a community poll, 56% of users rated its capability on par with Claude Opus 4.8.

Alibaba recently released Qwen 4 Max Preview, bringing a 1 million token context window, support for up to 393,000 output tokens when reasoning is enabled, and promotional discounts of up to 80%—pushing input costs down to roughly $2-3 under the token package model. The official documentation promises major leaps in agentic workflows and coding capability, but the first batch of independent graphics rendering tests and technical benchmarks reveals a huge gap between different tools.

Release Channels and Billing Model

Qwen 4 Max Preview launched directly on QwenWork, Coder, QwenApp, and QwenStudio, while also covering text, image, and video modalities through aggregation platforms such as AI Hub Mix, with an API compatible with both OpenAI and Anthropic formats. Developer XODUD confirmed that the preview only runs under the token package model, with no pay-as-you-go option before November 25, and that the Qwen team did not publish an official metrics table.

It is precisely the absence of official numbers that makes community testing the primary reference. According to multi-party hands-on tests summarized by YouTube channels, the model's performance varies significantly across different environments—something potential users should pay particular attention to.

The token package model refers to a platform providing a fixed monthly quota of token usage for a flat fee, with overages billed separately or throttled, as opposed to traditional pay-as-you-go. For developers, the "$2-3 input cost" under the package model is usually an equivalent per-million-token price derived from the package price rather than a directly listed rate, so actual spending fluctuates with call density and package tier.

An API compatible with both OpenAI and Anthropic formats means developers can reuse existing SDKs, toolchains, and prompt structures, switching models without rewriting the integration layer. This lowers migration costs, but it can also mask behavioral differences outside the model's native environment—precisely the external framework compatibility issues mentioned later.

Benchmark Controversy: GameFeel RE and ARI Bench

The first empirical shock came from the GameFeel RE benchmark shared by Bebu, covering more than 8,000 coding and game design tasks. Qwen 4 Max Preview scored 10.3, trailing Qwen 3.8 Max's 12.1 and Qwen 3.8 Flash Next's 13.0.

On the same technical chart, unreleased variants scored even higher: the open-weight model Qwen 427B Early (codename QDuck) scored 14.9, surpassing Sonnet 5.5 Extra High's 14.3; Qwen 4 Flash Early (codename QHawk) reached 19.9, easily clearing Claude Opus 5.5 Medium's 15.3. Creator Follistatin confirmed the low score on GameFeel RE and pointed out a paradox: the model has the second-lowest cost structure across all evaluation systems, yet its index is so weak.

Unreleased variants score higher on the technical chart

On ARI Bench, a reference benchmark for recursive AI self-improvement, Qwen 4 Max Preview reached 23% accuracy in Extra High mode according to the benchmark's official account data, still far behind frontier models such as GPT-6 Astra and Opus 5.5.

Open-weight models refer to release forms where model parameters are publicly downloadable and allow local deployment and fine-tuning, as distinct from closed-source models that only offer API access. The Qwen 427B Early and Qwen 4 Flash Early mentioned in the article are early variants not yet officially released; their scores appearing on the same technical chart suggests these data may come from internal or pre-release testing and are not on the same evaluation basis as the official Preview.

ARI Bench is a reference benchmark for recursive AI self-improvement, examining a model's accuracy when iteratively improving its own outputs or code. Scores on such benchmarks are highly sensitive to prompting strategy and reasoning mode; the 23% accuracy should be interpreted in the context of the Extra High reasoning tier and is not directly comparable to general capability benchmarks.

Single-Pass Generation: High-Density Reasoning and Cost Advantages

In a practical single-batch procedural generation test, creator AJ asked the model to build a maple and birch forest at peak autumn using only code, including trees, continuous animation, and interactive controls, without external tools, test frameworks, or third-party libraries. Qwen 4 Max Preview fully executed the request, consuming 83,173 tokens in a single high-density reasoning call and delivering usable scene controls and natively code-synthesized audio. This token usage was even lower than DeepSeekAd v4.1 Flash 10.17's consumption on the same task.

In another procedural test simulating an island lighthouse environment, Qwen 4 Max generated a 3D mesh, lighting loop, and interactive control panel in a single pass.

The cost comparison is equally stark. Developer JAZII compared Qwen 4 Max Preview with Minimax M3.1 Flash using the same maximum-complexity, maximum-reasoning prompt: Qwen 4 completed processing in 30 minutes at negligible cost under the Coda promotional tier, while Minimax M3.1 Flash took 70 minutes and billed $6.30 for the same task.

Qwen4 completes processing in 30 minutes at extremely low cost

But when JAZII tested through cloud code outside the model's native environment, he encountered slow responses, parameter-following failures, and logical hallucinations, pointing to compatibility issues with external test frameworks.

Procedural generation refers to dynamically building scenes, terrain, or animated content through code rules rather than manual modeling, common in game development. In the context of language models, it tests whether the model can write structurally correct, directly runnable code in one go rather than merely generating descriptive text.

A single high-density reasoning call means the model, with reasoning mode enabled, completes a large amount of internal thinking and outputs a complete result within one request; the 83,173 token consumption includes both the reasoning process and the final delivered content. This metric reflects "one-shot" capability density, differing from workflows that build things up over multiple turns of dialogue—cost comparisons should ensure consistent accounting.

Environment Sensitivity: The Gap Between Native Apps and External Frameworks

Bavi's Rooster Run challenge, run inside Coder, further broke down this environment sensitivity. Within Alibaba's own app, Qwen 4 Max Preview completed interactive generation in 11 minutes, while DeepSeek v4.1.10.17 took 34 minutes to compile the same structure.

In pure HTML game development, Mark Santos asked the model to build a classic Crossy Road downloadable file directly in the browser. Both Qwen 4 Max Preview and Claude Opus 5.5 Medium produced playable versions with complete crossing mechanics and effective collisions, with the latter narrowly winning on physics responsiveness and polish.

For vector animation, visual complexity split the results. User Shiax used Coder to generate a 2D SVG animation of a pelican riding a bicycle, noting clear improvements in element recovery and instruction following compared with the experimental Kodak model, but no major aesthetic leap. Creator LeoBuildingWithAI observed incremental progress in vector geometry with the same prompt. The official Coder Profile also reported that a Kodak test build, quietly running for four days before the Qwen 4 Max release, generated a voxel white house and floating palace with 68 navigable rooms in a single HTML file.

Narrative and 3D Tasks: Coherent but Under-Polished

In narrative simulation, Leo compared Qwen 4 Max Preview with Claude Fable 5.2 in creating a robot drama animation set in post-apocalyptic ruins. Qwen 4 needed 2 to 3 hours to work out the story arc and render a usable scene, maintaining narrative coherence but showing a clear gap in visual polish compared with Fable 5.2.

Qwen4 takes 2 to 3 hours to complete the animation

In pure aesthetic SVG tasks, such as a mechanical glass peacock displaying its tail feathers, Leo noted smooth interpolation math and transparency handling; on static renders of the Chinese zodiac, Qwen 4 Max surpassed DeepSeek V4.1 Flash, with visual fidelity second only to GPT-6 Astra. In a developer poll Leo launched after 24 hours of testing, 56% of users considered the preview's overall capability on par with Claude Opus 4.8, 26% thought it matched Opus 5, and 13% considered it equivalent to Sonnet 5.

When handling 3D through the DeepSeek test framework, user Shiaks reported a quality drop when generating a rainy-night convenience store scene: Qwen 4 Max's lighting was flatter and textures weaker than in the earlier Kodak preview tests. Leo confirmed this 3D sluggishness when generating a floating voxel pagoda: a full hour of rendering yielded only geometry density close to DeepSeek V4.1 Flash.

Parametric Assets and Spatial Computing: Highlights and Weaknesses

In the field of parametric assets for game engines, creator Wass3 reported that a single prompt in Coder generated a complex industrial 3D model, with each component individually named, meshes cleanly decoupled, real-world scale units, and clean topology, importable directly into Blender without manual cleanup.

In a volumetric vegetation analysis recorded by a Qwen AI fan, the model segmented an orchard tree into 92,582 voxels, isolating and quantifying a leaf volume accounting for 81.3% of the total, demonstrating sharp spatial computing ability for organic structures.

The model quantifies leaf volume at 81.3% of the total

But physics-controlled dynamics exposed severe limitations in 3D vehicle navigation. Developer Lu Yao documented a series of tests in which language models drove a vehicle in a 3D driving environment: Tencent Hunyuan completed the track and crossed the finish line in 56 minutes, DeepSeek V4.1 finished in 82 minutes, while Qwen 4 Max Preview required 125 minutes of continuous execution, repeatedly failing at spatial trajectory computation—the vehicle was driven off cliffs, restarted over and over, and never converged onto the track.

On the financial side, creator IAM tested against GPT 6.1 Sol Ultra as a reference and found that generating a complete photorealistic scene took about an hour and consumed only 1.5% of its $5 monthly token package. At the platform level, Leighton Gott noted that Qwen 4 Max integration already runs on ClaudeCode, Codex, Cursor, and OpenCode, with base monthly subscriptions around 60%.

In technical community discussion, MD Ismail Sojal suggested that a dense 27-billion-parameter variant planned for open-weight release could offer stronger practical efficiency than the current Max Preview on high-end local hardware, especially given the heavily inference-driven self-improvement that Alibaba has begun promoting.

The picture of Qwen 4 Max Preview is therefore quite split: within its native ecosystem, it completes complex tasks at extremely low cost and with considerable single-pass generation density; but in external frameworks and scenarios such as 3D physics navigation, environment sensitivity and spatial reasoning weaknesses are clearly visible. For developers, the choice between adopting it now or waiting for the open-weight dense version depends on whether the workload falls in the range where it excels or lands on its soft spots.

Voxels are volumetric pixels in three-dimensional space, commonly used in voxel modeling and destructible terrain in games. The "92,582 voxels" and "leaf volume at 81.3%" in the article reflect the model's ability to spatially segment and quantify organic structures, which falls under spatial computing and is a different task from generating mesh models.

Parametric assets refer to 3D models with named components, real-world scale units, and clean topology that can be imported directly into tools like Blender without manual cleanup. Clean topology means a reasonable mesh face count and smooth edge flow—an important criterion for whether an asset can be used directly in a production pipeline, and a key standard for measuring the engineering usability of model output.

#Qwen 4 Max Preview#million-token context#GameFeel RE benchmark#ARI Bench#token package model#3D navigation
Share:

Related articles