DeepSeek V4.1 Flash Beta Tested: Six Problems, Four Models, Head-to-Head

A Time-Limited Beta: V4.1 Flash Quietly Arrives

DeepSeek recently released a beta version of V4.1 Flash for public testing, with an expiration date baked right into the API endpoint ID — September 10th. The team announced in their internal beta group that they wanted users to experience firsthand how it stacks up against V4 Pro. There's no barrier to entry: just plug into the API and go, and the pricing is identical to the current V4 Flash.

Before diving into the results, a word of caution: this is almost certainly not V4.1 Flash's final form. The short testing window feels more like feedback collection than a full release. Looking back at V4 Flash's launch, the gap between the beta and final versions was substantial — post-training improvements tend to land after the official release. So treat these results as observations about this specific build, not a definitive scorecard.

Evaluation Approach: Six Composite Problems, Four Models, Back-to-Back

This evaluation deliberately avoids single classic benchmark questions. The reason is simple: models have likely been specifically trained on them, and getting one right says little about genuine capability. Instead, the test uses six composite problems spanning front-end development, reasoning, and causal inference, with all four models running the same prompts back-to-back — comparison is what gives results their meaning.

The four models in the lineup are: DeepSeek V4.1 Flash (the subject under review), V4 Flash, V4 Pro with deep thinking enabled, and JLM 5.3 Flash as a reference point. All three Flash variants run in direct-output mode; V4 Pro uses thinking mode. All models were accessed through the official API with single-shot generation — no retries or cherry-picking.

Problem 1: Physics Constraints in a Wind-Blown Cloth Simulation

The first problem asked each model to generate an interactive cloth — displayable, draggable, and cuttable, with any cut realistically affecting subsequent motion. The four models produced noticeably different cloth aesthetics: V4.1 Flash rendered a dark gray checkerboard pattern, V4 Flash a two-tone grid, V4 Pro constructed an elaborate fabric with 3,648 physics springs, and JLM 5.3 Flash produced a fine mesh with an off-white, textured appearance.

The real test came after dragging and slicing. The moment V4.1 Flash's simulation started, particle coordinate calculations exploded — the entire cloth simply vanished, leaving only two fixed anchor points. V4 Flash, V4 Pro, and JLM 5.3 Flash all successfully simulated a real physical cut with proper gravity-driven falling.

The key criterion here is physics constraints and gravity-driven separation. V4.1 Flash stumbled right out of the gate.

Problem 2: Cube Net Unfolding and 3D Folding Animation

The second problem asked each model to transform a flat sheet into a rotatable, unfoldable cube, with the outer face always remaining connected to its initial state. Both Flash-tier models immediately produced blank screens — V4.1 Flash and V4 Flash failed to render any geometry at all. V4 Pro and JLM 5.3 Flash successfully drew complete six-face flat nets.

In the subsequent 3D folding test, V4 Pro's net was correct but it got the rotation axis and closure logic wrong — the top face swung sideways and intersected other panels, making it impossible to close into a box, earning a failing grade. JLM 5.3 Flash delivered smooth spatial animation with five faces closing tightly, with only the top T-face not fully latching — the best result overall.

Worth noting: the two Flash models' blank screens had completely different root causes. V4 Flash was given a 65,536-token budget but fell into a loop of self-doubt while deriving the top-face matrix, repeating the same uncertain derivation in the code 1,026 times in a row, ultimately blowing through the entire token budget and getting truncated. V4.1 Flash, on the other hand, hand-rolled a 3D engine and got the projection sign backwards, which caused all geometry faces to be culled entirely.

The "token budget explosion" V4 Flash experienced is a classic failure mode in autoregressive language models during inference: when a model lacks confidence in an intermediate step while generating code comments or inline derivations, it repeatedly outputs the same uncertainty-marked text ("repetition looping") until it hits a hard context window or max-output-length cutoff. 65,536 tokens is the single-output cap for many model APIs — once hit, generation is forcibly terminated, leaving incomplete code and a blank screen. This is distinct from a model "not knowing the answer" — it wasn't silent; it burned through its entire budget in a highly repetitive way, representing a convergence failure during inference. By contrast, V4.1 Flash's blank screen was a pure geometric calculation error (inverted projection matrix sign) — a deterministic logic bug. Both led to failure, but their root causes are entirely different.

Problem 3: Vector Design Tool and State Machine Logic

The third problem asked for a functional vector design tool with add, delete, and recolor operations, plus correct undo/redo functionality. This time all four models delivered high-quality results: V4.1 Flash produced a Bauhaus-inspired dark theme, V4 Flash a classic white-background toolbar, V4 Pro a feature-complete implementation, and JLM a Swiss poster-style layout with strong design sensibility.

Under a stress test — doing a long sequence of operations, undoing all the way back to the initial state, then drawing new shapes — all four models' state machine logic passed with full marks. The key criterion is state machine correctness and the undo/redo chain. All four passed, and this was V4.1 Flash's most stable performance, with the best industrial-quality UI of the four.

Problem 4: Minimum Conflict Reasoning on an Unsolvable Layout

The fourth problem was pure reasoning: a layout table with no valid solution, requiring identification of all minimal conflict sets and the minimum set of modifications needed to fix it. The correct answer is five complete minimal unsatisfiable subsets (MUSes); because of the disjoint rule sets, at least two rules must be removed, and there are exactly two valid ordering fixes.

The results diverged sharply: V4.1 Flash found only two conflict groups before contradicting itself; V4 Flash repeatedly reversed its own conclusions and was truncated without converging; JLM 5.3 Flash, in a lightweight direct-output pass, correctly identified all five conflict groups and produced the optimal solution; V4 Pro with deep thinking enabled produced a 37,000-character chain of thought with textbook-quality rigorous mathematical proofs. JLM 5.3 Flash was a standout here, V4 Pro also scored full marks, while both direct-output Flash models failed to converge.

A Minimal Unsatisfiable Subset (MUS) is a core concept in constraint satisfaction problems (CSPs): within an unsolvable set of constraints, a MUS is the smallest subset such that removing any single constraint from it makes the whole system satisfiable. A problem can have multiple mutually disjoint MUSes simultaneously — which is why the correct answer requires finding "all five" of them. Missing any one means the model hasn't fully grasped the conflict structure. Finding all MUSes and determining the optimal deletion strategy requires not just enumeration ability, but also understanding the covering relationships between MUSes (the "hitting set" problem), which is NP-hard combinatorial optimization. JLM 5.3 Flash completing all five MUS derivations in a lightweight direct-output pass suggests strong systematic search ability in constraint reasoning — not just memory or pattern matching.

Problem 5: Timeline Robustness in a 12-Second Title Sequence

The fifth problem was relatively approachable: a 12-second text-only title sequence, testing typography and timeline correctness — scrubbing to any point in time should render correctly. The four outputs had distinct styles: JLM produced a beautifully composed three-column layout, V4 Pro and V4 Flash were clean and restrained, and V4.1 Flash was visually rich with cards carrying realistic business labels.

Playing from the end backwards, all four models maintained stable temporal determinism — a clean sweep. V4.1 Flash showed strong business-context aesthetics, while JLM and V4 Pro led on typographic quality.

Problem 6: Causal Inference with Simpson's Paradox

The hardest problem embedded a Simpson's Paradox scenario, requiring the construction of two worlds with completely opposite causal effects, plus derivation of the upper and lower bounds of the treatment effect. The acceptance criteria were bounds of −73 to +27 percentage points, with all real observed data preserved in full.

Surprisingly, all four models demonstrated strong mathematical foundations on this rigorous causal inference problem — all identified the Simpson's Paradox reversal, all constructed counterfactual worlds consistent with the observed data, and all derived theoretically correct bounds. The depth of mathematical and statistical reasoning in domestic Chinese models is genuinely impressive.

Simpson's Paradox refers to a phenomenon where each subgroup in partitioned data shows a correlation in one direction, but the aggregated overall data shows the opposite. This appears frequently in medical trials and social statistics, with the core cause being a "confounding variable" that simultaneously influences both treatment assignment and outcomes. Addressing Simpson's Paradox in causal inference requires introducing a Structural Causal Model (SCM) or do-calculus to distinguish between "observed correlation" and "post-intervention causal effect." The "effect bounds" correspond to the Partial Identification framework — when confounding cannot be fully controlled, researchers derive the theoretical range of extreme values for the causal effect while preserving all real observed data, rather than giving a single point estimate. This is logically more rigorous than simply answering "does the treatment work?" and is considerably harder for models to handle correctly.

Summary: V4.1 Flash Scores 3 — Adequate but Unremarkable; JLM 5.3 Flash Leads

Across all six problems, JLM 5.3 Flash claimed first place with 5 points, demonstrating comprehensive seasoned-level capability. V4 Pro with thinking enabled also scored 5 points, showing deep grounding in causal reasoning, code, and complex inference. V4 Flash came in at 4 points with a solid performance. The star of the review, V4.1 Flash, earned only 3 points — adequate but unremarkable among direct-output models.

It bears repeating: this was a single-pass, direct-output test. Results represent only this specific model's performance on these specific problems, and V4.1 Flash has considerable room to grow before its final release. Overall, JLM 5.3 Flash has set a benchmark-level standard in the lightweight direct-output category — but models with that combination of performance and pricing remain rare among domestic Chinese models. Here's hoping more of them emerge.