DeepSeek V4.1 Flash 1017 Benchmark Comparison: The Cheaper Model Beats Pro Across the Board?

DeepSeek's pricing strategy has produced a counterintuitive situation: the cheaper model beats the more expensive version on every officially published coding and Agent benchmark. V4.1 Flash 1017 (released October 17) versus V4 Pro 0813: Terminal-Bench 4.0 scores 42.3 versus 12.4, and DeepSWE v1.1 scores 77.0 versus 62.7. Meanwhile, Pro's price per output token is still 3.3× that of Flash, and a single costed Agent session works out to roughly four times the Flash bill.

On paper, the winner seems decided. But if you're running a pipeline tuned for deepseek-v4-pro-0813, those numbers are DeepSeek^Ad's, not yours. The real engineering question is validating the switch on your own workload—pass rate, output length, cost per run—measured with the same methodology across both models.

The Officially Published Numbers

Pricing: Where Pro Costs More

According to DeepSeek's pricing page, in USD per million tokens, V4.1 Flash 1017 is $0.30 input, $1.20 output, $0.006 cache hit; V4 Pro 0813 is $1.32 input, $3.96 output, $0.044 cache hit. Both are half price during off-peak hours. Third-party platform AIHubMix lists slightly higher prices (Flash 1017 at $0.3098 / $1.2392, Pro 0813 at $1.3944 / $4.1832), but the ratios are consistent.

Converted, Pro's input price is 4.4× Flash's, output 3.3×, and cache hits over 7×. In addition, on DeepSeek's own API, Pro's concurrency cap is only one-fifth of Flash's: 500 versus 2500. This means Flash's throughput advantage in high-concurrency scenarios isn't just about price.

Benchmarks: Flash Leads Nearly Everywhere

Pro's numbers come from Table 3 of the DeepSeek V4.1 Flash paper, all measured at maximum effort; 1017's numbers come from the model card. Note that all figures are DeepSeek's own measurements, with no third-party reproduction yet.

The key comparisons are as follows: on Terminal-Bench 4.0, Pro 12.4, September Flash 31.2, 1017 42.3; on DeepSWE v1.1, Pro 62.7, September Flash 74.2, 1017 77.0; on ProgramBench, Pro 15.5, September Flash 20.3, 1017 29.1; on NL2Repo-Bench, Pro 61.5, September Flash 65.4, 1017 68.2; on Automation-Bench, Pro 43.2, September Flash 54.8, 1017 60.5; on Agents' Last Exam, Pro 25.7, September Flash 31.8, 1017 35.4.

The September Flash already beat Pro on every coding and Agent metric, and 1017 widens the gap further. On the pure-text HLE subset, 1017's 44.0 also exceeds Pro's 42.7. Pro's only possible remaining advantage is GPQA Diamond (92.4 versus September Flash's 90.9), but DeepSeek hasn't published 1017's score on that item. Artificial Analysis independently reached the same conclusion on its general index in September, calling V4.1 Flash DeepSeek's "new flagship," ranking ahead of V4 Pro 0813.

Terminal-Bench is a benchmark evaluating AI agents' ability to execute tasks in real terminal environments, covering file operations, process management, network configuration, and more, requiring models to autonomously call command-line tools to complete multi-step operations, scored by task completion rate. DeepSWE series benchmarks focus on software engineering scenarios, typically including locating defects in real codebases, generating patches, and validating them through test suites, scored by whether the patch passes hidden tests. NL2Repo-Bench measures the ability to generate runnable code repositories from natural language specifications, Automation-Bench focuses on browser and desktop automation, and Agents' Last Exam is a comprehensive evaluation set for long-horizon agent tasks. The common feature of these benchmarks is: the higher the score, the stronger the model's reliability in completing end-to-end engineering tasks without human intervention, making them closer to the real performance of Agent pipelines than simple code completion metrics.

Cost Estimate for One Session

Since no public test provides per-task token counts for the two models, here's an estimate based on explicit assumptions: an Agent bug-fixing session, 40 model calls, totaling 1.2 million input tokens (90% cache hits), 60,000 output tokens (including thinking). Both models use the same token counts, calculated at AIHubMix list prices: Flash 1017 peak $0.118, off-peak $0.059; Pro 0813 peak $0.469, off-peak $0.233.

Pro's cost per session is about four times Flash's. To break even, Pro would need to complete the same task with roughly one-quarter of Flash's token volume. But the only public verbosity measurement points in the opposite direction: Artificial Analysis found September Flash used 1.62× the output tokens of Pro 0813 on its index, meaning Pro needs only about 62% of the token volume, not 25%.

Cache hit refers to input tokens in a request that duplicate previous conversation being reused via server-side caching, billed at a price far below regular input. In Agent sessions, system prompts, tool definitions, and historical conversation often account for the vast majority of input, so cache hit rate directly determines the actual bill. The 90% cache hit assumption in this article is precisely the typical form of Agent scenarios; if your pipeline injects fresh context every round, cache benefits shrink dramatically, and the cost multiplier between the two models changes accordingly. When evaluating a switch, you should also record the proportion of cache-hit tokens, otherwise rough estimates based on list input prices will systematically overestimate costs.

How to Validate the Switch on Your Own Workload

Prerequisites

You need an AIHubMix API key (exported as AIHUBMIX_API_KEY), the OpenAI Python SDK installed (pip install openai), and pointing to https://aihubmix.com/v1. The two model IDs are deepseek-v4.1-flash-1017 and deepseek-v4-pro-0813, and one key covers both. Of course, the most important prerequisite is an evaluation set from your real workload and a pass/fail judgment you already trust.

Run Both Models with the Same Prompt

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["AIHUBMIX_API_KEY"], base_url="https://aihubmix.com/v1")

for model in ("deepseek-v4-pro-0813", "deepseek-v4.1-flash-1017"):
    resp = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": "Find the bug: def mean(xs): return sum(xs) / len(xs) - 1"}],
        max_tokens=16000,
    )
    print(model, resp.usage.completion_tokens, resp.choices[0].message.content[:200])

Each line outputs the model ID, completion_tokens, and the first 200 characters of the answer. Replace the single prompt with your evaluation set, and record three things for each model: pass rate, completion_tokens per prompt, and cost per run read from AIHubMix request logs rather than estimated from list prices. 1017 was trained for shorter outputs, but no one has measured it against Pro yet, so your logs will be the first measurement applicable to your workload.

completion_tokens is the usage field returned by the API, representing the number of tokens the model actually generated this time, including the thinking process (if the model outputs reasoning content) but excluding input. It is crucial to the switching decision: a model with a higher pass rate but verbose output may actually have a higher cost per task. max_tokens limits the maximum generation length of a single reply; setting it too small truncates long answers and lowers pass rates, while setting it too large may let verbose models consume more tokens. When comparing two models on the same evaluation set, fix max_tokens and record completion_tokens for each call, then combine with unit price to calculate cost per run, rather than just looking at pass rate figures.

Several Failure Modes That Distort Comparisons

Flash prints more tokens and looks more expensive. Token counts alone don't determine cost. At the prices above, Pro only breaks even when using about one-quarter of Flash's token volume. Multiply by unit price before comparing.

Two runs of the same evaluation can double in cost. Both models are half price during off-peak hours. If one model's run falls during peak and the other during off-peak, the cost gap halves or doubles. Run both back-to-back within the same price window.

Image use cases can't be scored on Pro. Pro doesn't accept image input at all; 1017 does. Split screenshot, chart, and UI-related use cases out of the shared evaluation set. If your workflow includes any of these tasks, the answer is already self-evident.

Your scientific QA workload has no 1017 score. GPQA Diamond is the only benchmark where Pro (92.4) beats September Flash (90.9), and DeepSeek hasn't published 1017's score on it. If scientific Q&A is your primary workload, test with your own questions before migrating.

What Hasn't Been Verified Yet

All the benchmark numbers above come from DeepSeek itself, with no third-party reproduction. The Pro column in the paper doesn't specify the 0813 build; its Terminal-Bench 2.1 score matches the 0813 card, so it's interpreted as 0813 here. 1017's output length relative to Pro hasn't been measured; Artificial Analysis's 62% figure is for September Flash. Per-task token counts for both models are also not public; the session cost above is an estimate based on assumptions, not measured.

Decision Rules

If you have a production pipeline validated on Pro 0813 and no time to rerun evaluations this month, or if the GPQA gap in your scientific QA workload has already shown up in your own tests, then keep Pro.

For coding Agents, terminal automation, tool-calling pipelines, any task involving images, and any high-traffic scenario where concurrency matters, switch to Flash.

For long-form writing and analysis where you've relied on Pro's shorter, denser answers, test before migrating.

There's no deadline forcing migration: DeepSeek's changelog says V4 Pro will continue to be available, with no retirement date announced. AIHubMix's comparison page shows both models' specs and prices side by side while you run your evaluation.

Key Takeaways

• DeepSeek's official benchmarks show V4.1 Flash 1017 beating V4 Pro 0813 on all published coding and Agent metrics, while output pricing is only about one-third of the latter's.
• In a hypothetical Agent session estimate, Pro costs about four times Flash; Pro would need to complete the same task with about one-quarter the token volume to break even, and existing verbosity data doesn't support this.
• Before switching, you must measure pass rate, output token count, and cost per run on your own evaluation set, rather than relying on official numbers.
• Image input, concurrency cap (2500 versus 500), and high-concurrency cost scenarios are Flash's decisive advantages; scientific QA (GPQA) is Pro's only possible remaining stronghold.
• All benchmarks are DeepSeek's own measurements, with no third-party reproduction; 1017's output length and per-task token counts remain unknown.

One-line Summary

DeepSeek's cheaper Flash beats Pro across official benchmarks, but switching requires validation on your own workload.