Ascend 950PR Single-Card DeepSeek V4-Flash Inference Test: 208 TPS, Now Open Source

Running DeepSeek V4-Flash on a Single Card with 128GB VRAM

Domestic AI inference hardware is advancing rapidly. According to hands-on test results shared by a Bilibili content creator, Huawei's Ascend 950PR successfully ran a trimmed version of DeepSeek^Ad V4-Flash (ds-v4-flash) on a single card, achieving a peak decode throughput of 208 TPS. The related code has also been open-sourced — a noteworthy signal for developers following the progress of domestic AI compute hardware.

Device information confirmed via npu-smi shows the card as an Ascend 950PR with 128GB VRAM. The test focused on validating inference capabilities using the MXFP4 and MXFP8 low-precision formats on the Ascend platform — both of which are key approaches for reducing the cost and increasing the efficiency of large model inference.

It's worth noting that the creator originally planned to use a VLM (Vision-Language Model) solution, but abandoned it due to high integration complexity, ultimately opting for an SGLang-based inference stack. This kind of engineering trade-off is very common in real-world deployments — theoretically superior solutions are often impractical due to ecosystem maturity constraints.

---

MXFP4 and MXFP8 are "Microscaling Floating Point" formats defined by the OCP (Open Compute Project) specification, and represent a frontier direction in large model inference quantization. Unlike traditional INT4/INT8 quantization, MX formats share a single scaling factor across a group of weights (typically 16), preserving a better numerical representation range at very low bit counts. This allows them to reduce VRAM usage and memory bandwidth demands while maintaining inference precision. MXFP4 compresses each parameter to 4 bits, while MXFP8 retains 8 bits; both formats have been designated as natively supported targets by NVIDIA Blackwell and certain domestic chips. The Ascend platform's validation of these two formats indicates that its hardware instruction set and driver stack now have the corresponding support capabilities — which is practically meaningful for deployment scenarios prioritizing low-cost, high-throughput inference.

VRAM Allocation and Context Length Testing

The creator provided relatively clear data on VRAM planning. After launching the inference service, approximately 109GB of the 128GB VRAM is used for full model weights, with the remaining ~11GB available for KV Cache, supporting approximately 800K context length. When extended with additional system memory, peak single-request decode throughput reaches the aforementioned 208 TPS.

In more extreme scenarios, the creator reported even more aggressive numbers: at approximately 1.6M ultra-long context with 128 concurrent requests, throughput can reach around 3,700 (presumably aggregate tokens/s). These figures reflect the trade-off space in the trimmed model between long context and high concurrency — by sacrificing some model scale, you gain VRAM headroom and greater service capacity.

It should be noted that these results come from a single source's preliminary testing in a short-term trial environment obtained through a promotional event (the creator mentioned receiving hundreds of hours of 950PR access time). These are not rigorous standardized benchmarks and should be treated as directional reference points rather than definitive conclusions.

---

KV Cache (Key-Value Cache) is the VRAM region in Transformer inference used to store the historical key and value tensors from the attention mechanism. Its size directly determines the maximum context length the model can handle. KV Cache VRAM usage scales proportionally with sequence length, number of layers, number of attention heads, and precision format. Therefore, when weights already occupy the majority of VRAM, how the remaining space is allocated has an enormous impact on the usable context window. The ratio of approximately 11GB KV Cache for 800K context suggests that this trimmed model has reduced KV dimensions at the architecture level — otherwise, full-scale DeepSeek V3/V4 series models would typically struggle to support such long contexts within the same VRAM budget. Extending CPU RAM as a KV Cache overflow area is a common engineering technique, at the cost of increased access latency, making it suitable for batch-processing scenarios that prioritize throughput over single-request latency.

Letting the Model Handle Its Own Hardware Adaptation

One interesting highlight of this test was the creator's use of npu-top related tools combined with Pin Agent, allowing the local model to autonomously complete the adaptation work for the 950PR. According to the description, after enabling acceleration (approximately 6x speedup), the model completed all adaptation tasks in a single pass within about 6 minutes and successfully passed the npu-top validation tests.

This closed loop of "using a model to adapt hardware, then using that hardware to run the model" demonstrates the potential of current Agent toolchains for engineering automation. Having a model autonomously handle tedious environment adaptation can significantly lower the barrier to entry for domestic hardware — which has been one of the historically criticized pain points of the Ascend ecosystem.

---

Pin Agent is a class of automated engineering agents based on large language models. The core idea is to describe tedious tasks — such as environment configuration, driver adaptation, and parameter tuning — as natural language instructions or structured tool calls, allowing the model to autonomously plan and execute a series of command-line operations to ultimately complete and verify the target. In domestic AI chip scenarios, the value of such tools is especially pronounced: the software ecosystem documentation for platforms like Ascend is fragmented, interfaces change frequently, and manual adaptation often requires extensive trial and error. By having an Agent automatically read error logs, search documentation, generate fix scripts, and iteratively verify results, what would otherwise take hours or days of adaptation work can be compressed to minutes. This is also a concrete example of the emerging "model-driven DevOps" paradigm.

Remaining Issues and Next Steps

The creator also candidly acknowledged the limitations of this test. The main shortcoming is that the trimmed model cannot run with reasoning intensity set to MAX, as doing so easily leads to inference loops. This indicates that model trimming, while reducing size and improving throughput, inevitably sacrifices some reasoning stability — particularly affecting long chain-of-thought capabilities.

Additionally, the creator mentioned currently attempting to run higher-spec models on the 950PR, but the speed currently falls short of the Pro7000 platform, and deep optimization has not yet been performed. In other words, the 208 TPS result was achieved without thorough tuning, meaning there is still room to squeeze out further performance.

The code has been open-sourced, and the creator indicated a more detailed benchmark evaluation is planned. For teams looking to validate large model inference solutions on domestic compute hardware, this kind of open-source practice provides a reproducible starting point — well worth following for future optimization progress.

Summary

Successfully running a trimmed DeepSeek V4-Flash on a single Ascend 950PR card and open-sourcing the work represents a pragmatic milestone in the domestic inference hardware ecosystem. 128GB VRAM, 200+ TPS single-card decode throughput, million-token-scale concurrent context handling, combined with an Agent-driven approach to hardware adaptation — together these sketch out a snapshot of domestic compute hardware's transition from "can run" to "actually usable." That said, the reasoning intensity limitations introduced by model trimming, and the as-yet-unoptimized state of the implementation, are reminders to maintain measured expectations about these preliminary figures.