DeepSeek V4.1 Fusion Operators Demystified: How Ascend 910 Pushes Hardware Performance to the Limit

A neural network is fundamentally a stack of operators, and operator performance largely determines the efficiency of the entire network. On high-performance hardware like the Ascend 910, writing high-performance fusion operators is the key to unlocking the chip's full potential. This article, based on a technical talk shared on Bilibili, uses the DeepSeek V4.1 network as a case study to break down the design philosophy behind fusion operators, the challenges developers face, and how Agent-assisted development combined with a DSL offers a way forward.

Why Fusion Operators?

Taking the Attention layer in DeepSeek^Ad V4.1 as an example, the overall flow can be divided into three stages: pre-processing, Attention computation, and post-processing. Each stage contains a large number of small operators. The problem is that these small operators tend to achieve poor hardware utilization in practice — once a single operator finishes, hardware resources sit idle, and intermediate results must be repeatedly read from and written to memory.

A natural solution follows: if small operators working independently are inefficient, why not merge several of them into one large fusion operator? The benefits of fusion are obvious, making it a core tool for building high-performance networks. Complex fusion operators like FlashAttention are a direct product of this thinking.

Pre- and Post-processing Can Be Fused Too

Beyond the main Attention computation, the small operators in the pre- and post-processing stages also have room for fusion. Looking at portions of the DeepSeek V4.1 scripts, the Index pre-processing step involves a serial flow of Cube computation (white boxes) and Vector computation (red boxes). Through fusion operators, this serial arrangement can be transformed into CV parallelism, improving hardware utilization.

Additionally, consecutive Vector computations can be fused to reuse on-chip data, reducing the overhead of reading and writing intermediate results. These two types of optimization — compute unit parallelism and on-chip data reuse — are the two primary levers through which fusion operators improve performance.

FlashAttention is a landmark achievement in the fusion operator space, proposed by Dao et al. in 2022. Its core idea is to merge multiple independent operators — matrix multiplication, Softmax, Dropout, and others — from standard Attention computation into a single operator, and to use a tiling strategy to keep intermediate results in SRAM (on-chip high-speed cache), avoiding frequent round-trips to HBM (High Bandwidth Memory). Standard Attention has O(N²) memory read/write complexity; FlashAttention reduces this to O(N), with particularly dramatic speedups in long-sequence scenarios. This design philosophy — reducing memory transfers and increasing on-chip data reuse — is the core source of performance gains from fusion operators, and represents the fundamental methodology for operator optimization on hardware like Ascend.

Ascend NPU compute units fall into two categories: Cube units handle high-parallelism operations like GEMM (matrix multiplication) and are the workhorse for large-scale tensor computation; Vector units handle element-wise operations such as activation functions, normalization, and quantization. The two are physically independent and can execute in parallel. In the unfused small-operator model, Cube and Vector computations are typically serialized — Cube runs, then waits for Vector, then Vector runs, then waits for Cube — meaning one type of compute unit is always idle. Fusion operators solve this by carefully scheduling the pipeline so that Cube and Vector run simultaneously (CV parallelism), theoretically doubling hardware utilization. This is the underlying reason why fusion operators achieve significant speedups over combinations of small operators on the Ascend platform.

The Development Challenges of Fusion Operators

Given how clear the benefits are, why have networks historically only had a handful of fusion operators like FlashAttention? The answer lies in several difficult barriers to development.

Model churn is the first barrier. Model architectures evolve relatively quickly, while fusion operators are tightly coupled to network structure and are highly customized. Pre- and post-processing operators in particular are affected by factors like cache management and quantization modes — they are essentially different for every network, making reuse nearly impossible.

The mismatch between development timelines and model iteration pace is the second barrier. Fusion operators take longer to develop than small operators, while model iteration cycles keep getting shorter — from years or half-years in the past, to months or even weeks today. A common frustration is finishing a fusion operator for one model only to find that the new model has completely changed its architecture.

Development complexity itself is a challenge. Fusion operators involve many computation stages, intricate synchronization dependencies, and complex code logic, making them inherently difficult to build.

Two Core Pain Points in Agent-Based Development

In the era of rapidly advancing AI agents, many of these challenges have been partially alleviated — using agents to generate operator code has become a new possibility. But in practice, two significant pain points remain when agents are used for operator development.

Synchronization bugs are frequent and hard to diagnose. Because fusion operators have complex overall computation logic, cross-core synchronization pipelines are very long, with large gaps between synchronization points. Synchronization-related bugs account for a high proportion of issues in agent-generated fusion operators, and they are extremely difficult to track down.

Tuning lacks stability. For an agent, each round of performance tuning typically involves large-scale code changes. Anyone who has worked with agent-generated code will recognize this: large diffs, poor code stability, and as a result, no significant improvement in tuning efficiency. In other words, the agent can get code running, but it struggles to converge reliably to a high-performance state.

Campbot DSL: An Agent-Friendly Solution

To address these development challenges, the DeepSeek V4.1 fusion operator development process introduced an agent-friendly Campbot DSL as the solution.

The core idea is to define clear interaction boundaries between the agent and the software framework: domain knowledge that is complex and error-prone but does not directly affect performance is captured through modeling and embedded in the Synth3C layer, establishing a stable software abstraction. This frees the agent from tedious low-level details, allowing it to focus exclusively on performance-related decisions and generate complex, high-performance operators.

For developers or agents, the development process only requires focusing on algorithm design itself, significantly lowering the barrier to building high-performance operators while improving development efficiency. In essence, this approach hands off the "error-prone but low-value" parts to the framework, while leaving the "truly performance-critical" decisions to the agent and human developer.

Measured Results and Open Source

Using the Campbot DSL and leveraging model-specific characteristics, the team rapidly developed 8 fusion operators for the DeepSeek V4.1 network, including index, prolog, kingram, git, and others. Compared to serialized small-operator execution, all of these fusion operators achieved significant performance gains — the talk presented performance comparison data for 5 pre- and post-processing fusion operators against their small-operator counterparts.

The relevant operator code has been open-sourced in the Campbot DSL repository in the CANN community. Interested developers are welcome to explore the implementation details further.

DSL (Domain-Specific Language) is a programming language designed for a particular application domain, as opposed to general-purpose languages like C++ or Python. In operator development, the value of a DSL lies in encapsulating low-level complexity — hardware architecture details, synchronization primitives, memory management — as higher-level abstractions, allowing developers (or agents) to express computation logic in terms closer to algorithm descriptions, while a compiler or runtime framework handles the mapping to specific hardware instructions. The Synth3C layer mentioned in Campbot DSL plays exactly this role — it formalizes error-prone logic like cross-core synchronization and pipeline scheduling into rules, narrowing the agent's output space to performance-relevant parameters. This fundamentally reduces the probability of synchronization bugs occurring and makes each tuning round's code changes more localized and controlled.

Conclusion

Fusion operators have long been a key technique for extracting hardware performance, but their high customization requirements, long development cycles, and complex synchronization logic make them feel cumbersome in an era of rapid model iteration. The value of the Campbot DSL lies in using a stable software abstraction to free agents from error-prone low-level details, letting intelligent agents focus on performance decisions. This may represent a viable path forward for operator development: humans, agents, and frameworks each playing their respective roles to collaboratively produce high-performance code.