DeepSeek Open-Sources DeepGEMM-Ascend: Can Ascend Challenge the CUDA Ecosystem?

The Real Problem Behind a Matrix Multiplication Migration

Migrating a matrix multiplication from an NVIDIA GPU to a Huawei Ascend chip looks like a simple hardware swap on the surface — but in practice, it involves a complete engineering effort around how software arranges data and routes numbers into compute units. Both chips can do multiplication, so why is switching so hard? The core reason: data needs different arrangements and different pathways into working memory.

On September 30, 2026, DeepSeek^Ad released the Ascend version of DeepGEMM (DeepGEMM-Ascend), providing a familiar interface and corresponding mechanisms for operations that were previously unsupported. This is the right starting point for understanding the release — it doesn't complete the entire migration for you, but at the lowest level of matrix kernels, it shoulders a significant amount of the layout conversion and data movement engineering on behalf of developers.

The math of matrix multiplication itself hasn't changed: a row of blue data meets a column of orange data, corresponding elements are multiplied and summed, producing one position in the output grid, repeated across the entire grid. What changes is the machinery surrounding that math — how data is moved through memory, how it's arranged, and how it's fed into the compute units.

What the Kernel Does: Memory Hierarchy and Fractal Layout

The specialized code that handles these operations is called a "kernel." The Ascend kernel moves matrix tiles from global memory into L1 cache, then into L0 cache for computation. Think of it as a large storage rack (global memory) and smaller work trays close to the compute units (caches) — there's always a trade-off between capacity and access efficiency: the rack holds more data, but where data lives matters when it's time to compute.

During loading into L1 cache, the arrangement of tiles changes — the code uses what's called a "fractal layout" for indexing. The arithmetic doesn't change, but the "seating chart" does. Even when the goal looks identical at the application layer, moving operations between different chips requires genuine engineering investment: you're asking for the same result, but the implementation must be organized according to how that particular chip wants to work.

Low-precision computation also involves scaling factors — values that set numerical scale — think of them as labels attached to tiles. Ascend expects these labels in specific packed formats and layouts, so migrating also means preparing this auxiliary data. Conversion helper functions in the library pack scaling factors in pairs into 16-bit integer containers, arranged in what the code calls MN-major layout. If a tensor is already in the correct format, the helper can be reused directly; otherwise a conversion is triggered — even this preparation step requires decision-making.

Fractal layout is a multi-level nested data arrangement unique to Ascend AI Cores. Unlike the row-major or column-major linear memory layouts common on NVIDIA GPUs, fractal layout partitions a matrix into tiles, then recursively sub-tiles each level, arranged according to the access patterns of the hardware compute units (the Cube Core matrix compute units). This "tiles within tiles" structure ensures that hardware reading sequential memory can maximize L0 cache bandwidth, avoiding latency from strided access. The trade-off: any standard contiguous layout data coming from outside (such as a PyTorch tensor) must first go through a conversion before it can be fed into the compute units — a conversion step that itself requires extra memory copies and alignment operations, making it an unavoidable engineering hurdle in cross-chip migration.

Reuse and Pipelines: Hidden Performance Mechanisms

Back to the work-tray perspective: when a blue tile stays in place while a new orange tile arrives, operand reuse allows inputs to be retained for use by another piece of work, without fetching them again for the next computation. Keeping inputs nearby reduces repeated transfers. The source doesn't provide specific speedup numbers, but it's clear which redundant data movements are being avoided.

Add a "second work tray": while one set of tiles is feeding data to computation, another stage can simultaneously prepare incoming data, using staged buffers and asynchronous requests to overlap these activities. The whole workbench starts to look like a production line — that's the "pipeline." But overlapping has prerequisites: computation can't use data that hasn't arrived yet, and when one stage depends on another, the parallel stages need to synchronize — like having a "ready signal" on each work tray.

These memory hierarchies, operand reuse, and pipeline overlapping together form the source of kernel performance — and precisely the part of engineering that DeepGEMM-Ascend encapsulates for developers.

Coverage and Explicit Gaps

From this perspective, DeepSeek's contribution is easier to characterize. The original DeepGEMM is a CUDA kernel library, and the Ascend version provides a relatively familiar interface for Ascend, handling the layout and data movement described above behind the scenes.

In terms of numerical formats, the library supports BF16 (16-bit), FP8 (8-bit), and FP4 (4-bit). It also covers specific operations such as MQA Logit (handling attention scores in multi-query attention) and MegaMode (serving mixture-of-experts MoE models).

But supporting these operations doesn't mean an entire application has been migrated — coverage has concrete gaps. Despite the repository claiming full API compatibility, the implementation explicitly marks the grouped matrix multiplication (grouped GEMM) interface as "not implemented." A familiar entry point is helpful, but you still need to check which operations your application actually calls, one by one.

Environment requirements are also hard prerequisites: according to the repository, development and validation are based on the Ascend 950 series, requiring CANN 9.20 and Torch NPU. These requirements come before any performance discussion — if it won't run, speed is irrelevant.

Grouped GEMM refers to executing multiple independent small matrix multiplications in a single call, where each group of matrices can have different dimensions and produces independent results. This operation is especially critical in mixture-of-experts (MoE) architectures: when a model routes different tokens to different experts, each expert receives a different number of tokens, forming a batch of matrices with varying sizes. Processing them all at once with Grouped GEMM can dramatically reduce kernel scheduling overhead. The original CUDA version of DeepGEMM provides this interface, and DeepSeek's own MoE models depend on it. The Ascend version marks this interface as "not implemented," meaning developers using MoE architectures — including DeepSeek's own V3/R1 series — face a substantial gap before a complete migration is possible.

CANN vs. CUDA: The Real Ecosystem Comparison

A familiar function name does a lot of "PR work": calling code gets a recognizable entry point, which is helpful — but once inside, data preparation, supported operations, and required software still determine whether an application can actually run.

CUDA is not just kernels — it encompasses compilers, runtime and driver interfaces, math libraries, debugging tools, and profilers. Matrix kernels are only a small fraction of this platform, and after replacing them, migrating an application still requires considering the entire surrounding software stack. Huawei's CANN provides the broader platform for Ascend, likewise including compilers, runtime interfaces, operator and communication libraries, and profiling tools — which is why CANN is the more appropriate comparison to CUDA, while DeepGEMM-Ascend is simply one piece within this broader environment, providing specific available operations for applications.

NVIDIA's years of accumulated ecosystem — application porting, partner libraries, language integrations, deployment tools — is the fundamental reason switching requires engineering investment. But these sources can't tell you how much work a specific team will face: if the reusable library covers the operations you need, you may be able to use its implementation directly rather than writing the mechanisms yourself for that chip.

CANN (Compute Architecture for Neural Networks) is Huawei's heterogeneous computing architecture software stack for Ascend AI processors, positioned as the counterpart to NVIDIA's CUDA platform. CANN includes an operator development framework (TBE/DSL), a graph compilation engine (ATC/GE), a runtime (ACL), a communication library (HCCL, corresponding to NCCL), and a model conversion and tuning toolchain. Operators written by developers using CANN are described in TBE (Tensor Boost Engine), compiled into native Ascend instructions by a TVM-derived compiler. The main gap relative to the CUDA ecosystem lies not in the compute capability of individual kernels, but in CANN's third-party library coverage, the maturity of debugging tools, and the depth of integration with mainstream deep learning frameworks like PyTorch and JAX — areas where the CUDA ecosystem has accumulated over a decade of development, and where much of the migration friction originates.

How to Read Performance Numbers, and What Developers Should Do

On performance, DeepSeek reports selected dense matrix multiplication results approaching what they claim as the hardware ceiling, measured on Ascend 950D with CANN 9.20 using L2 performance analysis. To be clear: these are DeepSeek's own numbers, not independently reproduced measurements, and they represent selected kernel results under specific conditions. They do not establish parity or a win over NVIDIA systems, nor do they represent end-to-end model throughput.

For developers considering Ascend, a reasonable path forward is:

• Map your operations: List the operations your application needs and compare them against the library's coverage one by one, paying particular attention to interfaces explicitly marked as not implemented (such as grouped GEMM).
• Check layouts and environment: Confirm that your expected data layouts and software version requirements (CANN 9.20, Torch NPU, Ascend 950 series) are all satisfied.
• Measure your real workload: Benchmark tables cannot substitute for integrated measurements you run inside your application, together with the surrounding software. You need to know how it actually performs under conditions that make the comparison meaningful.

Reusable engineering is the most promising part of this release: layout conversion, operand reuse, coordinated loading — these mechanisms don't need to become separate projects every time. A well-maintained library can handle these low-level mechanisms for applications that fall within its coverage. But broader adoption still depends on three things: coverage, reliable tooling, and measured real-world performance.

The conclusion is measured: DeepGEMM-Ascend is worth evaluating for the operations it supports — it can shoulder part of the migration work for you. But which parts you still need to handle yourself depends on your specific application. It does not establish a universal software layer applicable to all chip vendors; the goal of this release is explicit and limited to Huawei Ascend.