NPUs Can Pretrain Too: VKORE Hits 170% of H100 Throughput on Domestic Chips

NPUs have long been positioned as "inference-only chips," with training workloads still dependent on Nvidia GPUs. But VKORE, a hardware-agnostic acceleration layer, is breaking down this division of labor. Measured data shows that NPUs equipped with VKORE achieve 170% of a single H100's pretraining throughput and 75% of a single B200's, while inference performance improves 3.8x. This means that in sovereign compute scenarios, NPUs can independently complete the full "build-serve-improve" loop.

Why sovereign compute suddenly matters

Access to top-tier Nvidia GPUs is no longer a given. In the Chinese market, domestic companies are being guided to reduce purchases of Nvidia accelerators, with domestic chips from Huawei, Cambricon, Alibaba, and Baidu filling the gap. Supply chain, pricing, and policy risks exist in other regions as well.

This creates a dilemma for operators building systems on non-Nvidia NPUs: inference is fine, but what about training? The honest answer used to be "training still requires GPUs." VKORE aims to eliminate that dependency.

Architectural differences between NPUs and GPUs

To understand why NPUs have long been confined to inference, you first need to see how their architecture diverges from GPUs. GPUs (graphics processing units) were originally designed for graphics rendering, with a large number of general-purpose compute cores that can be flexibly scheduled through programming models like CUDA, making them naturally suited to the frequent matrix operations and gradient backpropagation in training. NPUs (neural processing units) are specialized chips custom-built for neural network inference, typically using systolic arrays or TPU-like dataflow architectures. They are highly optimized for fixed operators such as convolution and matrix multiplication, with outstanding energy efficiency, but their programming models are closed and support for dynamic computation graphs is limited. Training workloads require backpropagation, optimizer state maintenance, mixed-precision scheduling, and other complex control flows, which traditional NPUs struggle to handle. VKORE's value lies precisely in filling this capability gap with a software layer, enabling NPUs to execute training workloads without changing their silicon design.

What is VKORE

VKORE is a software acceleration layer for NPUs. If CUDA is the key that makes GPUs programmable for deep learning, VKORE plays the same role for NPUs. The chip itself doesn't change—only the software layer does. It is designed to be hardware-agnostic, allowing data centers to mix GPUs and NPUs while still training and serving inference from the same stack.

The core value of this approach lies in decoupling: compute procurement is no longer locked to a single vendor, and operators can flexibly allocate heterogeneous hardware based on cost, supply, and policy conditions.

CUDA's moat and the significance of a hardware-agnostic layer

CUDA is the parallel computing platform and programming model Nvidia has built since 2006, enabling developers to use languages like C/C++ to call GPUs for general-purpose computing. It has gradually accumulated a complete ecosystem including cuDNN, NCCL, and TensorRT. This ecosystem constitutes Nvidia's deepest moat: hardware performance can be caught up to, but the extremely high migration cost of the software stack makes it hard for customers to switch vendors. The "hardware-agnostic acceleration layer" approach that VKORE represents is precisely about rebuilding a CUDA-like abstraction layer on NPUs—providing a unified training and inference interface upward, while adapting to different vendors' NPU instruction sets downward. Once this abstraction layer matures, data centers can swap accelerators as easily as they swap storage or network cards, without rewriting model code, thereby truly breaking single-vendor lock-in.

Pretraining results

The core conclusion is that pretraining can now run on NPUs, not just inference.

• Single-device pretraining throughput reaches 170% of a single H100.
• The same workload reaches about 75% of a single B200.
• Numerical behavior is consistent with standard training—it is real training, not an approximation.

Put plainly: a sovereign NPU that most industry insiders regard as "inference-only" surpasses the H100 in pretraining throughput and approaches the level of the current flagship B200.

How to interpret throughput benchmarks

The figure "pretraining throughput reaches 170% of a single H100" needs to be understood in context. Throughput is typically measured in tokens or samples processed per second, and is jointly affected by batch size, sequence length, model scale, precision format (FP16/FP8/BF16), and interconnect bandwidth. Single-device comparisons exclude multi-card communication overhead, thus reflecting the collaborative efficiency of the chip and software stack more purely. An NPU surpassing the H100 in a specific configuration may benefit from more aggressive operator fusion, higher on-chip memory bandwidth utilization, or a dedicated dataflow tailored to that workload. But "170%" is not a universal conclusion—the ratio may change with a different model or hyperparameters. Reaching 75% against the B200 indicates the NPU has entered the same order of magnitude as the current flagship, though a gap remains. Readers should treat it as a measured value under a specific configuration, not a universal performance promise.

Inference performance was already strong

Before its training capability, VKORE had already improved inference serving on the same NPU hardware:

• Single-user response speed improved 3.8x.
• Server throughput improved 4.1x.
• Per-token latency remains stable under high concurrency, with no failed requests.

The same chip serves faster during the day and can still train at night.

Data center economics change as a result

Competition in AI data centers has shifted from "buy more GPUs" to "get more intelligence from the same power and the same chips." If NPU clusters can both serve and pretrain, operators no longer need to build separate GPU clusters for training. For teams built on sovereign chips, this means the full loop of building, serving, and improving does not require Nvidia on the critical path.

Combined with a decision engine that can make judgments in a single forward pass, the night-shift loop becomes concretely actionable: serve inference during the day, score the day's traffic, and pretrain or fine-tune on the same batch of NPUs at night.

The "night-shift loop" and the decision engine

The "night-shift loop" mentioned in the article refers to a time-multiplexed data center operating model: during the day, NPU clusters handle online inference services, generating real traffic and feedback data; at night, the same batch of hardware switches to pretraining or fine-tuning tasks, updating the model with the day's data. The key prerequisite for this model is a decision engine that can make judgments in a single forward pass—it can instantly score each piece of traffic and filter high-value samples, preventing nighttime training from being diluted by noisy data. Compared with building separate GPU clusters for training, time multiplexing raises hardware utilization from "inference hours" to nearly all day, significantly amortizing cost per unit of compute. For sovereign compute scenarios, this means that without changing power, facility, or procurement budgets, operators can simultaneously gain serving capability and model iteration capability, forming a self-cycle of "build-serve-improve."

FAQ

Does NPU pretraining lose precision? No. In our measurements, numerical behavior is consistent with standard training.

Is 170% a typo? An NPU beat the H100? This is single-device pretraining throughput with VKORE in our configuration, using the H100 as the 100% baseline. Against the B200, the NPU reaches about 75%.

Does VKORE lock you into a particular NPU vendor? No. It is a hardware-agnostic layer for mixed GPU and NPU data centers.

Can it be used only for inference? Yes. Inference acceleration (3.8x single-user, 4.1x throughput) runs independently of the training path.

One-line Summary

VKORE, a hardware-agnostic acceleration layer, gives NPUs pretraining capability with throughput reaching 170% of H100 and 3.8x inference improvement, breaking the division of labor that training must depend on Nvidia GPUs.

Paragraph Summary

VKORE is a hardware-agnostic software acceleration layer for NPUs that enables them—originally positioned as "inference-only"—to perform pretraining. Measurements show that NPUs equipped with VKORE achieve 170% of a single H100's and 75% of a single B200's single-device pretraining throughput, with numerical behavior consistent with standard training rather than an approximation. On the inference side, it had already delivered 3.8x single-user response and 4.1x server throughput improvements while maintaining stable latency under high concurrency. This capability frees operators in sovereign compute scenarios from building separate GPU clusters for training, allowing the same batch of NPUs to serve inference during the day and pretrain at night, forming a complete "build-serve-improve" loop and reducing dependence on Nvidia.