Training Large Models Without GPUs: Backpropagation Runs on Inference-Only NPUs

Using a chip that excels only at "inference" to train a model sounds like asking a sprinter to run a marathon—the hardware design simply doesn't support it. But VIDRAFT's recently released VKORE-46M-NPU proves that this path is engineering-feasible.

The Core Event: A 46M-Parameter Model Completes Pretraining on an Inference NPU

VIDRAFT has released a small language model called VKORE-46M-NPU, with 46 million parameters, pretrained entirely from random weights without using any GPU. The hardware platform is a first-generation inference-only NPU—according to conventional machine learning engineering wisdom, such chips are only suitable for forward inference, not training.

The model was trained on public datasets and can generate short English stories. The weights, model card, and comparative analysis materials between NPU and GPU training have all been published on Hugging Face.

The significance of this release lies not in the model's own capabilities, but in the fact that it is a verifiable engineering proof: backpropagation can run on inference-only chips. The VKORE software layer that VIDRAFT is building aims to become "CUDA for NPUs," covering three capabilities: inference acceleration, full pretraining, and LoRA fine-tuning. This release is the publicly verified output of the pretraining milestone.

Engineering Challenges: Why Inference NPUs Can't Train Directly

Inference NPU architectures are heavily optimized for forward propagation, but they largely lack the capabilities required for backpropagation. Gradient computation and weight updates demand flexible memory access patterns, high-precision intermediate buffers, and in-place tensor modifications—features that are default configurations in GPU training but often missing or restricted on inference NPUs.

VIDRAFT's approach is to build a software abstraction layer on top of the NPU. Since inference-only chips typically lack vendor-provided training stacks, they implement backpropagation themselves through the VKORE layer, making the NPU behave as a trainable compute substrate.

Training starts from random initialization without using any pretrained checkpoints. This point is crucial—it rules out the possibility of "actually fine-tuning but packaging it as pretraining." To verify correctness, VIDRAFT also ran the exact same training task on a high-performance GPU, comparing the loss trajectories of both under matched conditions.

The specific implementation details (operator fusion strategies, memory management, gradient accumulation mechanisms) have not yet been disclosed and will be described in a follow-up paper.

Core Hardware Requirements for Backpropagation

The hardware bottleneck for backpropagation is not compute power, but memory access and precision. In forward propagation, each layer's activations can be discarded once used; in backpropagation, each layer's activations must be retained until gradient backpropagation is complete, so memory usage scales with network depth. Meanwhile, weight updates require gradients to be accumulated onto parameters at high precision, typically requiring FP32-precision gradient accumulation buffers—yet inference NPUs' on-chip SRAM is often optimized for INT8/FP16 forward inference, and FP32 buffers may simply have nowhere to live. Additionally, gradient descent involves a three-step serial operation of "read weights—compute gradients—update weights in place," heavily dependent on in-place updates and flexible random memory access, whereas inference NPUs' DMA engines and schedulers typically assume data flow is unidirectional and immutable. These architectural assumptions all fail in training scenarios, which is why a backpropagation abstraction must be rebuilt on top of the chip.

Key Metric: What Does a 0.99 Loss Trajectory Correlation Mean?

Between NPU training and GPU training, the loss trajectory correlation coefficient reaches 0.99. This number needs to be understood correctly.

It measures the similarity of training dynamics, not throughput equivalence. VIDRAFT explicitly states that this does not mean NPU training speed can match GPU—inference NPUs are typically slower for training. What 0.99 means is: the training process on the NPU converges along a statistically nearly identical path to the GPU baseline, and gradient computation is numerically consistent, not approximate or divergent.

Additionally, VIDRAFT verified that general public models can perform inference and quantized model execution on the same NPU, indicating that the VKORE layer covers both training and deployment ends.

Statistical Meaning of the Loss Trajectory Correlation Coefficient

The loss trajectory correlation coefficient typically refers to the Pearson correlation coefficient, measuring the degree of linear correlation between two loss curves along the training step dimension. 0.99 means the two curves move almost completely in the same direction—the ratio relationship between the NPU's per-step loss decrease and the GPU baseline is highly stable. However, the correlation coefficient does not reflect absolute differences: even if the two curves always differ by a fixed offset, the correlation coefficient can still approach 1. Therefore, VIDRAFT's comparison more likely used step-aligned loss value sequences to verify the numerical consistency of gradient computation, rather than final alignment of generalization performance. What truly needs to be ruled out are training failure modes such as gradient divergence, gradient vanishing, or excessive gradient variance—0.99 indicates these risks are eliminated at the numerical level.

How to Obtain and Use

The model weights, model card, and comparative analysis materials are publicly available on Hugging Face:

pip install huggingface_hub
huggingface-cli download vidraft/VKORE-46M-NPU

Note that the repository slug is subject to VIDRAFT's Hugging Face page. The core training code is not open-sourced and is temporarily withheld for intellectual property protection; a technical paper is planned for subsequent release. No public API endpoint for this model has been mentioned.

Several Questions That Need Clarification

Is this真正的 reverse-mode backpropagation? VIDRAFT explicitly states in its press release that it implemented backpropagation (역전파) on the NPU, described as standard reverse-mode gradient computation, not a gradient-free alternative. The technical mechanism is undisclosed.

Does a 0.99 correlation mean NPU training is as fast as GPU? No. This coefficient measures the similarity of loss curves as they evolve with steps, unrelated to wall-clock throughput or sample efficiency. The significance of the result lies in correctness verification, not speed benchmarking.

When will LoRA fine-tuning and broader model pretraining be supported? LoRA fine-tuning is a roadmap item for the VKORE software layer, alongside inference acceleration and pretraining. No public timeline is available.

Can I train on my own NPU hardware? Not for now. The training stack is internal; only the trained model weights and analysis materials are public. Keep an eye on the follow-up technical paper and the release of the VKORE SDK.

Industry Significance of This Event

For ML engineers, the value of this case is: when you are forced to consider non-GPU accelerator options, inference-only NPUs are at least no longer a completely infeasible training platform. The 0.99 loss correlation coefficient demonstrates that numerical correctness can be guaranteed, even if speed is not advantageous.

Of course, the limitations are equally obvious. The model is only 46M parameters, the training data is public datasets, the training code is not open-sourced, and speed metrics are undisclosed. How many engineering obstacles lie between 46M and larger-scale models cannot be judged at present. Whether the VKORE software layer can form an ecosystem like CUDA also depends on subsequent LoRA support and the degree of SDK openness.

VIDRAFT's release is more of a starting point than an endgame. Its value lies not in proving that "NPUs can train large models," but in proving that "backpropagation on inference NPUs is numerically credible"—this is the precondition for all subsequent work.

The Analogy Boundary Between NPU Training Stacks and the CUDA Ecosystem

Comparing VKORE to "CUDA for NPUs," the key difference is that CUDA's success is built on the synergy of four layers: hardware instruction set, compiler, runtime libraries, and framework adaptation, with NVIDIA having complete control over the hardware. A third-party software layer aiming to implement training abstraction across multiple NPU hardware platforms must contend with incompatible operator sets, memory layouts, and driver interfaces from each chip vendor. VIDRAFT has currently achieved pretraining on a single first-generation inference NPU, proving single-point feasibility, but it remains three dimensions away from a CUDA-style ecosystem: hardware coverage breadth, operator library completeness, and framework integration depth. Whether the follow-up paper discloses operator fusion and memory management strategies will determine whether this solution is a custom engineering effort or a reusable general-purpose approach.

One-line Summary

VIDRAFT pretrained a 46M-parameter model from scratch on an inference-only NPU, achieving a 0.99 loss trajectory correlation with GPU.

Paragraph Summary

VIDRAFT has released VKORE-46M-NPU, a 46-million-parameter small language model pretrained entirely from random weights without any GPU, on a first-generation inference-only NPU. Inference NPU architectures are optimized for forward propagation and typically lack the flexible memory access, high-precision buffers, and in-place tensor modifications required for backpropagation. VIDRAFT implemented backpropagation through a self-built VKORE software layer and ran the same task on a GPU as a control, achieving a loss trajectory correlation coefficient of 0.99, indicating numerically consistent gradient computation. This number measures training dynamics similarity, not speed equivalence; NPU training is typically slower. Model weights and comparison materials are public on Hugging Face, but training code is not open-sourced, and there is no timeline for LoRA fine-tuning or the SDK.