VKORE: A Software Acceleration Layer That Lets NPUs Train Large Models Too

When NPUs Are No Longer Just "Inference-Only"

In AI data centers, NPUs have long been viewed as inference accelerators—vendors positioned them that way, and developers accepted the premise. The VKORE software acceleration layer, released by Korean AI startup VIDRAFT (비드래프트), aims to change that perception: without swapping chips or touching hardware, software-layer optimization alone can achieve high-throughput inference on NPUs—and for the first time, make Transformer training (validated via LoRA fine-tuning) feasible on these chips.

This positioning naturally invites comparison to the relationship between CUDA and NVIDIA GPUs—a foundational software layer that truly unlocks hardware compute. For developers working on ML infrastructure and LLMOps, this is worth paying attention to.

How NPUs Differ from GPUs and CPUs

An NPU (Neural Processing Unit) is a class of accelerators purpose-built for tensor operations like matrix multiply-add. Typical examples include Google's TPU, Apple's Neural Engine, and various vendors' edge/data-center AI chips. Compared to GPUs, NPU instruction sets and on-chip memory hierarchies are more closely aligned with the fixed dataflow of neural network inference, typically delivering higher inference energy efficiency per unit of power. However, their programming models are closed and operator libraries are limited, so they have long been considered unsuitable for training workloads that require backpropagation and dynamic graph scheduling. GPUs, by contrast, cover both training and inference thanks to the programmable CUDA ecosystem. Thus, "NPUs can only do inference" is more a result of software ecosystem limitations than a hard physical constraint of the silicon—and this is precisely the gap VKORE seeks to exploit.

How VKORE Works

VKORE is essentially an optimized execution layer sitting between ML frameworks and NPU hardware. By analogy, it functions like a custom kernel/runtime that schedules and batches operations in ways the default NPU driver does not.

The most intuitive comparison comes from a concrete number: for a given LoRA training task, if it takes about 1 hour on an NVIDIA B200 GPU, it would take roughly 320 hours on a server CPU—and a naively driven NPU performs even worse than that CPU baseline. VKORE restores much of that efficiency by restructuring how operations are dispatched to the NPU, without modifying the underlying silicon.

VIDRAFT envisions an operational model of a day-night cycle: during the day, NPU clusters serve real-time inference traffic; at night, the same NPU clusters, driven by VKORE, train on the day's accumulated judge logs and feedback signals. The goal of this cyclical architecture is to reduce reliance on additional GPU purchases and to support data center environments where GPUs and NPUs coexist.

Why LoRA fine-tuning is an ideal validation scenario

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method: it freezes all original weights of a pretrained model and trains only low-rank matrix pairs (A, B) attached beside certain layers, typically reducing trainable parameters to roughly one-thousandth of full fine-tuning. This yields two NPU-friendly properties: first, backpropagation covers only a tiny subset of parameters, greatly reducing memory overhead for gradients and optimizer states; second, the bulk of forward and backward computation remains standard matrix multiplication, which can theoretically reuse existing NPU inference operators. Even so, LoRA training still requires a complete training stack including automatic differentiation, gradient accumulation, and optimizer updates—which is why "running LoRA on an NPU" is often regarded as the minimum threshold validation of whether the hardware has training capability.

Benchmarks and Real-World Performance

All data comes from AI타임스 reporting; the following are the core metrics.

VKORE inference and training improvements (same NPU, VKORE vs. baseline NPU):

• Single-request response latency: 3.8x faster
• Server-level concurrent throughput: 4.1x higher
• LoRA Transformer training: successfully completed (previously infeasible on NPU)

Darwin-27B-ZTC-v5 (judge model) — S1MB global benchmark:

• Ranked 1st out of 152 models on the public S1MB benchmark
• 9.72 points higher than the official JEV reference model
• In 137 head-to-head benchmark comparisons: 137 wins, 97 losses
• Rankings of competitors on the same leaderboard: Perplexity (2nd), Cloudflare (17th), Meta (27th)

Darwin-27B-ZTC adopts a zero-token-cost (ZTC) inference design: it requires no output token generation and completes judgments via a single forward pass. For high-frequency operations like response quality scoring and task routing (millions of calls per day are common in production-grade LLM systems), this design offers clear energy-efficiency advantages.

RSI inference engine — math benchmarks:

• AIME 2026: 100% (perfect score)
• HMMT 2026: 100% (perfect score)—reported as the first perfect score on the certified math leaderboard
• GPQA Diamond (scientific reasoning): ranked 1st among models under 500B parameters

VIDRAFT states that full performance results will be released in the form of a research paper.

Reward models, cross-encoders, and zero-token-cost inference

A reward model is a component in the RLHF pipeline used to score model outputs, typically taking a "prompt + response" as input and producing a scalar score. A cross-encoder, meanwhile, concatenates a query and candidate documents and encodes them together, using full attention interaction to judge relevance. Both require a single forward pass over the complete input rather than autoregressive generation. Darwin-27B-ZTC's "zero-token-cost" design belongs to this category: it does not generate text token by token, but only produces a judgment at the end of the forward pass, thereby eliminating KV Cache growth and the decoding loop—making the compute and latency of a single call far lower than generative models. For high-frequency scenarios such as scoring, routing, and quality assessment called millions of times per day, the energy-efficiency gains of this design scale linearly with call volume.

How to Access It and Its Limitations

As of the report, VKORE, Darwin-27B-ZTC-v5, and the RSI engine are all not publicly available—there is no Hugging Face repository, GitHub codebase, or public API. VIDRAFT announced the technical results and said a paper is forthcoming, but disclosed no public access channel.

A key external question about VKORE is hardware compatibility. The source did not specify which NPU vendors or chip families are supported. VIDRAFT's CEO said the goal is to run on Korean domestic NPUs, but the supported hardware list has not been made public. This is also one of the core questions the paper needs to answer.

Another technical detail is worth noting: Darwin-27B-ZTC's single-forward-pass judgment mechanism is architecturally similar to reward models or cross-encoders in the RLHF pipeline, but is specifically optimized for high-throughput, low-latency production-grade LLM routing and quality assessment scenarios.

VIDRAFT is participating in a government-backed AI foundation model project (under the MSIT/NIPA framework) as part of the NAVER Cloud alliance, which may influence future release timelines.

Why the generality of software-layer acceleration is the biggest variable

NPU software stacks are typically provided by chip vendors in the form of drivers, compilers, and operator libraries, with upper-layer frameworks (PyTorch, TensorFlow, etc.) connecting through vendor-defined backends. The problem is that instruction sets, memory hierarchies, and operator coverage vary enormously across NPUs—a scheduling strategy hand-tuned for one specific chip may completely fail on another. As an optimized execution layer between frameworks and hardware, VKORE's value ceiling depends on how many chip-agnostic general optimizations it can abstract, and how much chip-specific adaptation must be written for each NPU. If the adaptation cost approaches rewriting the driver, the business model is hard to scale; if it can provide a portable intermediate representation and automatic scheduling, it could become the "quasi-CUDA" of the NPU camp. This is both a question the paper needs to answer and the key to judging its long-term value.

Why This Matters

If NPU training capability is unlocked by a software layer, the implications for data centers go beyond saving GPU procurement budget. A hybrid architecture where GPUs and NPUs coexist would make resource scheduling for inference and training more flexible—a cycle of running inference by day and fine-tuning by night, in a pure-GPU setup, means the same hardware must switch between two radically different workloads, whereas a heterogeneous NPU+GPU combination may make that switching smoother.

It should be noted that all current performance data comes from a single source (AI타임스 reporting) and lacks independent third-party verification. VKORE's claimed training speedup multiples and Darwin-27B-ZTC's benchmark rankings will need to be confirmed after the paper is released and more developers conduct hands-on tests. Whether the software layer can generalize across multiple NPU architectures, rather than being effective only for specific chips, is the key variable determining the actual value of this technology.