RKLLM
A dedicated large language model inference framework provided by Rockchip for its NPU platforms. It supports NPU-accelerated inference of autoregressive Transformer language models, and models must first be converted through the official toolchain before they can be loaded and executed.
Timeline (last 90 days)
RKLLM is a dedicated large language model inference framework that Rockchip provides for its NPU platform; model weights must be converted through the official toolchain and cannot load GGUF or PyTorch formats directly.
The language model uses the W8A8 quantized version in RKLLM format; the vision encoder uses the RKNN format.
All Facts (2)
RKLLM is a dedicated large language model inference framework that Rockchip provides for its NPU platform; model weights must be converted through the official toolchain and cannot load GGUF or PyTorch formats directly.
50%UnverifiedThe language model uses the W8A8 quantized version in RKLLM format; the vision encoder uses the RKNN format.
50%