Anifer
An open-source local large language model inference engine written in C++ and CUDA by an independent developer. It focuses on extreme performance optimization for a small number of specific models and hardware configurations, with high inference speed as its primary design goal.
Timeline (last 90 days)
Anifer is an inference engine open-sourced on GitHub, written in C++ and CUDA by an independent developer, focused on extracting extreme performance for a small number of models and graphics cards.
The official Anifer project initially only supported a single RTX 5090, and only five official models were registered, focusing on Qwen3.8 27B, Qwen3.6 27B, and Qwen3.6 35B MoE models.
The performance of CUDA Kernels is extremely dependent on alignment; slight mismatches in matrix dimensions, thread block sizes, or memory access patterns will cause throughput to drop significantly, and because Anifer only serves fixed model shapes and GPU architectures, each Kernel can be written in exactly the optimal form.
Anifer's maximum concurrency limit is 8.
Official benchmark results show that the Qwen3.6 27B dense model with NVFP4 quantization achieves 202 token/s at single concurrency, 400 token/s at dual concurrency, and 1147 token/s at 8 concurrency.
Official benchmark results show that the Qwen3.6 35B MoE model can reach 1.38k token/s at 8 concurrency.
Anifer supports MTP (Multi-Token Prediction) speculative decoding with an acceptance rate between 46% and 49%, and also supports D Flash speculative decoding but with higher VRAM usage.
Anifer uses a single-file format called .enifer, which packages model weights, chat template, and multimodal content into one file and cannot directly load standard GGUF files.
The official Anifer project only supports RTX 5090 and Linux or WSL/Docker systems, but community forks have added support for the 4090, RTX 2080Ti, and other 20-series GPUs, as well as a native Windows version.
Anifer's core trade-off is sacrificing generality for extreme speed, making it a near-perfect solution for users who have a 5090 and simply want to run mainstream Qwen models as fast as possible.
All Facts (10)
Anifer is an inference engine open-sourced on GitHub, written in C++ and CUDA by an independent developer, focused on extracting extreme performance for a small number of models and graphics cards.
50%UnverifiedThe official Anifer project initially only supported a single RTX 5090, and only five official models were registered, focusing on Qwen3.8 27B, Qwen3.6 27B, and Qwen3.6 35B MoE models.
50%UnverifiedThe performance of CUDA Kernels is extremely dependent on alignment; slight mismatches in matrix dimensions, thread block sizes, or memory access patterns will cause throughput to drop significantly, and because Anifer only serves fixed model shapes and GPU architectures, each Kernel can be written in exactly the optimal form.
50%UnverifiedAnifer's maximum concurrency limit is 8.
50%UnverifiedOfficial benchmark results show that the Qwen3.6 27B dense model with NVFP4 quantization achieves 202 token/s at single concurrency, 400 token/s at dual concurrency, and 1147 token/s at 8 concurrency.
50%UnverifiedOfficial benchmark results show that the Qwen3.6 35B MoE model can reach 1.38k token/s at 8 concurrency.
50%UnverifiedAnifer supports MTP (Multi-Token Prediction) speculative decoding with an acceptance rate between 46% and 49%, and also supports D Flash speculative decoding but with higher VRAM usage.
50%UnverifiedAnifer uses a single-file format called .enifer, which packages model weights, chat template, and multimodal content into one file and cannot directly load standard GGUF files.
50%UnverifiedThe official Anifer project only supports RTX 5090 and Linux or WSL/Docker systems, but community forks have added support for the 4090, RTX 2080Ti, and other 20-series GPUs, as well as a native Windows version.
50%UnverifiedAnifer's core trade-off is sacrificing generality for extreme speed, making it a near-perfect solution for users who have a 5090 and simply want to run mainstream Qwen models as fast as possible.
50%