#MoE
28 related articles
Colibri Pure-C Engine Goes Viral: Running a 744B Model on 25GB of RAM
Colibri is a zero-dependency pure-C inference engine that runs 744B MoE models on 25GB RAM using three-tier VRAM/RAM/NVMe scheduling. Supports GLM, Kimi, DeepSeek and more.
Dense or MoE? How to Choose the Right AI Model Architecture
Dense vs MoE: what's the difference and when should you use each? This guide breaks down sparse vs full parameter activation to help you choose the right AI architecture.
Dense Models vs MoE: Active Parameters, Throughput, and a Practical Selection Guide
A deep dive into Dense vs MoE models: active parameters, total parameters, and throughput explained. Includes a practical selection guide using NVIDIA Nemotron 3.5 Lightning as a case study.
Accelerating Dropless MoE Training in JAX: A Deep Dive into NVIDIA Transformer Engine
Learn how NVIDIA Transformer Engine accelerates Dropless MoE training in JAX, covering MoE architecture, token dropping, FP8 precision, and grouped GEMM optimizations.
DeepSeek-V4.1-Flash Lands on Fireworks: 552B MoE Model Targets Coding and Agents
DeepSeek-V4.1-Flash, a 552B MoE model built for coding, cybersecurity, and agents, is now live on Fireworks — claimed to outperform Opus 5 and GPT-5.6 at 1/40th the cost.
Expert OpinionsRethinking Scaling Laws: Parameters Are Not the Only Answer
Deep analysis of Scaling Law evolution from Kaplan to Chinchilla to the MoE era, exploring why blindly stacking parameters is a mistake, and how GLM-5.3 proves scaling has multiple knobs.
Deep DivesNVIDIA Dynamo Multi-Turn Agentic Interaction: Deep Integration of Streaming Tokens and Tool Calling
Deep dive into NVIDIA Dynamo's multi-turn agentic interaction support, covering streaming token output, structured tool calling, state management, and MoE synergy for production-grade AI agents.
Deep DivesMicrosoft Tutel: A Deep Dive into the MoE Model Acceleration Library with FP4/FP8 Low-Precision Support
A deep dive into Microsoft's open-source Tutel MoE optimization library, supporting FP8, NVFP4, and MXFP4 multi-precision computation for DeepSeek, Kimi-K2, Qwen3, and other leading MoE models.