During the Think 2025 conference, IBM released a preview of the Granite 4 model, which adopts a Mixture of Experts (MoE) architecture with parameter sizes ranging from 3 billion to 20 billion, and the smallest version requires only 15 GB of memory to run a 120,000-token context window.