#MoE

397 related articles

Jul 23, 2026·3 min

Qwen3.8 Deep Dive: How Does a 2.4 Trillion Parameter MoE Model Claim Second Place?

Alibaba's Qwen3.8 challenges larger models with a 2.4T-parameter MoE architecture, claiming second only to Gemini. A deep dive into MoE mechanics, continuous updates, two-speed release strategy, and real local deployment requirements.

Jul 22, 2026·2 min

Agent A1 Open Source: 35B MoE Local Inference Speed Test & Deployment Guide

Agent A1 is a 35B MoE open-source model from Shanghai's Intern Science, activating only 3B params. Hits 95 tok/s on M4 Mac Studio. Supports 256K context, tool calling, and free local deployment via Ollama.

Jul 21, 2026·2 min

MiniMax M3 Local Deployment: 23B Active Parameters ≠ 23GB VRAM — A Complete Guide to MoE Architecture

MiniMax M3 is a 428B MoE model. Its 23B active parameters reflect per-token compute, not VRAM needs. Learn the MoE trade-offs, quantization options, and deployment paths to avoid the most common misconception.

Jul 17, 2026·2 min

Training a CPU-Native LLM on Zero Budget: The Architectural Gamble of Ternary Quantization × Sparse MoE

An indie dev attempts to train a CPU-native LLM on $0 budget using ternary quantization, sparsity, and fine-grained MoE — with pre-registered success criteria and full public reporting.

Jul 13, 2026·2 min

15-Year-Old Trains 200M MoE Language Model from Scratch: Full Architecture & Hands-On Breakdown

A 15-year-old trained Tiny-MoE, a 200M-parameter MoE language model from scratch using free Kaggle GPUs, featuring MLA attention, RoPE+YaRN, and native PyTorch.

Jul 11, 2026·2 min

Deep Dive into Tencent Hunyuan Hy3: How a 295B MoE Model Cracks the Agent Deployment Challenge

Tencent Hunyuan Hy3 launches with a 295B MoE architecture activating just 21B params and 256K context. Hallucination cut from 12.5% to 5.4%, MRCR nearly doubled, with MTP and EAGLE decoding and Day-0 SGLang support.

Jul 10, 2026·2 min

Tencent Hunyuan V3 Open-Sourced: Full Breakdown of the 295B MoE Architecture — 256K Context Is Its Biggest Weakness

Tencent open-sources Hunyuan V3: 295B MoE model with 21B active params, claims to beat DeepSeek-V4-Pro. Supports FP8, vLLM/SGLang — but 256K context and no multimodal are key drawbacks.

Step 3.7 Flash: Deep Dive into the 198B Sparse MoE Multimodal Model

Tech Frontiers

May 30, 2026·2 min

Step 3.7 Flash: Deep Dive into the 198B Sparse MoE Multimodal Model

Deep dive into StepFun AI's Step 3.7 Flash, a 198B sparse MoE vision-language model with 256K context and 3-level reasoning, excelling in multimodal understanding, AI coding, and Agent tool orchestration.

LFM2.5-8B-A1B: A MoE Model with 1.5B Active Parameters Delivering 4x Its Weight Class Performance

Tech Frontiers

May 30, 2026·2 min

LFM2.5-8B-A1B: A MoE Model with 1.5B Active Parameters Delivering 4x Its Weight Class Performance

Liquid AI releases LFM2.5-8B-A1B, a MoE model with 8B total params but only 1.5B active, matching 6B-class models in tool calling. Supports 128K context, local deployment, multilingual, with SGLang Day-0 support.

Kimi K2.5 Fully Open-Sourced: Deep Dive into 1T Parameter MoE Architecture + Agent Cluster Capabilities

Tech Frontiers

May 27, 2026·2 min

Kimi K2.5 Fully Open-Sourced: Deep Dive into 1T Parameter MoE Architecture + Agent Cluster Capabilities

Deep dive into Moonshot AI's fully open-sourced Kimi K2.5: 1T parameter MoE architecture, Vision-to-Code capabilities, and 100-Agent parallel cluster system topping open-source benchmarks.

Wukong 2.2P Open Source: 35B MOE Model Outperforms Qwen3.6-27B at 3–5x the Speed

Tech Frontiers

May 20, 2026·1 min

Wukong 2.2P Open Source: 35B MOE Model Outperforms Qwen3.6-27B at 3–5x the Speed

Wukong 2.2P 35B MOE model is now open source. Using adversarial hybrid distillation, it outperforms Qwen3.6-27B. Runs at 158 tokens/s on RTX 4090 with only 8.9GB VRAM. Supports 256K context.

Microsoft Tutel: A Deep Dive into the MoE Model Acceleration Library with FP4/FP8 Low-Precision Support

Deep Dives

May 13, 2026·3 min

Microsoft Tutel: A Deep Dive into the MoE Model Acceleration Library with FP4/FP8 Low-Precision Support

A deep dive into Microsoft's open-source Tutel MoE optimization library, supporting FP8, NVFP4, and MXFP4 multi-precision computation for DeepSeek, Kimi-K2, Qwen3, and other leading MoE models.

Jul 27, 2026·3 min

Poolside Laguna Model Launch: Can Silicon Valley's Open-Source Push Catch Up to China's AI?

Poolside launches Laguna open-weight model after 18 months of silence, pitting 118B parameters against Kimi K3's 2.8 trillion. Can Silicon Valley's open-source push close the gap with Chinese AI?

Jul 27, 2026·3 min

Claude Opus 5 Officially Released: Performance Approaching the Top Tier as the Open-Weight Debate Heats Up Again

Anthropic releases Claude Opus 5 with near-frontier performance at lower prices. Same day, Jensen Huang co-signs open-weight letter with 20+ companies while DeepSeek fundraising rumors surface.

Jul 27, 2026·3 min

Kimi K3 Launch: Performance Rivaling Claude at One-Third the Price, US-China AI Gap Narrows to 3 Months

Moonshot AI launches Kimi K3 reasoning model with performance rivaling Claude and OpenAI's top models at one-third the price. The US-China AI gap narrows from 6-12 months to just 3 months.

Jul 27, 2026·3 min

DeepSeek Ignites Open Source vs. Closed Source Debate in Silicon Valley, Splitting Tech Giants into Rival Camps

DeepSeek's open source model shakes Silicon Valley. OpenAI defends closed source while Microsoft, NVIDIA, and Meta back open ecosystems. Analysis of the AI open/closed source debate, Apple-Micron chip tensions, and AI-driven historical disinformation.

Jul 27, 2026·3 min

Open Source vs. Closed Source: Who Will Win the US-China AI Battle?

Chinese open-source models DeepSeek and Kimi K3 are challenging OpenAI's closed-source dominance. Analyzing the business logic, chip ecosystems, and US-China strategic dynamics behind the open vs. closed AI debate.

Jul 27, 2026·3 min

Fields Medal Winner Joins OpenAI: A Triple Signal of Talent, Capital, and Capability

Fields Medal winner Jacob Tsimerman joins OpenAI's safety team on award day, declaring math careers won't survive. Meanwhile, NVIDIA finances a $250B data center and Kimi K3 open-sources 2.8T parameters.

Jul 27, 2026·3 min

Fields Medal Winner Joins OpenAI: A Triple Signal of Talent, Capital, and Capability

Fields Medal winner Jacob Tsimerman joins OpenAI's safety team on award day, saying math careers won't survive. NVIDIA finances a $250B data center. Kimi K3 opens a 2.8T-parameter model.

Jul 27, 2026·3 min

Kimi K3 Launches on Ollama Cloud: Free Tier, User Experience, and Developer Guide

Kimi K3 officially launches on Ollama Cloud as an "extra high usage" model. This guide covers free tier quotas, cloud inference experience, technical advantages, and how developers can seamlessly call this high-performance LLM.