Unverified50% confidenceFactTime unknown
In the Gemma 4 26B MOE model, the model contains 128 internal 'experts' but only 8 are activated per inference.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
Valid until: 9/30/2026
Sources
Related Claims
VerifiedGemma 4的26B MOE模型采用混合专家架构，拥有260亿总参数但任意时刻仅约40亿参数处于激活状态71% similarUnverified在MoE模型中每个token只激活少量Expert（通常2-4个，Top-K路由），实际计算量可能仅相当于6-7B的Dense模型68% similarUnverifiedMoE模型中'35B'指总参数量，每个token只激活少量Expert（通常2-4个），实际计算量可能仅相当于6-7B的Dense模型68% similarUnverifiedKimi K2.7 Code采用384个专家的MoE架构，每个token选择8个专家加1个共享专家，实际计算量约为稠密模型的2.3%67% similarUnverifiedQwen3.6 35B模型包含256个专家，每个Token仅激活8个路由专家和1个共享专家，实际推理时激活参数约为3B67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/57177API
curl https://kongchang.com/api/v1/knowledge/claims/57177MCP
get_claim(id=57177)