Unverified50% confidenceFactExact time
Aleph Alpha's Kolibri 1 is a MoE reasoning model focused on German and English, in which each of the 50 layers routes each token to 6 of 384 experts via sigmoid routing, plus an additional shared expert.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/15/2026
First Seen
Valid until: 1/13/2027
Sources
Related Entities
Related Claims
UnverifiedA typical MoE configuration such as Mixtral 8x7B has 8 experts per layer, and each token is routed to only 2 of them, with the parameter count activated in a single pass being about one third of the total (plus shared weights).66% similarUnverified在MoE模型中每个token只激活少量Expert（通常2-4个，Top-K路由），实际计算量可能仅相当于6-7B的Dense模型63% similarUnverifiedMoE routers typically employ a Top-K mechanism that selects only the K highest-scoring experts for each token, with K=2 being a common configuration.58% similarVerifiedKimi K2.6 adopts a Mixture of Experts (MoE) architecture, with a total parameter count at the trillion level and only 32B parameters actually activated.58% similarUnverifiedLFM2.5-24B-A2B has 24 billion total parameters and uses an MoE architecture with 64 experts per block, Top-4 dynamic routing, and only 2.45 billion parameters activated per token.57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1039064API
curl https://kongchang.com/api/v1/knowledge/claims/1039064MCP
get_claim(id=1039064)