Unverified60% confidenceFactExact time
LFM2.5-24B-A2B uses a 40-layer architecture: 30 double-gated short convolution blocks + 10 GQA attention layers, 64 experts per MoE block, top-4 routing, a 131,072 token context, and a 128K vocabulary.
2
Sources
60%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedLlama 3 8B has 32 layers, 8 KV heads (GQA architecture with 32 query heads), a head dimension of 128, and a KV cache of approximately 1.07 GB for a single sequence under FP16 with an 8192 context length.69% similarUnverifiedQwen3.8-Flash-Next's MoE layers use 512 experts with Top-10 routing, a native context window of 262K, and can scale up to 1 million tokens.68% similarUnverifiedIn the LFM2.5 series, text models besides 24B also include 230M, 350M, 1.2B (including Instruct, Thinking, and JP versions), 2.6B, and 8B-A1B MoE; 24B is the only model that enters the 30B-class MoE race.68% similarUnverifiedEach of Qwen3-27B's 16 full-attention layers has 4 KV heads and a head dimension of 256, occupying 64 KB per token at 16-bit precision.67% similarUnverifiedThe LFM 2.6B thinking variant achieves 12/16 substantive at approximately 97 tokens/s.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1029962API
curl https://kongchang.com/api/v1/knowledge/claims/1029962MCP
get_claim(id=1029962)