Unverified50% confidenceOpinionExact time
In the 120B-A12B model, the total model weights are 120 billion parameters, but only about 12 billion parameters actually participate in computation during each token's forward pass.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/15/2026
First Seen
Valid until: 1/13/2027
Sources
Related Entities
Related Claims
UnverifiedA model with 100 billion parameters requires approximately 80 GB of VRAM to process a single inference request, while a model with tens of billions of parameters requires only about 20 GB.77% similarUnverifiedA model with 11,000 parameters occupies less than 50KB of memory.75% similarVerifiedIf a MoE model activates only approximately 1/8 of its parameters per inference, the actual computational overhead may be equivalent to that of a dense model with tens of billions of parameters.71% similarUnverifiedThe model places n-gram lookup tables at layers 1 and 14, with 24 hashes each indexing approximately 16 million rows, totaling 196 billion parameters, exceeding one-third of the model's weights.69% similarUnverified根据DeepMind在2022年提出的Chinchilla缩放定律，一个700亿参数的模型需要约1.4万亿token才能达到计算最优69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1037572API
curl https://kongchang.com/api/v1/knowledge/claims/1037572MCP
get_claim(id=1037572)