Unverified50% confidenceFactExact time
Tiny-MoE模型总参数量约200M，但每个token激活的参数仅约90M
1
Sources
50%
Confidence
Long-term
Relevance
7/13/2026
First Seen
Sources
15岁少年从零训练200M MoE语言模型：完整架构与实战拆解
redditr/learnmachinelearning7/12/2026
Related Claims
Unverified对于 198B 总参数的稀疏 MoE 模型，实际激活参数量可能仅为 20-40B 量级83% similarUnverifiedAnalysis speculates that the model is an MoE model with 2 to 4 trillion parameters, activating only approximately 150 billion parameters per token.74% similarUnverified轻量小模型通常指参数量在1亿以下的BERT系列模型，如DistilBERT、TinyBERT71% similarUnverifiedA model with 100 billion parameters requires approximately 80 GB of VRAM to process a single inference request, while a model with tens of billions of parameters requires only about 20 GB.70% similarUnverifiedThe model context length currently invoked by Cursor is only 200,000 tokens.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/501380API
curl https://kongchang.com/api/v1/knowledge/claims/501380MCP
get_claim(id=501380)