Unverified50% confidenceFactExact time
LLaMA 2的32000词表在4096维隐层下嵌入层需占用约1.3亿参数
1
Sources
50%
Confidence
Long-term
Relevance
7/13/2026
First Seen
Sources
不到350元训练亿参数模型：SmoLLM从零构建全记录
redditr/learnmachinelearning7/12/2026
Related Claims
VerifiedQwen3-235B总参数量为2350亿，采用MoE架构，每次前向推理实际激活的参数量约22B65% similarUnverifiednomic-embed-text嵌入模型有137M参数，all-MiniLM-L6-v2有22M参数，BGE-small为24M-109M参数65% similarUnverifiedLFM2.5 has 8.3 billion total parameters but only 1.5 billion parameters activated per inference64% similarUnverifiedHy3采用混合专家（MoE）架构，总参数量达2950亿（295B），每次推理仅激活210亿（21B）参数63% similarUnverifiedxAI的Grok-1采用3140亿总参数、每次激活约25%专家的设计62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/502356API
curl https://kongchang.com/api/v1/knowledge/claims/502356MCP
get_claim(id=502356)