Unverified50% confidenceFactExact time
LLaMA系列采用过度训练策略，以7B参数训练了1T token，远超Chinchilla最优配比的140B token
1
Sources
50%
Confidence
Long-term
Relevance
9/11/2026
First Seen
Sources
Related Entities
Related Claims
Unverified根据Chinchilla缩放定律，一个30B模型的最优训练数据量约为600B-1T tokens69% similarUnverified700亿参数的Chinchilla模型在1.4万亿token上训练，性能超过了2800亿参数但仅用3000亿token训练的Gopher63% similarUnverifiedLoRA使可训练参数量通常仅为全参微调的0.1%-1%，可在单张RTX 3090/4090（显存24GB）上微调7B乃至13B级别的模型63% similarUnverified训练7B参数模型的全参数微调通常需要至少4张A100（80G）显卡，而LoRA使单张RTX 4090（24G）完成7B量级模型微调成为可能61% similarUnverified叠加cv=5的5折交叉验证后，144种参数组合需训练144×5=720次模型58% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/893366API
curl https://kongchang.com/api/v1/knowledge/claims/893366MCP
get_claim(id=893366)