Unverified50% confidenceBenchmarkExact time
LoRA reports on GPT-3 175B: compared with full fine-tuning using Adam, trainable parameters are reduced by about 10,000 times, and GPU memory during training is reduced by about 3 times.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
Unverified对LLaMA-7B所有注意力层应用r=8的LoRA，可训练参数量从70亿降至约400万，压缩比超过1750倍73% similarUnverifiedLoRA可以将可训练参数量减少99%以上72% similarUnverifiedExperiments conducted by the research team on models such as GPT-2 showed that after only 5–9 generations of recursive training, the quality of model outputs degrades significantly.72% similarUnverified一个拥有10,000块GPU的训练集群，假设单块GPU年故障率为3%，平均每天约有一块GPU出现问题71% similarUnverifiedMoE架构每次推理仅激活少数专家，训练成本相对同等能力稠密模型大幅降低，10B激活1B相比10B稠密模型训练成本低10倍且推理更快71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1047748API
curl https://kongchang.com/api/v1/knowledge/claims/1047748MCP
get_claim(id=1047748)