Unverified50% confidenceFactExact time
Taking GPT-3's attention layers as an example, the original weight matrix is approximately 12,288×12,288 (about 150 million parameters), while a LoRA decomposition with rank r=8 requires training only about 200,000 parameters, a compression ratio exceeding 700×.
1
Sources
50%
Confidence
Long-term
Relevance
10/11/2026
First Seen
Sources
Related Entities
Related Claims
Unverified以 GPT-3 为例，r=4 时 LoRA 可将可训练参数从 175B 降至约 4.7M，减少约 37,000 倍80% similarUnverified以GPT-3的1750亿参数为例，大约2/3的参数分布在FFN层中，1/3在注意力层中73% similarUnverifiedLoRA使可训练参数量通常仅为全参微调的0.1%-1%，可在单张RTX 3090/4090（显存24GB）上微调7B乃至13B级别的模型70% similarUnverifiedGPT-3 reached a parameter scale of hundreds of billions (175 billion parameters) and demonstrated few-shot and zero-shot capabilities.67% similarUnverifiedThe mini GPT model has a final parameter count of 13,693,824, which falls within the target range of 10–30 million.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/998068API
curl https://kongchang.com/api/v1/knowledge/claims/998068MCP
get_claim(id=998068)