Unverified50% confidenceFactExact time
Taking GPT-3 as an example, when d=12,288 and r=8, each low-rank decomposition matrix has about 197,000 trainable parameters, while the full update matrix has about 151 million parameters, a difference of nearly three orders of magnitude.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedTaking a 1000×1000 weight matrix as an example, if the LoRA rank is set to 8, only approximately 16,000 parameters need to be trained instead of the original 1,000,000 parameters.77% similarUnverifiedLoRA reports on GPT-3 175B: compared with full fine-tuning using Adam, trainable parameters are reduced by about 10,000 times, and GPU memory during training is reduced by about 3 times.71% similarUnverifiedAmong the 12 model × task combinations, only gpt-5.4-mini showed a statistically significant accuracy improvement on the logical reasoning task: jumping from 15% to 97.5%, with p=0.00056 after Holm correction.70% similarUnverifiedFrom GPT-2 to GPT-3 the parameter count increased by approximately 100 times, but the improvement on many benchmarks was far less than 100-fold.70% similarUnverifiedThe 29-million-parameter encoder trained by the researchers achieved a popularity-weighted cross-fold AUC of only 0.486 on 985 statements, indistinguishable from the corpus prior.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1047744API
curl https://kongchang.com/api/v1/knowledge/claims/1047744MCP
get_claim(id=1047744)