Unverified50% confidenceTradeoffExact time
LoRA's low-rank assumption constitutes a capacity ceiling, which limits the model's expressive power when the feature transformations required for a new modality cannot be effectively captured by low-rank matrices.
1
Sources
50%
Confidence
Long-term
Relevance
10/5/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedDuring LoRA fine-tuning, the original model parameters are frozen and only the inserted low-rank matrices are updated, significantly reducing VRAM requirements and training costs.73% similarVerifiedLoRA decomposes the weight update matrix ΔW into the product of two small matrices A and B whose ranks are much lower than the original matrix, reducing parameter count to as little as one ten-thousandth of the original or even less.71% similarVerifiedSmaller models are more prone to embedding collapse than larger models, due to reasons including limited parameter capacity, a tendency toward optimization shortcuts, and amplified anisotropy issues.71% similarUnverifiedFine-tuning on data with narrow defects can sometimes cause models to become broadly misaligned.69% similarUnverifiedUnder fixed computing resources, too many parameters with insufficient data prevents the model from learning adequately, while massive data with insufficient parameters means the model lacks the capacity to capture complex patterns69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/975913API
curl https://kongchang.com/api/v1/knowledge/claims/975913MCP
get_claim(id=975913)