Unverified50% confidenceFactExact time
Trained language models show a slope of α ≈ 1.4–1.7, whereas the baseline for initialized models is only α ≈ 0.2.
1
Sources
50%
Confidence
Long-term
Relevance
10/14/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe slope α correlates with the model's predictive performance and may capture structured knowledge learned by the model during training that goes beyond generic extreme-value effects.71% similarUnverifiedZero-initialized QKV expansion initializes the weights corresponding to newly added dimensions to zero, ensuring that the model's behavior at the start of adaptation training is identical to that before the expansion.64% similarUnverifiedDuring LoRA training, the pre-trained model weights W₀ are frozen and only the two matrices B and A are trained, with the forward propagation formula H = W₀X + (α/r)BAX.60% similarUnverifiedThe initial model f0(x) in gradient boosting typically uses the mean of y as the initial prediction for all samples.59% similarVerifiedLoRA achieves fine-tuning by inserting low-rank decomposition matrices alongside the model's weight matrices, typically requiring training of only 0.1%–1% of the original parameter count.59% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1023594API
curl https://kongchang.com/api/v1/knowledge/claims/1023594MCP
get_claim(id=1023594)