Unverified50% confidenceFactExact time
负采样将每步训练的复杂度从O(V)降低到O(k)，k是负样本数（通常取5-20）
1
Sources
50%
Confidence
Long-term
Relevance
8/18/2026
First Seen
Sources
Related Claims
Unverified困惑度是测试集上每个token平均负对数似然的指数，值越低代表模型对语料的预测越准确69% similarUnverifiedInfoNCE 损失通过在一批候选中最大化正样本对相似度、最小化负样本对相似度进行训练，温度超参数τ越小分布越集中69% similarPartially VerifiedFor a 4096×4096 weight matrix, LoRA may only need to train two 4096×8 matrices, reducing the number of trainable parameters from approximately 16 million to approximately 65,000, a reduction of 99.6%.67% similarVerifiedThe rank r of the low-rank matrices in LoRA is typically set to 8 or 16, reducing the number of trainable parameters from d×k to r×(d+k), a reduction of up to hundreds of times.66% similarUnverified在小样本LoRA训练中，rank值设置存在偏差-方差权衡，实践中rank=4至32是常见的安全区间66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/771179API
curl https://kongchang.com/api/v1/knowledge/claims/771179MCP
get_claim(id=771179)