Unverified50% confidenceFactExact time
YaRN方法通过高频部分与低频部分的混合插值策略，在不进行大规模重新训练的情况下将上下文窗口扩展8-16倍
1
Sources
50%
Confidence
Long-term
Relevance
7/20/2026
First Seen
Sources
Related Claims
UnverifiedUltra-long context in LLMs relies on Sparse Attention, RoPE (Rotary Position Embedding) extrapolation, and KV Cache compression techniques57% similarUnverifiedGradient accumulation accumulates gradients over multiple small batches before performing a single parameter update, with an effect equivalent to training with a large batch.56% similarUnverifiedEmbedding fine-tuning improves recall coverage, while reranking improves precision ranking accuracy; the combination of the two is the standard solution for production-grade RAG systems.56% similarUnverifiedFP8动态范围较窄，在反向传播和梯度累积时容易出现数值溢出或下溢，因此训练侧actor保持BF16精度56% similarVerifiedHNSW reduces search complexity to approximately O(log n) by constructing a multi-layer graph index, while IVF partitions the vector space via K-means clustering to achieve orders-of-magnitude speed improvements at the cost of approximately 5–10% recall.55% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/575505API
curl https://kongchang.com/api/v1/knowledge/claims/575505MCP
get_claim(id=575505)