#RoPE
2 related articles
·5 min
RoPE Decay Curve Is Noise: Taking Absolute Values Won't Fix It
The RoPE attention decay curve is actually sampling noise, and taking absolute values won't fix it. This article explains the relativity proof of rotary position encoding, the nature of phase dephasing, an LCG seed-related bug, and why NTK scaling cannot act on the fastest frequency pair, with 226 verification assertions and measured data.
Read more →
Tech Frontiers·2 min
DeepSeek-V3.2-Exp Discovers RoPE Implementation Bug: Interleaved Format Mismatch Causes Inference Performance Degradation
A RoPE interleaved format mismatch bug was found in DeepSeek-V3.2-Exp's inference demo, causing silent performance degradation. Learn the root cause, technical details, and fix.
Read more →