Conceptadditive sinusoidal positional encoding / 正弦位置编码
Additive Sinusoidal Positional Encoding
The positional encoding scheme used in the original Transformer paper, which adds the sine and cosine values of positions directly to input word embeddings, so that positional information is superimposed onto token representations as a signal.
Timeline (last 90 days)
Oct 12
Additive sinusoidal positional encoding gives different attention scores for the same offset but different absolute positions, because after expanding the dot product, cross terms related to the absolute positions remain.
Unverified50%