[KongchangAI]
Conceptnonlinearity-noise equivalence / 非线性-噪声等价性

Nonlinear Noise Equivalence

A core concept in linear attention theory. It states that linear attention, when processing nonlinear target functions, can only extract the linear Hermite component; nonlinear structure acts as effective noise from the model's perspective. This reveals a fundamental expressive limitation of linear attention architectures.

Source Articles