Born Rule
One of the fundamental postulates of quantum mechanics, proposed by physicist Max Born in 1926. It states that the probability of obtaining a specific outcome when measuring a quantum system equals the square of the modulus of the corresponding probability amplitude, expressed mathematically as |⟨φ|ψ⟩|².
Timeline (last 90 days)
A Reddit developer attempted to replace the Softmax attention mechanism in Transformers with the Born rule from quantum mechanics (|⟨Q,K⟩|²), and claimed that the model was able to train.
The Born rule is one of the fundamental postulates of quantum mechanics, proposed by physicist Max Born in 1926.
The Born rule states that when a system is in state |ψ⟩, the probability of measuring eigenstate |φ⟩ equals the square of the modulus of the probability amplitude, |⟨φ|ψ⟩|².
The squared modulus in the Born rule is naturally non-negative, and after normalization automatically satisfies the probability distribution constraint, without needing to rely on an exponential function to enforce non-negativity as Softmax does.
Softmax tends to produce sharp, winner-takes-all distributions, whereas the distribution shape of squared normalization may be more spread out or more concentrated.
Dot products can be negative; Softmax maps them to small positive numbers via exponentiation, whereas taking the squared modulus causes positive and negative inner products to produce symmetric weight contributions.
The derivative of the squaring operation |x|² is 2x, which is linear, so gradients will not explode or vanish sharply at extreme values as they do with exponential functions, giving it a natural advantage in numerical stability.
This Born rule replacement experiment lacks benchmark comparisons with standard attention—under the same parameter count and dataset—on convergence speed, accuracy, and stability in large-scale scenarios.
All Facts (8)
This Born rule replacement experiment lacks benchmark comparisons with standard attention—under the same parameter count and dataset—on convergence speed, accuracy, and stability in large-scale scenarios.
50%UnverifiedThe Born rule is one of the fundamental postulates of quantum mechanics, proposed by physicist Max Born in 1926.
50%UnverifiedThe Born rule states that when a system is in state |ψ⟩, the probability of measuring eigenstate |φ⟩ equals the square of the modulus of the probability amplitude, |⟨φ|ψ⟩|².
50%UnverifiedThe squared modulus in the Born rule is naturally non-negative, and after normalization automatically satisfies the probability distribution constraint, without needing to rely on an exponential function to enforce non-negativity as Softmax does.
50%UnverifiedSoftmax tends to produce sharp, winner-takes-all distributions, whereas the distribution shape of squared normalization may be more spread out or more concentrated.
50%UnverifiedDot products can be negative; Softmax maps them to small positive numbers via exponentiation, whereas taking the squared modulus causes positive and negative inner products to produce symmetric weight contributions.
50%UnverifiedThe derivative of the squaring operation |x|² is 2x, which is linear, so gradients will not explode or vanish sharply at extreme values as they do with exponential functions, giving it a natural advantage in numerical stability.
50%UnverifiedA Reddit developer attempted to replace the Softmax attention mechanism in Transformers with the Born rule from quantum mechanics (|⟨Q,K⟩|²), and claimed that the model was able to train.
50%