Concept正交变换预处理 / rotation matrix
Rotation Matrix
An orthogonal transformation technique used to preprocess weights during model quantization. By redistributing weight energy from a small number of extreme dimensions across more dimensions, it reduces quantization error and improves low-bit quantization accuracy. This approach originates from research including BitNet and QuIP#.
Timeline (last 90 days)
Oct 4
Before converting to ternary values, Prism ML applies a rotation matrix (orthogonal transformation) to smooth and disperse weight outliers so that the shared scale is more appropriate, an idea drawn from research such as BitNet and QuIP#.
Unverified50%