GGA
A mixed-precision allocation method for recurrent state quantization. It allocates bit widths by evaluating the gradient sensitivity of each attention head's quantization residual to the prediction loss, reducing overall quantization error under a fixed storage budget.
Timeline (last 90 days)
The study proposes two methods, global gradient-aware allocation (GGA) and pure-write sliding window (PW), to simultaneously address the challenges of precision allocation and recent-information maintenance within a storage budget.
GGA allocates bit widths to different heads under a model-level mixed-precision budget by evaluating the gradient impact of each attention head's candidate quantization residual on the future prediction loss.
On the Qwen3.5-4B and 9B models, using only GGA allocation without any other changes improved the scores on seven evaluations by 19.48 and 29.84 percentage points, respectively, compared with STEPQuant@4.
After adding PW-S8 on top of GGA, the seven-task macro averages reached 69.83% and 78.63%, respectively, an improvement of 38.39 and 41.21 percentage points compared with STEPQuant@4.
Under the @6 setting, GGA reduced the average generation length (equally averaged across tasks) by 10.45% and 14.65% relative to STEPQuant; adding PW-S8 on top of GGA@4 further reduced the average generation length by 32.32% and 36.77%.
GGA handles precision allocation along the spatial dimension, while PW handles information freshness along the temporal dimension; the two are complementary, forming a combined strategy of allocation by sensitivity and layering by time.
All Facts (6)
The study proposes two methods, global gradient-aware allocation (GGA) and pure-write sliding window (PW), to simultaneously address the challenges of precision allocation and recent-information maintenance within a storage budget.
50%UnverifiedGGA allocates bit widths to different heads under a model-level mixed-precision budget by evaluating the gradient impact of each attention head's candidate quantization residual on the future prediction loss.
50%UnverifiedOn the Qwen3.5-4B and 9B models, using only GGA allocation without any other changes improved the scores on seven evaluations by 19.48 and 29.84 percentage points, respectively, compared with STEPQuant@4.
50%UnverifiedAfter adding PW-S8 on top of GGA, the seven-task macro averages reached 69.83% and 78.63%, respectively, an improvement of 38.39 and 41.21 percentage points compared with STEPQuant@4.
50%UnverifiedUnder the @6 setting, GGA reduced the average generation length (equally averaged across tasks) by 10.45% and 14.65% relative to STEPQuant; adding PW-S8 on top of GGA@4 further reduced the average generation length by 32.32% and 36.77%.
50%UnverifiedGGA handles precision allocation along the spatial dimension, while PW handles information freshness along the temporal dimension; the two are complementary, forming a combined strategy of allocation by sensitivity and layering by time.
50%