[KongchangAI]
ModelDecision-Aligned Post-Training Quantization

DAQ

A post-training quantization method for spiking Transformers that calibrates quantizers so that the firing decisions of quantized neurons align with those of a floating-point reference model. It supports 4-bit quantization of weights and membrane variables, and is used to reduce storage and computation overhead.

Timeline (last 90 days)

Nov 18

DAQ (Decision-Aligned Post-Training Quantization) is a post-training quantization method for spiking Transformers that calibrates the quantizer to align neuron firing decisions.

Unverified50%
Nov 18

The DAQ method achieves efficient compression and energy reduction under 4-bit weight and membrane variable quantization.

Unverified50%
Nov 18

DAQ changes the calibration objective of the quantizer from minimizing numerical error to aligning firing decisions, making the neuron firing decisions after quantization as consistent as possible with the firing decisions of the floating-point reference model.

Unverified50%
Nov 18

DAQ analyzes how quantizer error affects firing at decision boundaries and how these errors propagate across time steps in recurrent neurons.

Unverified50%
Nov 18

DAQ constructs an objective function that encourages the quantized membrane variables to remain within the reference firing region along the quantized model's own state trajectory, thereby reducing the disruption of quantization to spike firing patterns.

Unverified50%
Nov 18

On the FE108 dataset, DAQ combined with the SDTrack model achieved 58.8% AUC and 90.9% PR while reducing energy consumption by 66.2%.

Unverified50%
Nov 18

DAQ is applicable to a variety of tasks such as image classification, semantic segmentation, and event tracking.

Unverified50%
Nov 18

The idea of DAQ may potentially be extended to lower bit widths (such as 2-bit) or more complex spiking architectures.

Unverified50%

All Facts (8)

Source Articles