DAQ
A post-training quantization method for spiking Transformers that calibrates quantizers so that the firing decisions of quantized neurons align with those of a floating-point reference model. It supports 4-bit quantization of weights and membrane variables, and is used to reduce storage and computation overhead.
Timeline (last 90 days)
DAQ (Decision-Aligned Post-Training Quantization) is a post-training quantization method for spiking Transformers that calibrates the quantizer to align neuron firing decisions.
The DAQ method achieves efficient compression and energy reduction under 4-bit weight and membrane variable quantization.
DAQ changes the calibration objective of the quantizer from minimizing numerical error to aligning firing decisions, making the neuron firing decisions after quantization as consistent as possible with the firing decisions of the floating-point reference model.
DAQ analyzes how quantizer error affects firing at decision boundaries and how these errors propagate across time steps in recurrent neurons.
DAQ constructs an objective function that encourages the quantized membrane variables to remain within the reference firing region along the quantized model's own state trajectory, thereby reducing the disruption of quantization to spike firing patterns.
On the FE108 dataset, DAQ combined with the SDTrack model achieved 58.8% AUC and 90.9% PR while reducing energy consumption by 66.2%.
DAQ is applicable to a variety of tasks such as image classification, semantic segmentation, and event tracking.
The idea of DAQ may potentially be extended to lower bit widths (such as 2-bit) or more complex spiking architectures.
All Facts (8)
DAQ (Decision-Aligned Post-Training Quantization) is a post-training quantization method for spiking Transformers that calibrates the quantizer to align neuron firing decisions.
50%UnverifiedThe DAQ method achieves efficient compression and energy reduction under 4-bit weight and membrane variable quantization.
50%UnverifiedDAQ changes the calibration objective of the quantizer from minimizing numerical error to aligning firing decisions, making the neuron firing decisions after quantization as consistent as possible with the firing decisions of the floating-point reference model.
50%UnverifiedDAQ analyzes how quantizer error affects firing at decision boundaries and how these errors propagate across time steps in recurrent neurons.
50%UnverifiedDAQ constructs an objective function that encourages the quantized membrane variables to remain within the reference firing region along the quantized model's own state trajectory, thereby reducing the disruption of quantization to spike firing patterns.
50%UnverifiedOn the FE108 dataset, DAQ combined with the SDTrack model achieved 58.8% AUC and 90.9% PR while reducing energy consumption by 66.2%.
50%UnverifiedDAQ is applicable to a variety of tasks such as image classification, semantic segmentation, and event tracking.
50%UnverifiedThe idea of DAQ may potentially be extended to lower bit widths (such as 2-bit) or more complex spiking architectures.
50%