Post-Training Quantization
Compression performed by directly rounding or calibrating the weights of an already trained model; the model is not optimized for low precision, which can lead to accuracy loss.
Core Facts
Using quantized versions (such as Q4, Q5) can significantly reduce VRAM usage with minimal performance loss.
Post-hoc quantization usually causes noticeable performance degradation at small bit widths, whereas QAT simulates quantization errors during training to maintain low-precision output quality.
PTQ (Post-Training Quantization) does not require retraining the model; quantization can be completed with only a small amount of calibration data, typically a few hundred to a few thousand calibration samples.
Timeline (last 90 days)
DAQ (Decision-Aligned Post-Training Quantization) is a post-training quantization method for spiking Transformers that calibrates the quantizer to align neuron firing decisions.
Calibration is a key step in post-training quantization (PTQ), using a small batch of representative samples to compute the dynamic range or distribution of activations at each layer in order to determine quantization parameters.
Post-hoc quantization usually causes noticeable performance degradation at small bit widths, whereas QAT simulates quantization errors during training to maintain low-precision output quality.
Using quantized versions (such as Q4, Q5) can significantly reduce VRAM usage with minimal performance loss.
All Facts (7)
Using quantized versions (such as Q4, Q5) can significantly reduce VRAM usage with minimal performance loss.
85%VerifiedPost-hoc quantization usually causes noticeable performance degradation at small bit widths, whereas QAT simulates quantization errors during training to maintain low-precision output quality.
80%VerifiedPTQ (Post-Training Quantization) does not require retraining the model; quantization can be completed with only a small amount of calibration data, typically a few hundred to a few thousand calibration samples.
75%VerifiedQuantization-Aware Training (QAT) lets the model retain a full-precision copy of weights, practicing rounding operations in each forward pass to learn weight distributions under extreme compression.
70%Unverified量化感知训练(QAT)在训练过程中模拟量化误差反向传播,效果通常优于训练后量化(PTQ)
60%UnverifiedDAQ (Decision-Aligned Post-Training Quantization) is a post-training quantization method for spiking Transformers that calibrates the quantizer to align neuron firing decisions.
50%UnverifiedCalibration is a key step in post-training quantization (PTQ), using a small batch of representative samples to compute the dynamic range or distribution of activations at each layer in order to determine quantization parameters.
50%