Voodoo Dynamic Quant
An open-source dynamic quantization tool that uses gradient descent to optimize per-tensor quantization layouts. Given a file size constraint, it automatically searches for the quantization scheme with the highest quality, using KL divergence and file size as joint optimization objectives.
Timeline (last 90 days)
Developer curvedinf open-sourced their dynamic quantization method Voodoo Dynamic Quant (VQ) under the MIT license.
The complete implementation of Voodoo Dynamic Quant has been made available on GitHub (repository: voodoo-dyn-quant).
Voodoo Dynamic Quant has been demonstrated to reach SOTA performance in extreme quantization compression scenarios for the Qwen3.5 series of small GGUF models.
The core method of Voodoo Quant is to use gradient descent to optimize the per-tensor quantization layout.
VQ assigns a trainable scalar gate to each quantization tier for each tensor, freezes the candidate weights, and determines the optimal quantization level through just one epoch of training.
During VQ training, the tau parameter is gradually annealed, and Softmax is used to ensure all quantization tiers continuously receive gradients.
The training objective of Voodoo Quant consists of two components: KL divergence and a file size target, allowing users to directly specify the desired file size.
At mid-to-high quantization tiers Unsloth Dynamic 3.0 performs better, while at aggressive (low-bit) quantization tiers VQ is superior.
The author believes Voodoo Quant is the first method to use backpropagation and gradient descent to select per-tensor quantization levels.
Using gradient descent to optimize the quantization layout requires a far more powerful compute system than static analysis (with a higher dependence on GPUs).
2 more timeline events
All Facts (12)
The VQ repository's default configuration targets the Qwen architecture, but the author states it can be quickly adapted to any model architecture.
50%UnverifiedUsing gradient descent to optimize the quantization layout requires a far more powerful compute system than static analysis (with a higher dependence on GPUs).
50%UnverifiedThe author positions VQ as a research-grade project, which has not yet been validated on larger-scale models and has not reached a scientific level of proof of validity.
50%UnverifiedVoodoo Dynamic Quant has been demonstrated to reach SOTA performance in extreme quantization compression scenarios for the Qwen3.5 series of small GGUF models.
50%UnverifiedThe core method of Voodoo Quant is to use gradient descent to optimize the per-tensor quantization layout.
50%UnverifiedVQ assigns a trainable scalar gate to each quantization tier for each tensor, freezes the candidate weights, and determines the optimal quantization level through just one epoch of training.
50%UnverifiedDuring VQ training, the tau parameter is gradually annealed, and Softmax is used to ensure all quantization tiers continuously receive gradients.
50%UnverifiedThe training objective of Voodoo Quant consists of two components: KL divergence and a file size target, allowing users to directly specify the desired file size.
50%UnverifiedAt mid-to-high quantization tiers Unsloth Dynamic 3.0 performs better, while at aggressive (low-bit) quantization tiers VQ is superior.
50%UnverifiedThe author believes Voodoo Quant is the first method to use backpropagation and gradient descent to select per-tensor quantization levels.
50%UnverifiedThe complete implementation of Voodoo Dynamic Quant has been made available on GitHub (repository: voodoo-dyn-quant).
50%UnverifiedDeveloper curvedinf open-sourced their dynamic quantization method Voodoo Dynamic Quant (VQ) under the MIT license.
50%