Voodoo Dynamic Quant Goes Open Source: Using Gradient Descent to Optimize Model Quantization

Developer curvedinf has announced the open-source release of Voodoo Dynamic Quant (Voodoo Quant, or VQ) under the MIT license — a dynamic quantization method previously kept under wraps. The tool had already demonstrated state-of-the-art performance in extreme quantization compression scenarios for small Qwen3.5-series GGUF models. Its full implementation is now available on GitHub (voodoo-dyn-quant), allowing the community to create their own dynamically quantized models.

For users running large models locally — especially those constrained by limited VRAM — this is a noteworthy development, as open-source dynamic quantization projects remain rare.

What Is Dynamic Quantization

To understand the value of Voodoo Quant, it helps to first grasp the difference between "dynamic quantization" and "static quantization."

Quantization is fundamentally about compression: reducing model weights from high precision (e.g., BF16) to fewer bits, shrinking file size and VRAM requirements. Formats like GGUF support assigning individual quantization levels to each tensor (a group of weights) in the model.

Static quantization applies fixed quantization level rules to certain types of tensors — essentially prescribing "which layer type gets which precision" in advance. Dynamic quantization, on the other hand, makes per-tensor decisions tailored to each target checkpoint size, theoretically preserving more useful information at the same file size.

The difference may seem subtle, but at the most aggressive (lowest-bit) quantization levels, fine-grained per-tensor decisions significantly impact output quality. This is precisely where Voodoo Quant claims its edge.

GGUF (GPT-Generated Unified Format) is a model storage format led by the llama.cpp project, designed for local inference. It supports storing different layers at different precisions — for example, attention layers at Q4_K and embedding layers at Q8_0. Common quantization levels range from Q2_K (most aggressive, ~2 bits/weight) to Q8_0 (near full precision), with file sizes varying by several times. This per-tensor configurability makes "which layers to keep at higher precision" a meaningful optimization problem. Static quantization answers this with hand-crafted rules; dynamic quantization tries to answer it with data-driven methods.

Optimizing Quantization Layout with Gradient Descent

The community had long speculated about the method behind VQ. The truth, it turns out, is "simple and elegant": using gradient descent to optimize the per-tensor quantization layout.

According to the author, the core VQ pipeline works as follows:

• Run all quantization levels in parallel: For each tensor, prepare weights for all candidate quantization levels simultaneously (converted directly via ggml, the underlying library of llama.cpp), then freeze all of these candidate weights.
• Train only a scalar gate: For each quantization level of each tensor, a trainable scalar gate is set up. One epoch of training determines which quantization levels are optimal.
• Annealing mechanism: A tau parameter is gradually annealed during training, guiding the model from a "blended multi-level" state toward a clear "single dominant level" selection.
• Softmax ensures gradient flow: Even when a tensor's selection has largely converged, Softmax ensures all quantization levels continue receiving gradients.

In short, VQ transforms "which precision to choose" into a differentiable optimization problem, letting the model learn the optimal quantization configuration through training on a diverse calibration dataset.

The scalar gate combined with Softmax is essentially a differentiable "hard attention" mechanism. For each tensor with N candidate quantization levels, there are N trainable scalars $g_1, g_2, \ldots, g_N$. After Softmax normalization, these produce a weight distribution, and during the forward pass the candidate weights are combined as a weighted sum. Early in training, when tau is large, the Softmax output is nearly uniform (blending multiple levels). As tau anneals toward zero, the Softmax approaches a one-hot distribution, effectively "selecting" a single level. This annealing strategy is common in discrete choice optimization and shares the same spirit as the Gumbel-Softmax trick — both use continuous relaxation to bypass non-differentiable discrete choices and allow gradient backpropagation.

Loss Function: KL Divergence + File Size Target

Voodoo Quant's training objective consists of two components:

1. KL Divergence (KLD): Measures the difference between the logits of the mixed-quantization model and the reference BF16 checkpoint. Lower KLD yields higher reward, meaning the quantized model's outputs are closer to the original full-precision model.
2. File size target: Rewards the model for approaching a user-specified target file size.

This design lets users simply specify "I want to compress to this size," and gradient descent automatically finds the best per-tensor quantization scheme within that constraint. Compared to repeatedly running benchmarks manually in a static analysis approach, this is far more efficient — equivalent to evaluating far more benchmark iterations than would be feasible by hand.

KL Divergence (Kullback-Leibler Divergence) measures the "information loss" between two probability distributions, and is widely used as a quality proxy metric in knowledge distillation and quantization-aware training. In VQ's context: the quantized model's logits are converted via Softmax to a probability distribution Q, and the reference BF16 model's logits become distribution P. KLD = Σ P(x) log(P(x)/Q(x)). A KLD of zero means the two models produce identical outputs on that input. Compared to simply comparing numerical weight errors, using KLD at the logits level more directly reflects whether the model's external behavior is consistent — making it one of the leading choices for quantization quality evaluation. The file size target is approximated by computing a weighted sum of the bit counts for each tensor's selected level, keeping the overall constraint differentiable.

Comparison with Unsloth Dynamic 3.0

The author candidly compares VQ against the well-known Unsloth Dynamic 3.0 (UD3), with a side jab: some had criticized him for not publishing his methodology, "but Unsloth hasn't published their method either, and they've been doing this for years."

Looking at the results, each has its strengths:

• Mid-to-high quantization levels: UD3 performs better.
• Aggressive (low-bit) quantization levels: VQ wins.

The author's assessment is that UD 3.0 is an advanced version of mainstream static analysis techniques — statistically analyzing weight distributions and iteratively tuning KLD benchmarks. Voodoo Quant, to his knowledge, is the first method to use backpropagation and gradient descent to select per-tensor quantization levels.

The tradeoff: optimizing quantization layout with gradient descent demands far more compute than static analysis, but in return extracts more performance.

Unsloth is a well-known quantization and fine-tuning tool in the local LLM ecosystem. Its Dynamic series works by statically analyzing numerical distribution characteristics of each layer's weights (e.g., kurtosis, outlier density), combined with iterative KLD benchmark testing, to assign optimal quantization levels to different layers. This approach is computationally cheap and can run on CPU, making it suitable for large-scale models — but its search space is limited, remaining fundamentally a local optimization within human-designed rules. VQ replaces hand-crafted rules with gradient descent, theoretically exploring solution spaces that static analysis cannot reach, though at the cost of significant GPU compute dependency. This explains why the two methods excel at different compression levels: the more extreme the compression, the less effective static rules become, and the more data-driven optimization shines.

Status: Research-Grade, Still Being Validated

The author is refreshingly honest about the project's maturity. He explicitly describes it as a research-grade project that has not yet been studied at larger model scales:

• It performs well on small models, especially at the lowest quantization levels — the levels that benefit most from complex, varied quantization choices.
• Testing was conducted with research-grade controls, but has not reached scientific-level proof of validity.
• There is still much unknown about how effective it can be, and the author hopes more researchers will explore this direction.

The current repository provides a complete toolkit for training dynamically quantized models using this method. The default configuration targets the Qwen architecture, but the author says it can be quickly adapted to any model architecture.

Why Open Source Now

The author's reasoning is straightforward: with over a dozen projects running simultaneously, he lacks the bandwidth to develop VQ into a full product worthy of its potential. Rather than let it sit idle, he's handing it to the community. He notes he's been doing open source for 20 years — this is just another ordinary act of sharing.

For the local LLM ecosystem, an open-source method that uses gradient descent for per-tensor quantization opens new territory for both low-VRAM users and researchers. Whether or not VQ ultimately scales, it introduces an approach to dynamic quantization that was previously rare in the field.