KLD
Kullback-Leibler (KL) divergence is a metric that measures the difference between two probability distributions. In the evaluation of quantized models it is used as a diagnostic tool: the smaller the value, the closer the output distribution of the quantized model is to that of the original model.
Timeline (last 90 days)
UDv3.1 is clearly better than GSQ-RCO on the KLD and top-1% metrics across multiple domains including coding, agentic coding and dialogue.
The KLD of models uncensored via traditional full fine-tuning typically falls in the range of 0.1 to 0.5 or even higher, while LoRA-based methods generally fall between 0.05 and 0.15.
kvarn quantization performs slightly worse than qx_x on the needle test but has better KLD metrics, and saves VRAM whereas qx_x increases VRAM usage.
All Facts (3)
UDv3.1 is clearly better than GSQ-RCO on the KLD and top-1% metrics across multiple domains including coding, agentic coding and dialogue.
50%UnverifiedThe KLD of models uncensored via traditional full fine-tuning typically falls in the range of 0.1 to 0.5 or even higher, while LoRA-based methods generally fall between 0.05 and 0.15.
50%Unverifiedkvarn quantization performs slightly worse than qx_x on the needle test but has better KLD metrics, and saves VRAM whereas qx_x increases VRAM usage.
50%