IQ3 Quantization
A non-uniform low-bit model quantization scheme belonging to the importance-aware quantization family. It allocates different levels of precision to weight layers based on their importance, preserving model performance within an overall 3-bit memory footprint. It is commonly used with GGUF-format models in the llama.cpp framework.
Timeline (last 90 days)
The demo used an RTX 5090 GPU paired with 96 GB of memory, running a quantized IQ3-level model, with overall performance being satisfactory.
IQ3 is a non-uniform quantization scheme under the GGUF format, developed by the llama.cpp community; compared to earlier uniform quantization methods such as Q3_K, it better preserves critical weights at the same compression ratio.
All Facts (2)
IQ3 is a non-uniform quantization scheme under the GGUF format, developed by the llama.cpp community; compared to earlier uniform quantization methods such as Q3_K, it better preserves critical weights at the same compression ratio.
50%UnverifiedThe demo used an RTX 5090 GPU paired with 96 GB of memory, running a quantized IQ3-level model, with overall performance being satisfactory.
50%