IBM Granite 4.1 Quantization Test: SVG Generation Comparison Across 21 GGUF Versions

IBM Granite 4.1 Model Family Officially Released

IBM recently released the Granite 4.1 series of large language models under the Apache 2.0 open-source license, available in three parameter sizes: 3B, 8B, and 30B.

Apache 2.0 is one of the most permissive open-source licenses available, allowing commercial use, modification, distribution, and patent grants without requiring derivative works to also be open-sourced (unlike the copyleft requirements of GPL-type licenses). IBM's choice of Apache 2.0 over more restrictive licenses (such as the custom community license Meta used for its Llama series) means enterprises can directly embed Granite models into commercial products without additional legal negotiations. This strategy aligns with IBM's recent open-source business model built around Red Hat and the watsonx platform—attracting developer ecosystems through open model weights, then monetizing through enterprise-grade deployment tools and services.

Granite team member Yousaf Shah published a detailed blog post on Hugging Face covering the model's training process, showcasing IBM's technical details and methodology in model construction.

Unsloth Releases 21 GGUF Quantized Versions

The Unsloth team promptly released a collection of GGUF-format quantized versions of the Granite 4.1 3B model (unsloth/granite-4.1-3b-GGUF), comprising 21 different model files ranging from 1.2GB to 6.34GB, totaling 51.3GB.

Unsloth is an open-source project focused on LLM fine-tuning and inference optimization, founded by brothers Daniel and Michael Han. The team is known for providing efficient LoRA/QLoRA fine-tuning tools, claiming 2-5x speedups while reducing VRAM usage by 80%. They are also among the most active GGUF quantization publishers on Hugging Face, typically providing a full range of quantized versions within hours of a popular model's release. Unsloth's practice of offering up to 21 quantization variants for a single model reflects the local deployment community's granular hardware adaptation needs—from Raspberry Pis with only 4GB of RAM to consumer GPUs with 24GB of VRAM, different users need different precision-performance trade-off points.

GGUF (GPT-Generated Unified Format) is a model file format designed by Georgi Gerganov, creator of the llama.cpp project, specifically optimized for CPU and hybrid CPU/GPU inference. It succeeds the earlier GGML format, addressing its shortcomings in metadata management and forward compatibility. Quantization is a technique that compresses model weights from high-precision floating-point numbers (e.g., FP16, 2 bytes per parameter) to lower-precision representations (e.g., INT4, only 0.5 bytes per parameter). Common quantization levels include Q2_K, Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, Q8_0, etc., where K indicates the use of the k-quant method—a grouped quantization strategy that preserves higher precision for more important layers while applying more aggressive compression to less critical ones. In theory, Q8 quantization is nearly lossless, while Q2 quantization significantly degrades model capabilities, but the actual impact is highly dependent on the specific task and model architecture.

Pelican Riding a Bicycle: Testing Quantization Differences with SVG Generation

Prominent developer Simon Willison used these 21 quantized versions for an intuitive comparison experiment: testing all model variants with the same prompt "Generate an SVG of a pelican riding a bicycle" to observe the actual impact of different quantization levels on output quality.

SVG (Scalable Vector Graphics) is an XML-based vector graphics description language that uses text tags to define geometric shapes, paths, and transformations. Asking a large language model to generate SVG images is actually an extremely complex comprehensive capability test: the model needs to simultaneously possess code generation ability (correct SVG syntax), spatial reasoning ability (understanding relative positions and proportions of objects), world knowledge (knowing what pelicans and bicycles look like), and compositional reasoning ability (combining them in a sensible way). Simon Willison has long used SVG generation as an informal model capability probe, having previously tested Claude, GPT-4, and other models with similar methods, finding that even the most powerful models show inconsistent performance in complex SVG scene generation. This makes SVG generation an interesting but extremely challenging evaluation dimension.

The experiment was inspired by the 51.3GB total file size—Simon had long wanted to do this kind of quantization comparison test, and the 21 clearly graduated quantization versions provided ideal experimental conditions.

Results: Surprisingly Small Differences Across Quantization Levels

The results were unexpected: there was no discernible quality difference pattern across quantization levels. Models ranging from 1.2GB to 6.34GB all produced SVG images of comparably poor quality, mostly consisting of abstract shape combinations.

Even more interesting, the smallest model actually produced the best bicycle graphic, while the largest model only managed something vaguely resembling a pelican. This counterintuitive result reveals several important patterns:

• The SVG generation capability of a 3B parameter model is inherently quite limited, making the precision loss from quantization negligible compared to the model's own capability ceiling

3B (3 billion) parameters places a model in the small category within today's LLM spectrum, typically targeting edge device deployment, low-latency inference, and resource-constrained scenarios. For reference, GPT-3 has 175 billion parameters, Llama 3's largest version reaches 405B, and a 3B model has only 1-2% of these models' parameter counts. Research shows that emergent abilities—such as complex reasoning, code generation, and multi-step planning—typically only manifest after parameter counts reach certain thresholds. For tasks like SVG that require precise spatial coordinate calculations, a 3B model's internal representation space may simply be insufficient to establish reliable mappings from semantic descriptions to geometric coordinates, explaining why all quantized versions produced similarly limited output quality—the bottleneck lies in model capacity itself, not precision loss from quantization.

• Quantization's impact on different tasks is not uniform—for tasks the model already struggles with, quantization differences get drowned out by fundamental capability limitations
• SVG code generation requires precise spatial reasoning ability, which is an extremely challenging task for small-parameter models

Practical Implications for Local Deployment Developers

Simon noted he plans to repeat this experiment with a model that's better at drawing pelicans. Although this small experiment's results may seem "not very interesting," it provides several valuable reference points for choosing GGUF quantization versions:

1. Quantization level selection must be evaluated in the context of specific tasks—you cannot simply assume larger quantized files always produce better output
2. A model's baseline capability is a prerequisite for any quantization discussion—if the model itself lacks a certain capability, differences in quantization level have no practical significance
3. The Apache 2.0 license lowers the barrier to experimentation—the value of the open-source ecosystem lies in the community's ability to quickly validate and provide feedback

For developers who need to locally deploy Granite 4.1 3B, if the primary use case involves text generation and standard reasoning tasks, smaller quantization versions (such as Q4 or Q5) may be perfectly adequate without needing to sacrifice storage space and VRAM resources in pursuit of maximum precision. In practice, developers should run simple A/B tests on their target tasks—as Simon's experiment reveals, quantization's impact on different task types may be vastly different from intuition.