GGUF vs safetensors: The Format Doesn't Decide Everything — the Runtime Does

In discussions of model deployment, the GGUF vs safetensors showdown is often reduced to a cage match between file extensions. But that's not what's happening — it's more like a gate check. Choose GGUF, and the doors open for llama.cpp, Ollama, LM Studio, Jan, and other runtimes; choose safetensors, and transformers, vLLM, and ComfyUI become your stage. The format itself only determines which doors open, and everything else follows.

This article rejects a common lie: that this contest is decided by speed numbers that were never verified on the same machine. What we can actually obtain for free — and what most arguments overlook — is the weight of each container itself, and where each one can be opened.

What Are You Actually Choosing?

GGUF is a single-file container built by the author of llama.cpp, with quantization embedded directly at the tensor level — the quantization ladder everyone argues about exists only inside this format. One file carries the weights, tokenizer, and chat template simultaneously: the file itself is the distribution unit.

Safetensors is a serialization container — a JSON header plus raw byte tensors — that carries the precision at which the training repository was published, usually BF16 or FP16. It does not quantize; it only stores. Its name comes from its single mission: ensuring that model loading cannot be exploited by pickle deserialization attacks. Python's pickle executes code when unpacking; safetensors executes nothing — that is its entire security claim, and the reason the Hugging Face ecosystem standardized on it.

Weigh the same model in both containers, and the design philosophy shows up in the bytes. The same Qwen3-0.6B model, published in different formats: Q4_K_M GGUF is 396,705,472 bytes (397 MB); BF16 GGUF is 1,198,182,848 bytes (1.20 GB); BF16 safetensors is 1,503,300,328 bytes (1.50 GB). The quantized container is a quarter the size of the full-precision one because it carries only a quarter of the precision. And these numbers can be verified in your terminal with a single curl -I command, no benchmarking rig required. Whether the missing precision affects quality is a fidelity question that bytes cannot answer.

A format is not a preference — it is the list of runtimes that can open your file.

Quantization refers to compressing model weights from high-precision floating point (such as FP16, BF16) into lower-bitwidth representations (such as 4-bit, 8-bit integers), thereby dramatically reducing model size and memory footprint. Q4_K_M in the GGUF format is a mixed quantization scheme: K stands for k-quant block quantization, M stands for the medium quality tier. It preserves higher precision in critical weight layers and uses 4-bit in others, balancing compression ratio against output quality. Quantization is typically performed after model training completes (post-training quantization), so it inevitably introduces some precision loss, with the specific impact depending on model size, task type, and quantization granularity. A small 0.6B model may show more noticeable quality degradation under Q4 than a 70B model, because small models have less redundancy to begin with.

Pickle deserialization attacks are a long-standing security vulnerability in the Python ecosystem: the pickle format can execute arbitrary code during deserialization, so loading .pt or .bin model files from untrusted sources is equivalent to running unknown scripts. Hugging Face has discovered model repositories with embedded malicious pickle code. Safetensors' design fundamentally eliminates this attack surface — it only stores tensor metadata and raw bytes, performing only memory mapping at load time, executing no code. This is one of the core motivations behind Hugging Face making safetensors the default format, beyond mere loading speed.

What Do the Bytes Reveal?

For an article that only reads records, the honest rule is: state only the numbers noted in the records, cite the source, and don't fake benchmarking. The table below shows the actual byte sizes of the same model in three containers — verify it yourself before trusting any data.

Container	Precision	Published Bytes	Share of Full Precision
GGUF Q4_K_M	4-bit quantized	396,705,472 (397 MB)	26%
GGUF BF16	Full precision, llama.cpp packed	1,198,182,848 (1.20 GB)	80%
safetensors BF16	Full precision, raw tensors	1,503,300,328 (1.50 GB)	100%

Verify it yourself, no benchmarking rig required:

curl -sIL https://huggingface.co/unsloth/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B-Q4_K_M.gguf | grep -i content-length

Use the same command for the BF16 and safetensors files — the byte count is plainly checkable at any time.

Read the table honestly: 26% is the entire promise of the quantization ladder — replace sixteen bits with four, on disk and in memory alike. The gap between the two full-precision rows (1.20 GB vs 1.50 GB) is packing overhead, not a law of physics: llama.cpp's BF16 packing stores the same tensors more compactly than a raw tensor dump. Yet bytes cannot speak to speed or quality; a tokens-per-second figure without a matched benchmarking environment is just forum noise, and this article refuses to provide one.

Three clear walls: first, quality — between Q4 and BF16 lies a fidelity trade-off that byte counts cannot measure. Second, speed — load times and tokens per second belong to benchmarks matched on a specified machine; this article has no such conditions, so it offers no numbers. Third, scale — the byte ratios of a 0.6B model take a similar shape at 70B, but the absolute wait times you'll feel at 70B are another article on different hardware.

Packing overhead refers to the extra bytes generated by non-precision factors such as alignment, metadata, and tensor layout when model weights are stored. GGUF BF16 is about 20% smaller than safetensors BF16 because llama.cpp serializes and packs tensors more compactly, whereas safetensors preserves the training framework's original tensor layout and JSON header. This difference does not affect model precision; it merely reflects the design trade-offs of the two containers in storage efficiency: GGUF leans toward single-file self-containment and compactness, safetensors toward seamless integration with the training ecosystem.

Where Does Each Format Actually Run?

This is the part that truly decides things. GGUF opens in llama.cpp, Ollama, LM Studio, Jan — the single-file local ecosystem. vLLM, through its plugin, can also read GGUF on GPU, but with some caveats. Safetensors opens in transformers, native vLLM environments, ComfyUI, and Apple's MLX toolchain. The overlap at the edges is telling: Ollama — an application built on a GGUF engine — still ships an MLX runner so Apple Silicon users can load weights originating from safetensors. Even the applications themselves hedge their bets.

You're Already Running	Format Auto-Selected	Why
Ollama, LM Studio, or llama-server	GGUF	Its engine natively supports GGUF; quantized files fit small disks
Python/transformers pipeline	safetensors	from_pretrained is the native path; no conversion step to maintain
vLLM on a GPU server	safetensors	Native support, no plugin, no GGUF caveats
ComfyUI or image pipeline	safetensors	The diffusion ecosystem uses safetensors checkpoints
Small disks and small devices	GGUF Q4_K_M	397 MB vs 1.50 GB for the same weights

GGUF treats quantization as the default; safetensors treats full precision as the default. That default is most of the decision.

vLLM's GGUF plugin allows vLLM to load GGUF quantized models on GPU, but with several limitations: not all quantization types are supported, in some cases you must manually specify the --quantization parameter, and performance may differ from the native safetensors path. The plugin is aimed primarily at users who want to use quantized models within vLLM's high-throughput inference framework, but the official recommendation remains safetensors as vLLM's preferred format. Understanding this distinction helps avoid compatibility issues caused by choosing the wrong format on a GPU server.

Which One Should You Deploy?

Before looking at any table, check what's already installed. If the machine responds to ollama run, you live in GGUF country — pick the Q4_K_M file and stop reading forums. If your daily work is in Python, or your serving layer is vLLM, then safetensors was never really an option; it's the ecosystem's native tongue. Conversion between the two formats is possible, but almost never worth it: every conversion is a fresh chance for weights to silently diverge.

Honesty principle: I have read the records, but I have not actually run them. All the numbers above come from published files pulled on the day this article was published — no benchmarking rig, no pretense of tokens per second. What the records establish is the shape of the trade-off — a quantized container using a quarter of the bytes, two barely overlapping runtimes, and a default precision on each side quietly making the decision for most people.

Nobody switches formats for speed. They switch runtimes for a feature, and the format changes along with it as the ticket of admission.

Over to You

This is a discussion-first journey, so the debate moves to the comments. My position is on the table: choose the runtime first, and let the format follow. I expect the pushback I'll receive — that a fair comparison requires a concrete test environment before any opinion can be formed, and that most format arguments are just runtime arguments in disguise.

So here are two things worth arguing about: which shelf does your daily tool actually open from? Has anyone felt Q4 quality degradation on a real task, or is that just forum legend? Name the runtime and the model; the best-documented answers will set the terms of the next article.