Mellum2.1 Real Download Size: 8.07GB for Q4 Quantization, Must-Read Before Deploying

Don't Be Fooled by Parameter Count: Check the Real File Size First

Every post about JetBrains Mellum2.1 repeats the same line: a 12B model that runs like a 2.5B. That's indeed a real characteristic of MoE architecture—12B total weights, roughly 2.5B activated per token, which is why it can fit on GPUs that a dense 12B cannot.

But nobody tells you how big the download is. Rather than trusting secondhand summaries, the author went straight to the Hugging Face API and pulled the complete file listing for the JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF repository (read on 2026-10-13). Here's what came back:

Quantization Scheme	File Size
BF16 (reference)	24.31 GB
Q8_0	12.93 GB
Q6_K	10.88 GB
Q4_K_M (recommended)	8.07 GB
MXFP4_MOE (smallest)	7.03 GB

The repository was created on 2026-10-07, and at the time of reading had 27,140 downloads and 78 likes, under the Apache 2.0 license—commercial use is fine.

Key facts: the recommended Q4_K_M build is 8.07 GB, the smallest MXFP4_MOE is 7.03 GB, and the reference-precision BF16 is a hefty 24.31 GB.

MoE stands for Mixture of Experts. Structurally, these models split the feed-forward network into several parallel "expert" subnetworks, then use a lightweight router to select only a small subset of them to participate in computation for each token. This produces two different ways of counting parameters: total parameters is the sum of all experts plus shared layer weights like attention and embeddings, while activated parameters (written here as A2.5B, i.e., active 2.5B) is the number of weights a single token actually passes through. Mellum2.1 having 12B total weights and roughly 2.5B activated per token is exactly this.

What's especially important to note: MoE saves compute and memory bandwidth, not memory capacity. At inference time, all 12B weights still need to be loaded into memory and ready to go—which expert gets routed to is determined dynamically at runtime, and you can't preemptively keep only a subset. This is also why download size (8.07 GB for Q4_K_M) is determined by total parameters, not activated parameters—parameter count and file size are two different things.

16GB Is the Real Threshold, and Context Window Is the Trap

The Q4_K_M file itself is 8.07 GB. Add KV cache and runtime overhead, and a 16 GB machine is the practical floor for the recommended build at short-to-medium context.

The problem lies in the example command that ships with the GGUF model card, which sets --ctx-size 131072, i.e., the model's full 131,072-token window. If you copy that line directly on a 16 GB laptop, you'll hit memory pressure before you ever approach the token limit. The author's advice is straightforward: start at 8k–16k and only go higher when you genuinely need to.

This is the part that "runs on a laptop" claims gloss over: the model fits, but the context may not.

KV cache refers to the Key and Value tensors cached during inference. When a Transformer generates each new token, it has to look back at the attention keys and values of all previous tokens. Recomputing them every time would be prohibitively expensive, so these intermediate results are stored per layer and per attention head in VRAM or RAM to avoid redundant computation.

The key point is that its size grows linearly with context length: double the sequence, and the cache roughly doubles too, and it's proportional to batch size as well. So on a 16 GB machine, the 8.07 GB of model weights alone takes up half, and the remaining space has to simultaneously accommodate KV cache, activations, tokenizer, and runtime overhead. This is why --ctx-size 131072 is dangerous: even if the model fits, at long context the KV cache alone can eat several GB and crush your memory. Using fewer layers or quantized KV cache (e.g., --cache-type-k q8_0) can help, but at the cost of precision and speed.

Two Details That Will Trip You Up

No Official Ollama Library Tag

Both ollama.com/library/mellum2.1 and ollama.com/library/mellum2 return 404. The ollama run hf.co/JetBrains/... command line you see in posts works because it pulls the GGUF directly from Hugging Face rather than resolving a local library name. If you thought ollama run mellum2.1 would work, it won't.

GGUF is the single-file model format championed by the llama.cpp project, short for GPT-Generated Unified Format. It packs weights, tensor metadata, tokenizer vocabulary, and quantization information into one binary file, with tensor naming conventions agreed upon in advance, so the same file can be read directly by llama.cpp, Ollama, LM Studio, and many other inference tools without extra configuration or conversion scripts.

Models in the official Ollama library (like ollama run llama3) are essentially pre-packaged manifests pulled from registry.ollama.ai. When you write ollama run hf.co/JetBrains/..., Ollama recognizes the hf.co/ prefix and instead resolves and downloads the corresponding GGUF directly from Hugging Face, then builds a local manifest. This also explains why the mellum2.1 tag can't be found in the official library yet still runs—the former depends on the community or vendor proactively publishing to the Ollama registry, while the latter just borrows HF's generic download path. Different prefix, two entirely separate repository systems behind it.

MXFP4_MOE Is Not in Older llama.cpp Versions

The smallest quantization scheme (7.03 GB) requires a fairly recent build. If your toolchain is a few months old, either pick Q4_K_M or upgrade first.

What JetBrains' Own Data Says

The following data comes from JetBrains' self-reported benchmark results, not an independent reproduction. The comparison target is a dense 9B model from another lab:

Benchmark	Mellum2.1	Qwen3.5-9B
SWE-bench Verified	47.0%	50.0%
SWE-bench Pro	28.0%	38.0%
Terminal-Bench 2.1	17.4%	21.7%
LiveCodeBench v6	82.0%	75.4%
HumanEval+	91.5%	89.6%

This table deserves a second read, because it tells a different story than the launch event. The 12B MoE wins on single-shot code generation tasks (LiveCodeBench, HumanEval+), but loses to a 9B dense model on multi-step agentic benchmarks (SWE-bench Pro, Terminal-Bench)—and the latter fits in a fraction of the memory.

JetBrains also reports 7,969 tokens/s on a single H200, versus 4,347 tokens/s for Qwen3.5-9B—again, their own testing.

If you're buying this model for throughput (high-concurrency generation, lint-level edits, a fast executor behind a slow planner), the data supports you. If you're buying it expecting it to independently solve complex repo-level issues, then by the vendor's own table, a smaller dense model scores higher.

Why I'm Not Providing Throughput Test Data

The author's test machine is 2 vCPU, 7 GB RAM (~4 GB available), no GPU. The smallest published build is 7.03 GB, so the model simply won't fit on this machine. It wasn't run, and no tokens/s number will be cited that was never measured. What was actually measured is exactly what's above: file sizes, repository dates, two 404s.

This kind of honesty is worth learning from. The community is full of speed and score comparisons that haven't been independently verified, and publishers often won't tell you their hardware conditions.

Pre-Download Checklist

• Check memory against 8.07 GB plus KV cache, not against the "12B parameters" number
• Go with Q4_K_M unless you have 24 GB+ of RAM
• Cut context length on first run; the model card example is the maximum, not the default
• Test on a bug you've already fixed first, then point it at the repo you care about
• Treat every speed and score comparison as vendor self-reported until someone publishes independent testing

The interesting question over the next month isn't whether a 12B MoE can write code—it's which one your agent ends up using when you're faced with a faster open-source model versus a slower model that solves more tasks.

Sources: GGUF repository, model card, Hugging Face API metadata