MiniMax M3 Local Deployment: 23B Active Parameters ≠ 23GB VRAM — A Complete Guide to MoE Architecture

A Number That Keeps Getting Misread

Since the release of MiniMax M3, the same question keeps popping up in community discussions: "If M3 only has 23B active parameters, can it fit on a single 48GB GPU?"

This is an understandable but fundamentally wrong assumption. The root cause: active parameters describe the compute load per token, not the memory required to store the model weights.

MiniMax M3 is a 428B MoE (Mixture of Experts) model. The core idea behind MoE architecture is to split the large feed-forward network layers into multiple "expert" sub-networks, with a lightweight router dynamically selecting a small subset of experts for each inference step. This design has been widely adopted in large language models since around 2022 — Mixtral, DeepSeek-V2, and MiniMax M3 are all canonical examples. The HuggingFace BF16 repository for M3 weighs in at roughly 854GB, which aligns perfectly with the estimate of 428B parameters × ~2 bytes per parameter. While the router only activates roughly 23B parameters per token at inference time — dramatically reducing per-token compute — the entire expert pool must remain "accessible" at all times, whether that's in VRAM, system RAM, or some offloading/tiered storage structure.

The reason is straightforward: the router has no way of knowing in advance which experts any given token will hit, so all expert parameters must be on standby and ready to be called at any moment.

The Core MoE Trade-off: Heavy on Storage, Light on Compute

To truly understand the deployment requirements for M3, you need to internalize the fundamental nature of MoE architecture.

Active parameters and total parameters are two completely different dimensions. In a dense model, every inference pass uses all the weights, so these two metrics are one and the same. MoE introduces sparse activation, which splits them entirely apart. Total parameter count is a static storage metric — it determines how much weight data must be fully retained in some storage medium. Active parameter count is a dynamic compute metric — it describes the scale of parameters actually participating in matrix multiplications during a single forward pass for one token. For MiniMax M3, 23B/428B ≈ 5.4% — this is the source of its computational efficiency advantage, but it does absolutely nothing to change the storage reality of needing to load all 428B weights.

Storage Floor: Theoretical Values Across Three Quantization Levels

Based on different quantization precisions, the theoretical storage floor for M3's raw weights breaks down as follows:

Precision	Storage Floor
BF16	~854GB
8-bit	~428GB
4-bit	~214GB

Quantization is the technique of compressing weights from high-precision floating-point numbers to low-bit integers: BF16 uses 2 bytes per parameter, INT8 compresses to 1 byte, and INT4 compresses further to half a byte. However, quantization is not lossless — the lower the bit depth, the greater the precision loss, and model performance on complex reasoning tasks can degrade noticeably. Mainstream approaches in the industry include GPTQ, AWQ (Activation-aware Weight Quantization), and GGUF, each striking a different balance between quantization precision and inference speed. For MoE models, quantization requires additional consideration of the differing sensitivity between expert layers and router layers — the router is typically kept at higher precision to maintain routing accuracy.

It's worth emphasizing: these numbers are only the storage floor for raw weights, not a complete runtime memory estimate. Actual requirements are also affected by the quantization format, higher precision retained for certain layers, runtime buffers, KV cache, context length, and the specific serving deployment setup.

The KV cache is a non-trivial additional overhead that can't be ignored. During autoregressive generation, the model caches the attention Key/Value matrices from previous tokens in VRAM to avoid redundant computation. Its size scales with context length, number of attention heads, and layer count. If M3 supports very long context windows, the KV cache alone can occupy tens of gigabytes or more in full-context scenarios, pushing actual memory requirements well beyond what you'd estimate from weights alone.

In other words, even with 4-bit quantization compressing weights down to 214GB, that still far exceeds the capacity of a single 48GB GPU. Current mainstream serving deployment solutions broadly require multi-GPU or large-memory server-class infrastructure.

Computational Advantage: What "23B Active Parameters" Actually Means

The real value of "23B active" is this: you get inference sampled from a 428B pool of specialized parameters at roughly the per-token compute cost of a 23B dense model. Compared to a dense model of equivalent scale (428B), M3's per-token compute is dramatically lower.

This is the complete MoE trade-off in a nutshell: storage-heavy (must accommodate the full 428B expert pool), but far lighter on compute than a dense 428B model.

The conclusion is clear — 23B active parameters does not make M3 behave like a 23B dense model, and it certainly doesn't magically shrink the 428B storage challenge.

Practical Local Deployment Options

Developers who want to run M3 locally can choose different paths depending on their situation.

Option 1: Full Control or Private Deployment

If you have hard requirements around data compliance or private deployment, and you have the corresponding infrastructure (multi-GPU, high-memory servers), self-hosting is a reasonable choice. This path offers maximum control at the cost of hardware investment and operational complexity.

Option 2: Quantization and Offloading to Lower the Barrier

If you just want to experiment with M3 locally, you can reduce the entry cost through several techniques:

• Quantization (8-bit / 4-bit) to compress weight size;
• Shorter context windows to reduce KV cache usage;
• CPU/GPU offloading to move some expert parameters to system RAM or disk.

The core idea behind offloading is to "spill" weights that don't fit in VRAM into larger-capacity storage tiers — typically CPU RAM or even NVMe SSDs. For MoE models, a more refined strategy is "expert offloading": keeping infrequently activated experts in CPU RAM and only transferring them to GPU VRAM when the router selects them for computation. llama.cpp is currently one of the most mature inference frameworks supporting this kind of multi-tier offloading.

Be aware that these approaches introduce new bottlenecks: prompt processing speed and memory bandwidth often become the limiting factors. Frequent transfers of expert parameters between storage tiers are constrained by PCIe bus latency; once concurrent requests increase, PCIe bandwidth becomes a bottleneck that causes inference throughput to degrade sharply and inference speed to slow significantly.

Option 3: Just Testing Capabilities? Call the API

If your goal is simply to test M3's coding or agent capabilities — rather than spending enormous effort building a massive local environment — calling the official API is the path of least resistance, and the practical choice for most developers.

Closing Thoughts

The M3 deployment question is, at its core, a practical lesson in MoE architecture.

When evaluating whether a MoE model can run locally, active parameters and total parameters are metrics on completely different dimensions — the former concerns compute, the latter concerns storage and bandwidth. Confusing the two leads to unrealistic expectations like "running a 428B model on a 48GB GPU."

For developers, the right decision framework is: first, clarify whether your goal is "can run" or "runs smoothly," then decide whether to invest in infrastructure for self-hosting, use quantization and offloading for a low-resource experiment, or simply go with the API. And for the industry as a whole, while MoE has made "massive parameters, low compute" a reality, it has simultaneously pushed memory capacity and bandwidth to the forefront as the new cost frontier.

Key Takeaways