Meta's Largest Open-Weight Model Muse Spark Open 816B: A Deep Dive

Meta Superintelligence Labs has officially released Muse Spark Open, a sparse mixture-of-experts model with 816 billion parameters, purpose-built for agentic coding and long-horizon reasoning. It is the largest open-weight model Meta has ever released, but what truly deserves attention is a series of architecture decisions aimed at real-world deployment.

Architecture: 816B Total, 42B Active

816 billion parameters is the sum of all expert weights. At inference time, only 42 billion parameters are activated per token. The model has 62 Transformer layers, each containing an MoE module with 256 experts, and each token routes to 8 of them via a learnable router. The hidden dimension is 8192, with grouped-query attention used throughout.

This sparse activation pattern is the key to making the model practically deployable. A dense 816B model would need roughly 1.6 TB of memory for BF16 weights alone; the MoE structure brings per-token compute close to that of a 42B dense model, even though the full parameter space is far larger.

On the multimodal side, the model has a built-in 1.8B-parameter dedicated perception encoder for images and video frames, with audio handled by a separate encoder. Inputs across all modalities ultimately converge into text output — natively multimodal on the input side, text-only on the output side.

Mixture of Experts (MoE) is essentially about splitting a large feed-forward network into several "expert" subnetworks, with each token activating only a small subset of them. Here, "8 of 256" means the routing network scores each token and selects the top 8 experts to participate in computation; the weights of the remaining experts are loaded into memory but do not take part in that forward pass. This is where the phrase "816 billion total parameters, 42 billion active parameters" comes from: total parameter count determines memory footprint, while active parameter count determines compute. Compared with dense models, MoE trades a larger parameter space for higher knowledge capacity while keeping per-token FLOPs far below what the total parameter scale would suggest. The cost is that memory must hold all expert weights, and routing itself introduces load-balancing and communication overhead — which is also why MoE is harder to deploy than a dense model with the same activation count.

Speculative Decoding is a technique for accelerating autoregressive generation without changing the output distribution. It has a small, fast "draft model" generate several candidate tokens in a row, then the full large model verifies those candidates in parallel in a single pass: accepted tokens are kept directly, all drafts after the first rejected position are discarded and replaced with the large model's output at that position. Since verification can be done in parallel, a generation process that would otherwise require multiple full-model forward passes is compressed into a few. The key metric is "average accepted length" — the more draft tokens accepted per step, the higher the speedup. The draft model must align as closely as possible with the main model's tokenizer and distribution; otherwise the rejection rate is too high and the speedup gains are eaten up by the draft model's own overhead.

How the Million-Token Context Window Is Achieved

Muse Spark Open supports a 1 million token context window, enabled by a hybrid attention architecture that alternates sliding-window attention and full attention across layers. This design keeps the KV cache within manageable bounds: if every layer used full attention to handle million-scale sequences, the KV cache would require about 266 GB in BF16, making practical deployment nearly impossible. Sliding-window layers dramatically cut cache usage for most tokens, while full-attention layers preserve global context at key positions.

Official deployment guidance notes that under full context, the KV cache consumes about 250 KB per token in BF16. For long-context workloads, the memory budget needs to be planned carefully.

KV cache is the memory structure that stores key-value vectors for historical tokens during autoregressive inference, used to avoid recomputing the entire history each time a new token is generated. Its footprint grows linearly with sequence length and layer count: each layer and each token stores a pair of K/V tensors. Under full attention, the cache for million-scale sequences can reach hundreds of GB, becoming the main bottleneck for long-context deployment. Sliding-window attention lets each token attend only to tokens within a fixed window in front of it, so cache usage no longer grows unbounded with sequence length; the tradeoff is that information beyond the window can only be obtained indirectly through inter-layer transmission. Alternating sliding-window layers with full-attention layers is a common compromise: most layers use low-cost local attention, while a few layers maintain a global information pathway. Grouped-query attention (GQA) then lets multiple query heads share the same set of key-value heads, further compressing the head-dimension overhead of the KV cache — especially critical in long-context scenarios.

Adjustable Reasoning Effort: Putting the Compute-Quality Tradeoff in Developers' Hands

A notable design choice is adjustable reasoning effort. The API accepts a reasoning_effort parameter with four levels — low, medium, high, and xhigh — defaulting to xhigh, meaning the model invests more compute in chain-of-thought reasoning before giving a final answer. Lower levels reduce latency at the cost of reasoning depth.

This is similar to the effort controls Anthropic introduced in Claude Haiku 5.5, reflecting an industry trend: labs no longer offer a single fixed reasoning mode, but expose the compute-versus-quality tradeoff directly to developers.

Performance Profile: Strong Tool Calling, Weaker Single-Shot Code Generation

Meta's benchmarks position Muse Spark Open as a specialist in tool calling and agentic workflows rather than a general coding leader. On the MCP-Atlas multi-step tool orchestration benchmark, it scores 89.2, ahead of Qwen3.8's 84.5 and GLM 5.3's 84.2. On the JobBench agent evaluation suite, it scores 61.8, far ahead of comparable competitor Inkling's 31.1.

But on raw coding benchmarks the picture differs. Terminal-Bench 2.1 scores 76.4, with Kimi K3 leading at 85.0. On DeepSWE 1.1 it scores 60, versus Kimi K3's 68. The Artificial Analysis Intelligence Index is 39, behind GLM 5.3's 45 and Kimi K3's 44.

The pattern is consistent: Muse Spark Open is optimized for the orchestration layer — choosing the right tools, managing multi-step state, handling long context — rather than the single-shot code generation that Terminal-Bench measures. Whether that tradeoff makes sense depends on what you are building.

Hardware Requirements and Deployment Options

Running Muse Spark Open demands a fairly high-end hardware configuration. The BF16 checkpoint requires about 2 TB of VRAM, equivalent to a cluster of 32 H200 GPUs. The NVFP4 (4-bit) checkpoint reduces requirements to about 530 GB, validated on four NVIDIA Blackwell Ultra GPUs or eight H200s.

For teams without large GPU clusters, Meta demonstrated the model running on two Apple M5 Ultra Mac Studios (512 GB unified memory each) via Thunderbolt 5 with RDMA and the MLX framework. AMD Instinct MI355X and MI350X are supported through FP8 and MXFP4 quantization. Unsloth provides 2-bit GGUF quantization that fits into about 300 GB of system memory.

Supported inference runtimes at launch include SGLang, vLLM, TensorRT-LLM, Hugging Face Transformers, and llama.cpp.

Quantization is a family of techniques for representing weights and activations at lower bit widths to reduce memory footprint and bandwidth pressure. BF16 is a 16-bit brain floating-point format used as the training and inference baseline; NVFP4 is NVIDIA's 4-bit floating-point format promoted on the Blackwell architecture with native hardware support, so the speed penalty is small and accuracy loss is manageable; FP8 and MXFP4 are the corresponding options for AMD Instinct; the GGUF family of quantizations is commonly used for CPU/hybrid inference paths like llama.cpp, with 2-bit quantization making more aggressive tradeoffs between accuracy and memory. The lower the bit width, the more parameters fit in the same VRAM, but output quality — especially stability in long-chain reasoning and tool calling — may degrade. This is exactly why the article notes that "quantization may change benchmark performance": benchmark scores are typically measured under BF16, while most teams actually deploy the 4-bit version, and the two are not equivalent.

Speculative Decoding: Custom Draft Models for Each Framework

Meta provides two dedicated draft models for speculative decoding, one for each inference framework. DFlash V2 is designed for SGLang, proposing blocks of tokens in a single CUDA graph pass; on AMD MI355X hardware, it is claimed to achieve a 3.7x throughput improvement, with an average of 6.4 draft tokens accepted per step. DSpark is designed for vLLM, proposing up to seven tokens per pass, with an architecture specifically tuned to stay efficient at million-token context lengths, leveraging the model's sliding-window attention layers.

Speculative decoding at this scale is no easy feat — the draft model needs to be fast enough, or the runtime overhead will eat up the gains from reducing full-model forward passes. Releasing dedicated draft models per framework rather than a single general-purpose version shows that Meta tuned carefully for each runtime's batching and scheduling behavior.

CUDA graph is an NVIDIA mechanism that captures a sequence of GPU operations into a static graph on first execution and replays it later as a single graph launch, thereby eliminating per-kernel CPU launch overhead and scheduling jitter. Inference frameworks invoke many small kernels during decoding, and enabling CUDA graphs often significantly improves throughput and reduces latency variance. For the draft model to "propose blocks of tokens in a single CUDA graph pass," it means multiple forwards in the entire draft stage are merged into a single graph replay — a prerequisite for achieving high accepted length without sacrificing speed. Different inference frameworks (SGLang, vLLM) have different batching and scheduling strategies, so the draft model's shape, graph capture method, and proposal length all need separate tuning. This is precisely why Meta releases a separate draft model for each framework.

License: Open Weights, Not Open Source

Muse Spark Open is released under the Muse Model License, which is worth reading carefully before building a product. Key terms include:

• Any product using the model must prominently display a "Built with Muse Spark Open" notice.
• Companies offering the model as an API service (model-as-a-service) with gross revenue exceeding $20 million within 12 months must sign a separate written agreement with Meta.
• Fine-tuning and redistribution are permitted within these constraints.

This is consistent with the overall structure of Meta's previous Llama series — open enough for most research and product use cases, but with guardrails for large commercial API providers.

Practitioner Takeaways

Muse Spark Open is a serious option for teams building agentic systems, especially for scenarios requiring strong tool orchestration and long-context handling. Its lead on MCP-Atlas indicates it is specifically optimized for multi-step, tool-intensive workflows — an increasingly common requirement in production AI systems.

The hardware bar is genuinely high. Unless you can get access to a multi-GPU cluster or two M5 Ultra Mac Studios, you will need to use quantized versions, which may change benchmark performance. The NVFP4 checkpoint is the pragmatic starting point for most teams.

The benchmark gap on Terminal-Bench and DeepSWE is real, and if your use case is primarily single-shot code generation rather than agentic orchestration, it needs to be factored in. For that kind of workload, Kimi K3 or GLM 5.3 may still be the better choice.

What is notable about this release is not the parameter count itself, but the targeted architectural choices: hybrid attention designed for million-token context efficiency, framework-specific speculative decoding draft models, and adjustable reasoning effort. These engineering decisions reflect a model built for deployment, not just for benchmark headlines.

One-line Summary

Meta releases Muse Spark Open, an 816B-parameter open-weight MoE model focused on agentic orchestration and million-token context.

Paragraph Summary

Meta Superintelligence Labs has released Muse Spark Open, a sparse mixture-of-experts model with 816 billion total parameters and 42 billion active parameters — Meta's largest open-weight model to date. It supports a 1 million token context, using a hybrid architecture alternating sliding-window and full attention to keep the KV cache at about 250 KB per token. The model is positioned as a specialist in tool calling and agentic orchestration: it leads competitors with scores of 89.2 on MCP-Atlas and 61.8 on JobBench, but lags behind Kimi K3 on single-shot code generation benchmarks such as Terminal-Bench 2.1 and DeepSWE. The API offers four reasoning effort levels: low/medium/high/xhigh. Deployment requirements are high: about 2 TB of VRAM for BF16 and about 530 GB for NVFP4, with a demonstration option using two M5 Ultra Mac Studios. The license is open-weight rather than open source; products must display a notice, and large API service providers need a separate agreement.