Mistral Large 4: A Deep Dive into Europe's First Trillion-Parameter Open-Source Model

European AI lab Mistral AI has released a public preview of Mistral Large 4, a multimodal Mixture-of-Experts (MoE) model with 1.05 trillion parameters, internally codenamed "Le Chonk." More notably, Mistral has promised to open the model weights by the end of the month, making it the largest open-weight model ever released by a European lab.

For teams looking to run frontier-scale models on their own infrastructure, this release deserves careful scrutiny.

Architecture: Fine-Grained MoE with a Standalone Vision Encoder

Mistral Large 4 uses a sparse Mixture-of-Experts architecture with a total parameter count of 1.05 trillion, but only 52 billion parameters activated per token. What makes this "fine-grained" MoE design distinctive is that the router selects from a larger pool of smaller experts, rather than routing tokens to a handful of large expert modules. Finer-grained routing control means each token can be assigned to more closely matched specialized capacity, which Mistral says improves both throughput and task specialization.

The model also comes with a standalone 1.6-billion-parameter vision encoder for processing image inputs. This encoder converts visual content—charts, engineering drawings, satellite imagery, PDFs—into representations the language backbone can reason over. This means a single model can handle instruction following, multi-step reasoning, and multimodal understanding without switching between multiple systems.

On the training side, the model was trained from scratch in Mistral's own European data centers using 3,800 NVIDIA Grace Blackwell GPUs, rather than renting cloud compute. The documented context window is 1 million tokens, though the preview API from some third-party providers is currently capped at 524,000 tokens.

Mixture-of-Experts (MoE) architecture works by splitting a single dense feedforward network into multiple parallel "expert" subnetworks, then using a lightweight router to dynamically select which experts to activate for each token. Since each token only passes through a small number of selected experts, the model can have an extremely large total parameter count while keeping the per-forward-pass compute low—this is precisely why "total parameters" and "activated parameters" are separated. Traditional MoE tends to use a few large experts, whereas Mistral Large 4's "fine-grained" design uses a larger pool of smaller experts, enabling more precise routing decisions so that each token matches more closely to specialized capacity.

This design introduces trade-offs on two fronts: on one hand, finer granularity typically improves the model's expressiveness and task specialization; on the other hand, the finer the experts, the harder it is to balance routing load, making it prone to the problem of "a few experts being overused while the rest are undertrained," which requires a load balancing loss during training to constrain. Additionally, MoE places extra demands on inference infrastructure—different tokens hit different experts, so VRAM must hold all expert weights, which is the direct reason trillion-parameter MoE has such a high barrier to self-hosting.

Large-Scale Reinforcement Learning: A Post-Training Pipeline That Adapts Dynamically

Mistral describes its post-training method as "RL at scale"—a reinforcement learning pipeline that dynamically adjusts as the model's capabilities change. Rather than running a fixed RL curriculum, the system continuously adjusts the task distribution so that training stays at the frontier of the model's current abilities.

At the current compute scale (roughly 3,000 GPUs), a single training run produces about 33 billion tokens per day, of which about 16 billion are trainable completion tokens (after filtering and masking). The pipeline is optimized for long trajectories, supporting million-token rollout budgets across multiple compression steps—which is key to the model's ability to learn from complex multi-turn agent tasks.

Mistral also revealed that the RL run behind the preview is still ongoing, and the model is expected to keep improving before weights are released. This level of transparency about release timelines is uncommon in the industry.

Rollout in "RL at scale" refers to the process during reinforcement learning training where the model generates a complete solution sequence for a given prompt. In agent scenarios, a single trajectory may contain dozens or even hundreds of steps of tool calls, observations, and reasoning, so a single rollout can consume hundreds of thousands or even millions of tokens. The "million-token rollout budget" and "across multiple compression steps" mentioned by Mistral mean the system needs to maintain trajectory coherence under ultra-long contexts and compress intermediate states to reduce VRAM and compute pressure.

The so-called dynamic adjustment of task distribution corresponds to the idea of curriculum learning: if tasks are too hard, the reward signal is sparse and little is learned; if too easy, there is no gradient value. The system continuously anchors difficulty at the "frontier" of the model's current abilities, keeping reward variance in the most informative range. The 33 billion tokens per day, of which about 16 billion are trainable completion tokens, indicates that nearly half of generated content is discarded after filtering and masking—these typically include format errors, invalid tool calls, or trajectories that fail validation. The filtering ratio itself is also an important metric for measuring pipeline efficiency and reward model quality.

Benchmarks: Coding, Security, and Multimodal Performance

Mistral published test results across several domains; here are the core figures:

Agentic coding:

• DeepSWE v1.1: 61.7%
• SWE-Atlas-QnA: 59.4%
• Terminal-Bench 4: 28.3%
• Composite coding agent index: 49.8% (ahead of DeepSeek^Ad V4 Pro and Qwen3.8 Max)

Agentic workflows:

• AutomationBench (657 business processes): 59.9%
• AA-Briefcase (long-horizon knowledge work): 1393 Elo

Cybersecurity:

• Cybench (40 CTF challenges): 93%
• Vulnerability reproduction and patching tests: 82%
• B3 AI safety benchmark (attack resistance): 93.3%

Multimodal:

• Dense 200 visual grounding: 42% (GPT-6 Astra is 41%)

In a blind evaluation organized by Surge AI, professional annotators gave the preview a 3.74/5, ranking second among five models—behind Claude Opus 5 (4.22) and ahead of GLM-5.3, Kimi K3, and GLM-5.2. Independent evaluator Artificial Analysis gave it an intelligence index of 38 in reasoning mode, a substantial improvement over the 9 scored by the non-reasoning Mistral Large 3. Open-source models from Chinese labs still lead on this index (GLM-5.3 at 45, Kimi K3 at 44), but Mistral Large 4 ranks first among open-source models developed outside China.

Cybersecurity: A First-Class Capability

One deliberate design choice in Mistral Large 4 is treating cybersecurity as a core capability rather than an add-on. The model is being red-teamed by cybersecurity agencies and government departments before weight release, and Mistral also offers vetted defenders a version with reduced refusals and expanded cyber capabilities.

This strategy has practical considerations: in incident response and vulnerability research, provider-level refusals block legitimate work. A model that can be self-hosted under an organization's own policies and won't refuse to analyze malware or reproduce known CVEs is fundamentally different from a hosted API with conservative safety filters. The Artificial Analysis cyber index ranks Mistral Large 4 in the global top five, leading among open-source models developed outside China.

Red teaming refers to the process in which independent security researchers or agencies actively try to induce harmful outputs, bypass safety guardrails, or discover capability boundaries. Having a model red-teamed by cybersecurity agencies and government departments before weight release is a common practice for building trust in open-weight models—because once weights are public, any safety filter can be bypassed locally, and security responsibility shifts from the provider to the deployer.

"Reduced-refusal / uncensored variants" are a compromise aimed at defenders in recent years: the model no longer categorically refuses requests such as analyzing malware, reproducing known CVEs, or writing exploit code, but instead leaves it to vetted organizations to manage according to their own policies. The controversy with such variants is that they also lower the barrier for attackers, so they are typically only distributed to security vendors, government agencies, and critical infrastructure operators that pass qualification review. For security teams, the practical value lies in the incident response time window—analysis work during an attack cannot wait for a hosted API's review process. This also explains why Mistral positions cybersecurity as an architectural choice rather than a filter bolted on afterward.

Pricing and the Self-Hosting Barrier

Preview API pricing is $1.36 per million input tokens and $4.18 per million output tokens (with a 50% discount for batch calls). Mistral also offers a European regional endpoint, api.eu.mistral.ai, to meet data residency requirements, priced at 1.1x standard pricing.

The hardware barrier for self-hosting is quite high. At 8 bits per parameter, 1.05 trillion parameters require about 1TB of memory, not including KV cache and runtime overhead. Full-precision inference requires a multi-node GPU cluster; quantized deployment can substantially reduce requirements. Mistral has not yet announced a license for the weights.

KV cache (Key-Value cache) is the VRAM overhead incurred during autoregressive generation to avoid recomputing attention: each time the model generates a token, it needs to cache the key-value pairs of all previous tokens. Its size is proportional to the number of layers, attention heads, head dimension, sequence length, and batch size, and at a 1-million-token context it can easily reach hundreds of GB, often exceeding the model weights themselves. So the "about 1TB of memory" given in the article is only an estimate for the weights portion.

Quantization refers to representing weights with lower bit widths, commonly 8-bit (INT8), 4-bit (INT4), or even lower. 8-bit quantization can roughly halve VRAM requirements, while 4-bit compresses further but introduces accuracy loss, and not all operators can execute efficiently at low bit widths. Moreover, quantizing MoE models is more complex than dense models: weight distributions vary significantly across experts, and a uniform quantization granularity can easily degrade some experts, typically requiring finer per-expert or per-channel scaling. Practical deployment also needs to consider partitioning strategies such as tensor parallelism and expert parallelism, as well as high-speed inter-node interconnect bandwidth—together these determine the true cost of self-hosting.

What It Means for Practitioners

The open-weight commitment matters. Most trillion-parameter models are closed APIs. Mistral Large 4 choosing open weights—even if the license hasn't been announced yet—means organizations can audit the model, fine-tune it, and run it on private infrastructure. This has a fundamentally different risk profile from a hosted API.

The RL training pipeline is the real story. The benchmark numbers are competitive, but the more interesting detail is the training infrastructure: 33 billion tokens per day, long-trajectory rollouts, and a pipeline that dynamically adjusts with the model's capabilities. This post-training mechanism is what distinguishes frontier labs from everyone else, and Mistral is building this capability in Europe.

Multimodality is now table stakes. The vision encoder is not a separate product but part of the base model. For teams building document processing, satellite image analysis, or engineering workflow tools, a single model handling both text and images can significantly simplify the stack.

The self-hosting hardware barrier cannot be ignored. At 8-bit precision, 1TB of memory is not a single-server deployment for most organizations. Quantized versions will help, but until smaller distilled variants appear, teams should be prepared for multi-node deployment or managed inference.

Mistral Large 4 is a technically serious release: trillion-parameter MoE, a fine-grained routing architecture, a dedicated vision encoder, and a reinforcement learning pipeline designed for long-horizon agent tasks. Weights are expected to be released by the end of the month, at which point full architecture details and the post-training methodology will also be made public.

One-line Summary

Mistral releases Mistral Large 4, a 1.05-trillion-parameter open-weight MoE model focused on fine-grained routing and large-scale RL.

Paragraph Summary

Mistral AI has released a public preview of Mistral Large 4, a multimodal Mixture-of-Experts model with 1.05 trillion total parameters and 52 billion activated per token, with a promise to open the weights by the end of the month, making it the largest open-weight model from a European lab to date. The model uses a fine-grained MoE architecture with a standalone 1.6-billion-parameter vision encoder, was trained from scratch on 3,800 NVIDIA Grace Blackwell GPUs, and has a documented context window of 1 million tokens. Post-training uses a dynamically adjusted "RL at scale" pipeline producing about 33 billion tokens per day, supporting long-trajectory agent tasks. On benchmarks, it scores 49.8% on the coding agent index, 93% on the Cybench cybersecurity test, and 38 on the Artificial Analysis intelligence index, ranking first among open-source models developed outside China. Cybersecurity is designed as a core capability, with reduced-refusal variants offered to defenders. Self-hosting requires about 1TB of memory at 8-bit precision, putting it in multi-node cluster territory, and the weight license has not yet been announced.