Nemotron 3.5 Dual-Model Release: Hybrid Mamba-Transformer Architecture Speeds Up Agentic AI

NVIDIA's Nemotron 3.5 Super and Nemotron 3.5 Lightning bring together the words "open" and "efficient" on top of both Mixture of Experts (MoE) and hybrid Mamba-Transformer architectures. Both models target reasoning, long-context understanding, and agentic workloads, but they take two complementary paths to scale.

How the Two Models Divide the Work

Super is a 120B-parameter, 12B-active multimodal model; Lightning is a 30B-parameter, 3B-active text-only model. Both emphasize "efficient deployment," with the difference being that Super handles multimodal and complex agentic tasks, while Lightning targets lighter-weight text scenarios.

This "large-and-sparse + small-and-sparse" combination is nothing new, but what makes Nemotron 3.5 notable is that both models share the same training philosophy: large-scale distillation, long-context training, and native speculative decoding. Super adds Indic continued pretraining and agentic software engineering mid-training on top—a key step in its evolution from a text-only base into a multimodal agent.

What MoE and Hybrid Mamba-Transformer Mean

Mixture of Experts (MoE) is a sparse activation architecture: the model has a large total parameter count, but each forward pass only routes to a small number of "expert" subnetworks, so the active parameters are far fewer than the total. Super's 120B/12B and Lightning's 30B/3B both follow this structure, with deployment costs closer to the active parameter scale than the total parameter scale.

Mamba, meanwhile, is the state space model (SSM) route, replacing standard attention's quadratic complexity with a recurrent state that grows approximately linearly with sequence length, making memory and compute overhead more controllable in long-context scenarios. Hybrid Mamba-Transformer interleaves Mamba layers with a small number of attention layers, balancing long-sequence efficiency with the precise retrieval capabilities attention excels at—this is the architectural foundation for both models' emphasis on "long-context understanding."

Post-Training Pipeline: From SFT to Multi-Teacher On-Policy Distillation

Nemotron 3.5's post-training pipeline covers both supervised fine-tuning and reinforcement learning, with the RL portion using both RLVR (Reinforcement Learning with Verifiable Rewards) and RLHF. Super additionally introduces dedicated teacher training and multi-teacher on-policy distillation (MOPD).

The intent behind this design is clear: to "transfer" the base model's capabilities into agentic behavior, enabling the model to reason across text, images, video, and interactive tool environments. In other words, Super is no longer just a model that answers questions, but an agent that can keep working within a tool-calling loop.

RLVR, RLHF, and On-Policy Distillation

RLVR (Reinforcement Learning with Verifiable Rewards) means rewards come from automatically verifiable outcomes—such as whether a math answer is correct or whether code passes unit tests—without human preference labeling, making it suitable for reasoning and software engineering tasks. RLHF relies on human preference data to train a reward model and is better at aligning tone, helpfulness, and safety. Using both means optimizing for both "getting it right" and "getting it right well."

Multi-teacher on-policy distillation (MOPD) means multiple teacher models with different capabilities score the student model's currently generated trajectories (on-policy) in real time and provide supervision signals, rather than doing offline distillation only on a fixed dataset. This allows the student model to correct errors along its own actual distribution—a common technique for transferring base capabilities to agentic behaviors such as multi-turn tool calling.

Efficiency Data: Fewer Active Parameters, Higher SWE-bench Pro Score

Several data points from the original are worth highlighting separately. Indic continued pretraining improved Indian-language benchmarks by up to 9.82 percentage points; agentic mid-training enhanced software engineering capabilities as well as robustness across agent harnesses.

The most striking comparison: after post-training, Super reaches 51.78% on SWE-bench Pro, surpassing Nemotron 3 Ultra, while having roughly only 1/4.6 of the latter's active parameters. This shows that under the MoE architecture, the traditional mapping between active parameter scale and task performance is being rewritten by training pipelines and data strategies.

Inference-Side Optimization: Speculative Decoding and NVFP4 Quantization

On deployment efficiency, MTP boosting during post-training improved speculative decoding's acceptance length by up to 21.8% under probabilistic draft sampling. NVFP4 quantization achieved 99.62% median accuracy recovery relative to BF16, meaning the accuracy loss from low-precision deployment has been compressed to a very small range.

Inference optimization reaches up to 4,856 output tokens/s per GPU on GB200 while maintaining a per-user experience floor of at least 50 tokens/s. These numbers are especially critical for agentic scenarios—agent workloads are typically multi-turn, long-output, and concurrency-intensive, so throughput and per-user latency must hold simultaneously.

Speculative Decoding, MTP, and NVFP4

Speculative decoding uses a lightweight draft model to first propose several tokens, which the large model then verifies in parallel in one pass; accepted tokens skip per-token generation, speeding things up without changing the output distribution. "Acceptance length" is the average number of draft tokens accepted each time—the core metric. The original's MTP (Multi-Token Prediction) boosting refers to using the model's built-in multi-token prediction head as the draft source, while probabilistic draft sampling makes drafts more diverse to improve hit rate.

NVFP4 is the 4-bit floating-point format introduced with NVIDIA's Blackwell platform, using micro-block scaling to preserve dynamic range at extremely low bit widths. GB200 is based on the Blackwell architecture and natively supports FP4 operations, so quantization gains translate directly into throughput. The "99.62% median accuracy recovery" in the text refers to the median proportion of task scores retained relative to the BF16 baseline after quantization.

Test-Time Scaling and Parallel Agents

The original also mentions exploring test-time scaling through parallel agent execution and self-verification. This continues a recent trend: capability gains no longer come only from piling on parameters during training—compute orchestration, multi-path sampling, and self-checking at inference time can also deliver gains. For agentic AI, this path may offer better cost-effectiveness than continuing to scale up the base model.

Common Forms of Test-Time Scaling

Test-time scaling means investing more compute at inference time to gain quality, with typical approaches including extended chain-of-thought, multi-path sampling with voting (such as best-of-N), tree search, and the parallel agent execution and self-verification mentioned in the text—having multiple agent instances attempt the same task in parallel, then having the model or a verifier filter and correct the results. The cost is multiplied latency and expense, so it is usually paired with high-throughput inference optimization and only pays off on medium-to-high-value tasks.

What It Means for Developers

The open nature of the Nemotron 3.5 series, combined with the hybrid Mamba-Transformer architecture, offers new options for self-deployment scenarios. Mamba's computational efficiency advantage on long contexts, combined with MoE's sparse activation, brings 120B-class models' inference costs closer to the deployment threshold of past small models.

However, note that Super's multimodal and agentic capabilities depend heavily on its post-training pipeline, and there is currently no public evidence as to whether community fine-tuning can reproduce these gains. Lightning, as a 3B-active text model, is better suited as a lightweight execution node in an agent pipeline rather than a primary inference engine.

One-line Summary

NVIDIA releases Nemotron 3.5 Super and Lightning, featuring a MoE hybrid Mamba-Transformer architecture for efficient agentic inference.

Paragraph Summary

NVIDIA introduces Nemotron 3.5 Super (a 120B-total-parameter, 12B-active multimodal model) and Nemotron 3.5 Lightning (a 30B-total-parameter, 3B-active text-only model), both using a MoE hybrid Mamba-Transformer architecture and sharing a training philosophy of large-scale distillation, long-context training, and native speculative decoding. The post-training pipeline covers SFT, RLVR, and RLHF; Super additionally introduces teacher training and multi-teacher on-policy distillation, along with Indic continued pretraining and agentic software engineering mid-training. Results show that after post-training, Super reaches 51.78% on SWE-bench Pro, surpassing Nemotron 3 Ultra, with only about 1/4.6 of its active parameters—showing that the traditional mapping between active scale and task performance is being rewritten by training pipelines.