NVIDIA Nemotron 3.5 Super VL: A Deep Dive into the Mamba-MoE Hybrid Architecture

The competitive focus in multimodal foundation models is shifting from pure text to high-density visual streams. NVIDIA recently integrated Nemotron 3.5 Super VL 120B A12B into its official repositories — a vision-language model with roughly 120B total parameters that activates only 12B per token. Its core breakthrough lies in weaving together three technologies that previously evolved independently: replacing the pure-attention decoder with Mamba state-space blocks, adopting an in-house RADIO vision encoder, and natively supporting NVFP4 4-bit quantization on the Blackwell architecture.

Official Integration and Engineering Deployment

What sets this release apart from previous ones is that NVIDIA simultaneously plugged the model into its own software supply chain, rather than stopping at the paper level.

According to PR 3801 in the NeMo AutoModel repository, multimodal support for this model was unified and integrated, incorporating 512 routable experts, a RADIO vision tower, and a normalized projection layer. Megatron-Bridge received the complete recipes for pretraining, SFT, and PEFT, targeting servers equipped with H100 and GB200. The TensorRT-LLM inference engine added support for continuous video input and the NVFP4 compressed format, with verified throughput exceeding 185 tokens/s on the Blackwell B200. The LangChain ecosystem also received an official connector under the model identifier nvidia/nemotron-3.5-super-vl-120b-a12b, allowing engineering teams to plug video analytics and visual audit pipelines directly into enterprise workflows.

Unlike closed cloud offerings, this model targets private cloud (VPC) and professional workstation deployments, offering a clearer path for data sovereignty and compliance governance.

Microstructure of the Decoder and Vision Tower

The key to understanding this model's efficiency lies in its hybrid decoder topology and visual perception subsystem.

Nemotron-H Decoder: Mamba SSM + Sparse MoE

In a pure-attention Transformer, processing video frames and high-resolution images causes KV Cache consumption to balloon dramatically — each video frame decomposes into hundreds of visual patches, quickly exhausting VRAM.

Nemotron 3.5 Super VL sidesteps this bottleneck with the Nemotron-H hybrid decoder:

• Mamba State-Space Layers: Interleaved throughout the network, with linear complexity O(N) with respect to sequence length. Visual and text tokens are processed as a continuous state stream, discretized via the zero-order hold (ZOH) formulation, filtering out background noise while keeping memory costs constant.
• 512-Expert MoE Layers: The model splits representational capacity into 512 sparse expert paths, dynamically routing to activate only 22 experts at inference time. That means of the 120B total parameters, only 12B participate in computation per step (A12B), delivering low latency and high domain specialization.
• Multi-Token Prediction (MTP): Inheriting shared-weight infrastructure to emit multiple tokens ahead, accelerating decoding through runtime speculative verification.

Mamba State-Space Model (SSM) is a class of sequence modeling architectures derived from the state-space equations of classical control theory. Unlike the Transformer self-attention mechanism, which computes correlations between each token and all other tokens in the sequence (O(N²) complexity), Mamba advances through the sequence via a learnable recurrent latent state, depending only on the previous step's state at each step, thus achieving O(N) linear complexity and constant inference memory. Its key innovation is the "selective scan": the parameters of the state transition matrix are dynamically generated from the input, enabling the model to selectively remember or forget information like a gating mechanism. For ultra-long sequences such as video, Mamba avoids the problem of KV Cache growing unbounded with frame count — this is the underlying reason Nemotron 3.5 Super VL can stably ingest million-token contexts.

Zero-Order Hold (ZOH) Discretization is a common method for converting continuous-time state-space equations into computable discrete recursions: assuming the input stays constant during the sampling interval, the discrete-step A and B matrices are derived accordingly. This mathematical step lets continuous signals (such as video frame streams) be consistently modeled in discrete token space, while acting as a low-pass filter to suppress high-frequency noise.

RADIO Vision Tower: Dynamic Resolution and Temporal Embedder

The visual input side relies on a 0.65B-parameter RADIO encoder, which addresses three shortcomings of traditional vision encoders:

• Adaptive Multi-Scale Resolution: Large images are cut into intelligent mosaics without destroying aspect ratio, preserving small fonts in technical diagrams, tiny text on receipts, and geometric details in GUI interfaces.
• Dedicated Video Temporal Embedder: Rather than treating video as a sequence of isolated images, a temporal embedding branch encodes inter-frame timing and transition speed, giving tracking of object motion and scene changes semantic continuity.
• Projection Layer with LayerNorm: The visual vectors produced by RADIO are projected into the Nemotron-H embedding space via a fully connected layer plus LayerNorm, suppressing gradient explosion in long multimodal streams.

RADIO (Reduced-bias Adaptive Dynamic-resolution Image-to-text encOder) is NVIDIA's in-house vision encoder family, whose core design goal is to reduce the inductive biases of traditional CLIP-style encoders. Traditional vision encoders typically force images to a fixed resolution (such as 224×224 or 336×336), which destroys small fonts and fine-grained geometric structures in text-dense images. RADIO adopts a "resolution-agnostic" training paradigm: it first learns general visual representations on large-scale unsupervised data, then decomposes images of arbitrary aspect ratio into semantically coherent patch sets through adaptive multi-scale splitting (mosaic), while preserving global context summaries. This allows a single encoder to serve multiple downstream tasks — classification, detection, segmentation, and multimodal alignment — which is why it was chosen as the Nemotron vision tower. At 0.65B parameters, it is lightweight among vision encoders, leaving room in the overall inference budget.

Real Benchmark Battle: Head-to-Head with Contemporary Frontier Models

The quality of a 120B vision-language model must be tested against strict comparisons with contemporary frontier systems. Nemotron 3.5 Super VL goes head-to-head with Claude Fable 5.1, GPT-6 Astra, DeepSeek^Ad 4.1, and Gemini 3.8 Flash Cyber on major multimodal evaluations:

Benchmark / Capability	Nemotron 3.5 Super VL	Claude Fable 5.1	GPT-6 Astra	DeepSeek 4.1	Gemini 3.8 Flash Cyber
Video-MME (video temporal understanding)	77.4%	78.2%	76.8%	74.5%	76.1%
MMMU Pro (advanced multimodal reasoning)	71.8%	73.4%	74.1%	70.2%	71.5%
MathVista (visual math and charts)	79.2%	80.5%	81.2%	78.0%	78.9%
DocVQA (complex document reading)	94.6%	95.1%	94.8%	93.2%	94.0%
B200 throughput (tokens/s)	185 tok/s	58 tok/s	62 tok/s	115 tok/s	120 tok/s
FP4 inference memory footprint	~60 GB	~280 GB	~310 GB	~160 GB	~140 GB
Weight license	Open weights	Proprietary	Proprietary	Open weights	Proprietary

Metric Interpretation

Continuous Video Understanding (Video-MME): The deterministic accuracy of 77.4% clearly surpasses DeepSeek 4.1 (74.5%) and GPT-6 Astra (76.8%), trailing Claude Fable 5.1 by only 0.8 percentage points. The dedicated temporal embedder and Mamba's linear efficiency allow long-video ingestion without prematurely dropping frames, delivering above-average stability.

Technical Document Reading (DocVQA): It achieves 94.6% on table extraction, financial reports, and architectural drawing tasks, reaching a technical tie with the strongest closed commercial offerings.

Generation Speed and NVFP4 Throughput: Nemotron's killer feature is throughput. Leveraging the compact MoE architecture of 12B activated parameters and optimized NVFP4 kernels in TensorRT-LLM, it generates 185 tokens/s on a single Blackwell B200 node — up to 3.2× faster than dense competitors.

Multimodal Pipeline from Video Ingestion to TensorRT-LLM Output

Piping real-time visual streams into enterprise systems requires an asynchronous data pipeline precisely calibrated between accelerators and host memory:

• Dynamic Sampling and Preprocessing: Video streams or high-resolution images are cut into dynamic patches by RADIO, bypassing the classic destructive 224x224 scaling and giving text-dense regions more patch quota.
• Projection and Temporal Alignment: Visual tensors pass through the temporal embedder to add temporal position vectors, then enter the normalized linear projection layer.
• Mamba SSM Backbone Absorption: Visual tokens are processed by Mamba state-space layers without bearing the quadratic cost of self-attention, while latent-state memory continuously retains global scene features.
• MoE Expert Activation and Output: The MoE layers activate the 22 experts relevant to the query domain, producing structured JSON, code, or natural-language responses with sub-50ms latency per token.

Production Deployment: NVFP4 Quantization and Hardware Planning

In enterprise deployments, the most attractive aspect of Nemotron 3.5 Super VL is the substantial compression of infrastructure costs.

Resource Planning on Blackwell Nodes

Unlike 120B dense models that need eight 80GB GPUs to hold weights in BF16 half precision, it is optimized for the 4-bit NVFP4 format: after compression, the weights occupy roughly 60 GB of VRAM, fitting entirely onto a single high-capacity professional GPU (such as the 192GB Blackwell B200), or comfortably spread across two smaller enterprise-grade GPUs / Blackwell fifth-generation tensor cores perform tiny floating-point matrix operations with dynamic exponent representation, avoiding the precision degradation common to older INT4 quantization. Paired with local TensorRT-LLM execution in a private cloud, this converts variable per-token inference budgets into amortizable fixed infrastructure investment.

NVFP4 is a 4-bit floating-point data format introduced by NVIDIA on the Blackwell architecture, fundamentally different from traditional INT4 integer quantization. INT4 uses uniform quantization steps, making it extremely sensitive to outliers and long tails in weight distributions, easily causing sharp precision drops; NVFP4 instead uses a "block-level shared exponent + micro-scaling factor" scheme: every 16 values share one FP8 scaling factor, and each element within a block is represented with a 4-bit mantissa plus the shared exponent, preserving floating-point dynamic range at extremely small bit widths. Combined with FP4 matrix multiply-accumulate instructions natively supported by fifth-generation tensor cores, the model can compress weights and activations to 4 bits with almost no loss in inference quality. This is the hardware foundation that lets a 120B model compress to about 60GB of VRAM and achieve 185 tokens/s throughput on the B200. By comparison, the same model at BF16 precision requires roughly 240GB of weight storage, typically requiring multi-GPU tensor parallelism to fit.

Cloud Trial and Open Weights

Teams that want to prototype before purchasing hardware can try the model on the NVIDIA Build platform: ready-made endpoints compatible with the OpenAI standard library or LangChain connectors are provided. The official Hugging Face repository will provide unified checkpoints for BF16 and NVFP4, along with conversion scripts and NeMo Framework orchestration recipes.

Frequently Asked Questions

What are the main differences from the previous-generation Nemotron 3 Super?

The core difference lies in input modality and the perception tower. Nemotron 3 Super is a pure-text 120B model; 3.5 Super VL adds a 0.65B RADIO vision tower, a video temporal embedder, and refined cross-modal alignment, enabling the network to handle text, video, and high-resolution images simultaneously, and adopts a Mamba-MoE hybrid decoder.

Can it run on an ordinary personal computer GPU?

In its native state, it's difficult. Although NVFP4 compression brings weights down to about 60GB of VRAM, that still exceeds the capacity of traditional consumer GPUs. Local execution requires a workstation equipped with high-memory-density professional cards, or an enterprise server with inference-specific GPUs.

Why adopt a Mamba-MoE hybrid architecture instead of a traditional Transformer?

Pure-attention models face O(N²) quadratic costs when processing images and video, because each visual element generates dozens of patches, overwhelming the KV Cache. Alternating Mamba blocks (linear O(N) complexity) with sparse MoE layers allows Nemotron 3.5 Super VL to handle context sequences up to 1 million tokens while keeping memory consumption predictable and activating only 12B parameters per generated token.

How does it perform on enterprise document extraction and code reading tasks?

It performs at a high level on document visual reading (DocVQA 94.6%) and chart math reasoning (MathVista 79.2%). The RADIO tower preserves the clarity of small fonts and complex table layouts, making it particularly suitable for contract process automation, GUI agent screen-stream analysis, and structured source-code auditing.

One-line Summary

NVIDIA releases Nemotron 3.5 Super VL 120B A12B, achieving efficient multimodal inference with a Mamba-MoE hybrid architecture and RADIO vision tower

Paragraph Summary

NVIDIA has integrated Nemotron 3.5 Super VL 120B A12B into its official repositories — a vision-language model with roughly 120B total parameters that activates only 12B per token. Its core breakthrough fuses three technologies: replacing the pure-attention decoder with Mamba state-space layers for linear complexity and constant memory; adopting an in-house 0.65B RADIO vision encoder supporting adaptive multi-scale resolution and a dedicated video temporal embedder; and natively supporting NVFP4 4-bit quantization on Blackwell, compressing VRAM to about 60GB. The model is integrated into the NeMo AutoModel, Megatron-Bridge, TensorRT-LLM, and LangChain ecosystems, targeting private cloud and professional workstation deployments. It reaches 77.4% on Video-MME and 94.6% on DocVQA, with 185 tokens/s throughput on B200 — up to 3.2× faster than dense competitors.