NVIDIA Releases Nemotron 3.5 Super VL: A 120B Open-Source Multimodal Agent Model

NVIDIA is doubling down on the open-source model race once again. According to MarkTechPost, NVIDIA has officially released Nemotron 3.5 Super VL — an open-source multimodal MoE model with 120B total parameters — alongside two Bolt-series models targeting edge scenarios. This move continues NVIDIA's recent dense push into open weights and the agent direction, and brings a seemingly contradictory combination — large parameter count, low activation, ultra-long context, and edge deployability — into the spotlight.

Core Specs: 120B Total, 12B Active, and 1M Context

The most closely watched aspect of Nemotron 3.5 Super VL is its combination of architecture and specifications.

This is an MoE (Mixture of Experts) model with 120B total parameters, but only 12B active parameters at inference time. This means that while retaining the capacity ceiling of a large model, it compresses the actual compute overhead to a level close to that of a small-to-mid-sized dense model. For deployers, this directly determines the order of magnitude of throughput and cost.

The context length reaches 1M tokens, placing it in a fairly aggressive tier among current open-source multimodal models. Long context is especially critical for agent tasks — multi-turn tool calling, long-document understanding, and cross-file code analysis all require the model to maintain a large amount of state within a single session.

In terms of weight format, Nemotron 3.5 Super VL adopts NVFP4. This is a low-precision numerical format promoted by NVIDIA, aimed at dramatically compressing memory footprint and bandwidth requirements with almost no loss in precision.

Runs on a Single DGX Spark

The detail that truly gave this release its viral reach is this: the NVFP4 weights can run on a single DGX Spark.

The DGX Spark is NVIDIA's compact device aimed at desktop-level/personal AI development scenarios. The fact that a 120B-scale model can run on a single device of this kind shows that the coordinated optimization of quantization and sparse activation has crossed a practical threshold. For researchers and small-to-mid-sized teams that cannot afford multi-GPU clusters, this is a substantive change in operability, not just a number on a spec sheet.

MoE (Mixture of Experts) is a sparse architecture that splits a single dense network into multiple groups of "expert" subnetworks. At inference time, each token is routed to only a few of these experts, while the rest do not participate in computation. As a result, the total parameter count of the model can be far larger than the number of parameters actually used in a single forward pass.

This is precisely the source of the "120B total / 12B active" combination: the total parameter count determines the model's capacity ceiling and knowledge storage scale, while the active parameter count (also called activated parameters) determines the actual compute consumed by each inference. Once the two are separated, the model can achieve a capability ceiling close to that of a 120B dense model while paying only the compute cost of a model close to 12B. The trade-off is that memory capacity requirements are still counted by the total — all expert weights must reside in memory or be efficiently swapped in — so the bottleneck of MoE is often memory rather than compute.

NVFP4 is a 4-bit floating-point numerical format heavily promoted by NVIDIA, belonging to a subcategory of the FP4 (4-bit floating point) family. Compared with integer quantization formats such as INT4, block floating point allows exponents to be shared within the same data block, thereby preserving a certain dynamic range even at extremely low bit widths and reducing precision loss from quantization.

Its core benefits are twofold: memory usage drops to roughly one quarter of FP16, and the bandwidth pressure of moving weights from memory to compute units is simultaneously greatly relieved — for inference, the latter is often more critical than the former, because the decoding stage of large models is usually memory-bound rather than compute-bound. However, the performance of low-precision formats depends heavily on whether the hardware natively supports the corresponding matrix operation instructions, which is also why this format is deeply tied to NVIDIA GPUs.

DGX Spark is a compact desktop AI computing device launched by NVIDIA, aimed at individual developers, researchers, and small teams, positioned between high-end workstations and cloud clusters. Its significance lies not in absolute compute power, but in compressing the scale of models that previously had to rely on multi-GPU servers to load into a machine that can sit on a desktop.

A 120B-scale model being able to be carried by a single device of this kind relies not on a single technology, but on the superposition of several optimizations: MoE compresses active compute to the 12B level, NVFP4 compresses weight memory to about one quarter of FP16, and on top of that there is KV Cache management for long context. Only through the coordination of these three does a "large model on the spec sheet" descend to the level of an "affordable single machine." For small-to-mid-sized teams lacking cluster resources, this kind of lowering of the threshold is often more practically valuable than pure leaderboard scores.

Multimodal + Agent: Positioning Rather Than Stacking Specs

Judging from the naming and specifications, Nemotron 3.5 Super VL is positioned as a "multimodal agent model," not a general-purpose chat model.

Multimodal capability means it can handle non-text inputs such as images, which is a prerequisite for agents to operate in real environments — reading screens, understanding interfaces, and parsing charts all fall into this category. Combined with 1M context and the low activation cost of MoE, it is better suited to Agent workflows that require continuous perception and planning, rather than single-turn question answering.

NVIDIA's choice to open-source the weights also reflects ecosystem considerations. Open-source models can attract developers to adapt and fine-tune on its toolchain and hardware, which in turn strengthens NVIDIA's software-hardware lock-in on the inference side.

Bolt Edge Models: Pushing Agents to the Device Side

Alongside Super VL, two Bolt edge models were also released.

Although official details are limited, the "edge" label itself conveys the direction: pushing agent capabilities down to local devices and reducing dependence on cloud inference. This echoes Super VL's ability to run on a single DGX Spark — NVIDIA is simultaneously laying out at two levels: "desktop-level large models" and "device-side small models."

The real-world constraints on edge models are more severe: power consumption, memory, and heat dissipation are all hard ceilings. Whether the Bolt series can retain usable multimodal and tool-calling capabilities under limited resources is a key point worth watching going forward.

A Few Sober Judgments

The specs are impressive, but several questions currently have no answers.

First, the actual performance of 12B active parameters on complex multimodal reasoning tasks needs independent evaluation and verification; the relationship between parameter count and capability has never been linear.

Second, the availability of NVFP4 weights on non-NVIDIA hardware. Low-precision formats are often strongly tied to specific hardware, which may affect the actual scope of use by the open-source community.

Third, the effective attention quality of 1M context on long sequences, and whether inference latency deteriorates linearly with context, are both key variables for engineering deployment.

The value of Nemotron 3.5 Super VL may not lie in refreshing a single-point metric, but in pushing "a large model can run on a single machine" from a slogan toward a verifiable configuration. The open weights have already been released, and the real test lies in developers' hands-on evaluations.

When evaluating such models, there are several often-confused metrics worth explaining separately. Effective context refers to the length over which a model can still reliably use distant information on long sequences, and it is not the same concept as the "maximum supported context length" — attention mechanisms generally decay on ultra-long sequences, and the industry commonly uses "needle-in-a-haystack"-style retrieval tests to measure the truly usable range.

Time to First Token (TTFT) and decoding throughput also need to be viewed separately: the former is mainly determined by the prefill stage and grows significantly with input length; the latter is determined by token-by-token decoding and is usually more constrained by memory bandwidth. Under 1M context, the computational load of prefill grows very quickly with length, and engineering methods such as cache reuse and chunked processing directly determine whether responses are usable. Therefore, "supports 1M" is only a declaration of the capability ceiling, and the actual experience depends on the latency curve — which is also the part independent evaluations should focus on most.

One-line Summary

NVIDIA releases Nemotron 3.5 Super VL, a multimodal MoE model with 120B total parameters and 12B active, runnable on a single DGX Spark.

Paragraph Summary

NVIDIA has released the open-source multimodal MoE model Nemotron 3.5 Super VL, with 120B total parameters and only 12B activated at inference, supporting a 1M token context. Its weights use the low-precision NVFP4 format and can run on a single DGX Spark desktop device, significantly lowering the deployment threshold. It is positioned as a multimodal agent model for long-horizon tasks requiring continuous perception and planning; two Bolt edge models were also released, laying out device-side agent capabilities. While acknowledging its specification breakthroughs, the article offers sober judgments: the real performance of 12B activation on complex multimodal reasoning, NVFP4 compatibility with non-NVIDIA hardware, and effective attention quality and latency under 1M context all require independent evaluation and verification.