Running Nemotron 3.5 Super VL Locally: 128GB of RAM Matters More Than Blackwell

NVIDIA's Nemotron 3.5 Super VL is a 120B total-parameter, 12B activated-parameter vision-language MoE model. The officially promoted NVFP4 4-bit format can only be natively executed on Blackwell, which has led many people to assume that local deployment requires upgrading to a new card. But when you put the model card, the quantization conversion repositories, and Unsloth's documentation side by side, the conclusion is exactly the opposite: what actually holds you back is memory capacity, not GPU architecture.

What Exactly Is Nemotron 3.5 Super VL

According to the official model card, this is not just a vision extension—it is the 3.5-generation Super-tier model itself. It uses a Mamba-2/Transformer hybrid MoE architecture, with 512 routed experts per MoE layer; on the vision side it pairs a C-RADIO encoder and connector, handling text, JPEG/PNG images, and MP4 video up to 2 minutes long; the context ceiling reaches 1M tokens, and it natively supports Multi-Token Prediction for speculative decoding.

One detail worth noting: there is no separate text-only "Nemotron 3.5 Super"—the VL model is 3.5 Super. Its backbone was initialized from Nemotron 3 Super Base, distilled 1T tokens from Nemotron 3 Ultra Base, extended to 1M context, then went through 67B tokens of software-engineering mid-training, and only then received vision training. So its true predecessor is the text-only Nemotron 3 Super released in March.

What MoE (Mixture of Experts) and Activated Parameters Mean

An MoE architecture replaces the dense feed-forward layers in a Transformer with multiple parallel "expert" sub-networks, and a router selects only a few of those experts to participate in computation for each token. The phrase "120B total parameters, 12B activated parameters" means the model weight files store roughly 120B parameters, but processing any single token activates only about 12B—so per-token compute is close to that of a 12B dense model, while the model's expressive capacity approaches 120B.

This distinction is critical for local deployment: memory overhead is determined by total parameters, because all expert weights must reside in VRAM or unified memory; compute speed and throughput, on the other hand, are closer to the scale of activated parameters. That is why Nemotron 3.5 Super VL, despite having only 12B-level compute requirements, still demands 128GB-class memory—if the model doesn't fit, no card is fast enough to help.

What NVFP4 Actually Saves and What It Costs

NVFP4 is not whole-model 4-bit. The vision encoder, connector, Mamba convolutions, and embeddings remain BF16, and the MoE router stays FP32. The model card states that inference "uses W4A4 on Blackwell and W4A16 on non-Blackwell platforms." This means Hopper cards can load this smaller checkpoint, but matrix multiplication still runs with 16-bit activations—you save VRAM but don't get the speed dividend of FP4 tensor cores.

On the accuracy cost side, the official comparison under the same harness between BF16 and NVFP4 shows most rows within noise range, but a few are not: WebVoyager drops from 77.72 to 66.31, DeepSWE from 24.65 to 18.2, and MMLU-ProX from 76.91 to 73.76. If your target is a browser agent, an 11-point drop on WebVoyager is a real risk—and the model card doesn't mention this at all outside the table.

The Difference Between W4A4 and W4A16

In quantization terminology, W refers to the bit width of weights, and A refers to the bit width of activations. W4A4 means both weights and activations are compressed to 4-bit, so matrix multiplication can run directly on FP4 tensor cores, saving memory and gaining speed at the same time; W4A16 compresses only the weights while activations remain 16-bit, so computation requires dequantizing the 4-bit weights before multiplying them with 16-bit activations.

The hardware difference lies exactly here: Blackwell's fifth-generation tensor cores natively support the FP4 data path and can execute true W4A4; Hopper and earlier architectures lack FP4 multiplication units and can only fall back to W4A16. As a result, the same NVFP4 checkpoint loaded on different cards yields similar memory usage but very different speed performance. This also explains why "non-Blackwell platforms use W4A16" is a hardware limitation rather than a software option.

Which GPUs or Macs Can Actually Run It

The core numbers are stark: the repository size is 75.1 GB, spread across 88 safetensors shards. A single consumer-grade card cannot fit 120B at any bit width.

Hardware	Memory	Realistic Option
DGX Spark (GB10)	128 GB unified memory	Official NVFP4 + vLLM + LiLiCorr drafter, --max-num-seqs 1
RTX Pro 6000	96 GB	NVFP4 GGUF (79 GB, Unsloth estimates 96 GB needed) or UD-Q3_K_XL (65.8 GB)
128 GB+ Mac	Unified memory	MLX 4-bit (69.5 GB, text only) or llama.cpp Metal + UD-Q4_K_XL (86.9 GB)
RTX 5090	32 GB	Doesn't fit; even UD-IQ1_M needs 46.1 GB
1× H200 / 2× H100	141 GB HBM3e / dual-card	Official NVFP4 running as W4A16, or RedHatAI FP8 (128.8 GB)

For RTX 5090 users, NVIDIA's suggested alternative is Nemotron 3.5 Bolt 74B-A9B at 24GB—compressed from Super VL, natively 2-bit quantized, and aimed at llama.cpp on 32 GB+ discrete GPUs. Mac users should note that rapid-mlx's card reports an observed peak of 72.7 GB on M3 Ultra at 1K context, while explicitly stating that 96 GB Macs are "not yet validated" and that image input and MTP are not enabled. If you want vision capability on a Mac, you currently have to go through llama.cpp plus the mmproj file.

GGUF, MLX, and UD Quantization Naming

GGUF is the model file format of the llama.cpp ecosystem, supporting execution on CPU, CUDA, Metal, and other backends, and loading the vision encoder separately via an mmproj file; MLX is Apple's unified-memory framework for its own chips and its associated model format, often used for local inference on Macs. Both are community ports created to work around the official FP4 dependency.

The UD-Q4_K_XL, UD-Q3_K_XL, and UD-IQ1_M names appearing in this article belong to the Unsloth Dynamic (UD) quantization naming scheme: Q4/Q3 indicate the approximate target bit width, K indicates that the k-quant family group quantization strategy is used, and suffixes like XL or _M represent finer gradations of mixed precision within that quantization tier. These nominal bit widths are not equal to the actual average bit width, so the file size of the same "Q4" label varies with model architecture—for example, UD-Q4_K_XL reaches 86.9 GB, far exceeding the Q4 size of a typical 70B model.

Deployment Paths: vLLM on DGX Spark, llama.cpp Everywhere Else

On DGX Spark, the official Quick Start uses vllm/vllm-openai:v0.32.0, with key parameters including --max-num-seqs 1, --gpu-memory-utilization 0.75, speculative decoding with the LiLiCorr drafter configured for 7 draft tokens, and --reasoning-parser nemotron_v3, among others. There are two pitfalls here: on the model card, num_speculative_tokens 7 \ is followed by a tab, and copying it directly will cause bash to end the command early; additionally, the card's deployment table says "MTP with 5 draft tokens," which is inconsistent with LiLiCorr 7 in the command, and the card itself recommends using LiLiCorr for "all scenarios" on Spark—so trust the command.

Other environments go through llama.cpp. Unsloth's approach is to compile with -DGGML_CUDA=ON (OFF on Mac, where Metal is the default), download the UD-Q4_K_XL GGUF and mmproj, and then start llama-server. It is recommended to start from 16K context rather than 1M—the KV cache, video frames, and the 1.73 GB BF16 mmproj all come out of the same memory budget. Video processing also requires ffmpeg and ffprobe on the PATH.

The OpenMDW 1.1 License Is the Real Highlight

Nemotron 3 Super used the NVIDIA Nemotron Open Model License, while 3.5 Super VL switched to OpenMDW 1.1, granting the right to "dispose of the model materials without restriction": no field-of-use clauses, and no output restrictions. You only need to retain the notice when redistributing, and you cannot sue claiming that the model infringes patents or copyrights. This is closer to MIT than most "open" model licenses.

Two Common Restrictions in Open Model Licenses

Many model licenses advertised as "open" are not equivalent to open-source software licenses, and there are typically two types of restrictions. One is field-of-use clauses, such as prohibitions on military, surveillance, or certain competitive uses; the other is output restrictions, such as requiring downstream applications not to generate specific content, or requiring that license terms be passed on to end users. Both types of clauses directly affect compliance review for commercial deployment.

OpenMDW 1.1 removes field-of-use and output-level constraints, retaining only the notice obligation on redistribution and the mutual non-assertion clause for patents/copyrights. This is precisely why it is compared to MIT: it is closer to a permissive software license than to a usage agreement with an attached code of conduct. For teams that need to embed the model into a product and then release it externally, fewer clauses mean lower legal evaluation costs.

Conclusion: Buy Memory, Not FP4 Tensor Cores

NVFP4 is a good format, but it is tied to a sales pitch. Quality loss is small on most metrics and large on a few agentic ones; the speed advantage exists only on Blackwell, and the community produced usable GGUF and MLX builds within a day. If you already have a DGX Spark, the official path is the least hassle; if you don't, spend your money on memory, not on FP4 tensor cores.