Nemotron 3.5 Bolt 74B-A9B
An open-source agentic large language model from NVIDIA that uses a sparse mixture-of-experts architecture with 74B total parameters and about 9B parameters activated per token. After 2-bit quantization-aware training, it can run within 24GB of memory, and its weights, training data and recipe are all publicly released.
Core Facts
NVIDIA claims that the weights of Nemotron 3.5 Bolt 74B-A9B can run in 24GB of memory, and that 24GB refers only to the weights themselves, excluding context, KV cache and runtime overhead.
Nemotron Bolt 3.5 74B A9B is a mixture-of-experts (MoE) model with about 74B total parameters, activating only about 9B parameters per token.
Timeline (last 90 days)
Nemotron 3.5 Bolt 74B-A9B uses native 2-bit quantization; the model was designed for low-bit precision from the training stage rather than compressed after the fact.
The official tweet did not provide specific benchmark scores, tokens-per-second generation speed, or accuracy comparison data after 2-bit quantization for Nemotron 3.5 Bolt 74B-A9B.
Fitting the model into 24GB and running it smoothly on 24GB are two separate questions, and peak VRAM under long context and the stability of multi-turn tool calls still await user empirical feedback.
NVIDIA's Nemotron 3.5 Bolt 74B-A9B vision-language model is now officially available on Ollama and can be pulled and run with the command `ollama run nemotron-3.5-bolt`.
The official positioning places Nemotron 3.5 Bolt 74B-A9B's use cases in "everyday agentic tasks" and emphasizes that the model can share memory with other applications and run simultaneously.
NVIDIA claims that the weights of Nemotron 3.5 Bolt 74B-A9B can run in 24GB of memory, and that 24GB refers only to the weights themselves, excluding context, KV cache and runtime overhead.
Nemotron Bolt 3.5 74B A9B is a mixture-of-experts (MoE) model with about 74B total parameters, activating only about 9B parameters per token.
All Facts (7)
NVIDIA claims that the weights of Nemotron 3.5 Bolt 74B-A9B can run in 24GB of memory, and that 24GB refers only to the weights themselves, excluding context, KV cache and runtime overhead.
80%VerifiedNemotron Bolt 3.5 74B A9B is a mixture-of-experts (MoE) model with about 74B total parameters, activating only about 9B parameters per token.
75%UnverifiedNemotron 3.5 Bolt 74B-A9B uses native 2-bit quantization; the model was designed for low-bit precision from the training stage rather than compressed after the fact.
50%UnverifiedFitting the model into 24GB and running it smoothly on 24GB are two separate questions, and peak VRAM under long context and the stability of multi-turn tool calls still await user empirical feedback.
50%UnverifiedThe official positioning places Nemotron 3.5 Bolt 74B-A9B's use cases in "everyday agentic tasks" and emphasizes that the model can share memory with other applications and run simultaneously.
50%UnverifiedThe official tweet did not provide specific benchmark scores, tokens-per-second generation speed, or accuracy comparison data after 2-bit quantization for Nemotron 3.5 Bolt 74B-A9B.
50%UnverifiedNVIDIA's Nemotron 3.5 Bolt 74B-A9B vision-language model is now officially available on Ollama and can be pulled and run with the command `ollama run nemotron-3.5-bolt`.
50%