Unsloth Local Support for Nemotron 3.5 and Meta Muse Spark Open: A Full Breakdown

The open-source fine-tuning tool Unsloth has released v0.1.908-beta, pushing three things into the spotlight at once: NVIDIA Nemotron 3.5 series now runnable locally, Meta's latest 816B multimodal MoE model Muse Spark Open available for download, and official training and chat support for text-diffusion LLMs. For developers long constrained by VRAM and compute, what this update really changes is the question of "what can I run on my own machine."

Nemotron 3.5: From 1M Context to Consumer-Grade VRAM

This release includes two Nemotron 3.5 variants with clearly differentiated positioning.

Super VL is the full version: 120B parameters (12B activated), supporting up to 1 million tokens of context, aimed at document understanding, computer use, and agent scenarios. Unsloth's VRAM references are roughly 64GB at 1-bit quantization and roughly 128GB at 4-bit.

Bolt is a lightweight version compressed from Super VL: 74B parameters (9B activated), 128K context, and runs on roughly 32GB of memory at 1-bit or 2-bit. This means a machine with 24GB to 32GB of VRAM can directly host a multimodal-capable large model.

On performance, the official claim is that Super VL generates roughly 5x faster than Inkling Small at 100K input length, with comparable intelligence. Part of the speedup comes from acceleration mechanisms Unsloth enables by default: Bolt uses a DSpark drafter (an extra 1–2GB), while Super VL uses built-in MTP (Multi-Token Prediction); both can be disabled in settings.

On the comparison front, the trade-offs of the compressed model are worth noting — Bolt retains 128K context but takes a clear cut from Super VL's 1M, making it suitable for dialogue and medium-length code tasks, while long-document scenarios still call for Super VL.

MoE (Mixture of Experts) is the core concept for understanding the relationship between parameter counts and VRAM across these models. Traditional dense models activate all parameters at every inference, whereas MoE splits the feed-forward layers into multiple "experts," routing each token to only a few of them, so "total parameters" and "activated parameters" are two different numbers. Nemotron 3.5 Bolt is listed at 74B total parameters and 9B activated, meaning VRAM must hold all the weights, but the actual compute per token is equivalent to a 9B dense model — this is precisely why it can still maintain relatively high throughput on 32GB of memory.

This also explains the repeated quantization terminology in the article: 1-bit, 2-bit, and 4-bit refer to weights compressed to 1/2/4 bits per parameter; lower numbers mean less VRAM usage but greater precision loss. A 120B model at roughly 64GB under 1-bit is precisely the rough estimate from 120B × 1 bit ÷ 8, and in practice it will fluctuate slightly due to embedding layers, KV cache, and runtime overhead.

Meta Muse Spark Open: 816B Parameters Running on 256GB

Meta's Muse Spark Open release this time is an 816B-parameter MoE model with roughly 42.7B activated parameters, supporting 1 million context and requiring about 256GB of memory at 2-bit quantization.

Its input modalities cover text, images, video, and audio, and reasoning effort can be adjusted from low to xhigh — a design that lets the same model handle fast-response everyday tasks while also switching to complex scenarios with a high reasoning budget.

On training data, the official description says it covers whole-repository-level code generation, end-to-end project building, and automated research tasks. In positioning, this is a model aimed at multimodal reasoning, programming, and agent workflows, rather than a pure chat model.

The 256GB deployment threshold is clearly not something consumer-grade hardware can bear, but under the MoE architecture, the 42.7B activation volume means actual inference compute overhead is far lower than the total parameter count, making it feasible on multi-GPU servers or high-memory workstations.

Reasoning effort is an inference-time parameter that controls the model's "thinking budget." Unlike training-time scale, it typically determines at inference time how many steps of chain-of-thought the model unfolds internally and how many tokens are allocated to intermediate reasoning, thereby sliding between latency and accuracy. Muse Spark Open allowing adjustment from low to xhigh essentially lets the same model handle lightweight Q&A while also switching to high-budget mode for complex reasoning, without deploying two models of different sizes.

Additionally, the 1 million token context mentioned in the article has its biggest engineering cost not in weight VRAM but in KV cache. The longer the context, the more historical key-value pairs must be cached, and VRAM usage grows linearly with sequence length — which is also why long-context models often need specialized paged or quantized KV cache techniques for support.

Text Diffusion Models Enter the Trainable Stage

On the training side, this update introduces a direction previously less covered by mainstream toolchains: text diffusion language models.

Unsloth now supports training and chatting with diffusion language models such as LLaDA, Dream, SDAR, LLaDA2, Nemotron-Labs-Diffusion, and DiffusionGemma inside the app. Chat responses display a live denoising canvas, visually showing how the diffusion process gradually generates text from noise.

On training capabilities, DeepSpeed ZeRO-2 and ZeRO-3 support has been added, including 4-bit QLoRA, full fine-tuning, DPO, and CPU offload. The DDP + gradient checkpointing LoRA training issue that previously caused crashes has also been fixed. GRPO combined with fast_inference can now run on Nemotron 3 Nano and block-FP8 MoE models, and shares weights with vLLM.

For researchers, diffusion language models represent a technical route distinct from the autoregressive paradigm, and being able to directly train such models in a local tool lowers the barrier to experimentation.

Text diffusion language models are fundamentally different from mainstream autoregressive models in their generation mechanism. Autoregressive models predict token by token from left to right, with each step depending on the already-generated prefix; diffusion models start from pure noise or a random token sequence and, through multiple rounds of denoising iterations, gradually converge to the target text. Their advantage lies in parallel iteration and naturally supporting global corrective generation, but they also bring new problems such as step scheduling and denoising consistency. LLaDA, Dream, SDAR, and others mentioned in the article are open-source representatives of this direction.

The companion DeepSpeed ZeRO-2/ZeRO-3 is Microsoft DeepSpeed's VRAM optimization technique: ZeRO-2 shards gradients and optimizer states across multiple GPUs, while ZeRO-3 further shards the model parameters themselves, allowing models too large for a single GPU's VRAM to be trained across cards. The CPU offload mentioned in the article temporarily offloads some states to memory, trading speed for VRAM headroom; together, these three are precisely the key to training diffusion models locally.

Prompt Caching and Local Inference Speedups for Coding Agents

For developers who regularly use Claude Code, Codex, and OpenCode, this update includes several changes that directly affect efficiency.

When these coding agents run on local GGUF models, the conversation content that needs reprocessing in subsequent turns is greatly reduced, and the first response in a newly opened Claude Code session starts up to 4x faster. On the Python side, single-prompt model.generate is up to 2.7x faster for Llama and Qwen3 models at 32K context; for bitsandbytes 4-bit models generating 2 to 4 prompts at once, it is up to 2.8x faster on certain NVIDIA GPUs.

A practical engineering detail is the --auto-route parameter: when running Claude Code, OpenCode, or Codex on cloud models, adding it to unsloth start routes routine coding work such as reading files, searching, and executing commands to the local model, with only tasks that truly need a strong model going to the cloud. This is a compromise between cost and capability.

The model aliases feature is also worth mentioning — name it once in Settings > API > Model aliases, and when the underlying model is swapped, upper-layer applications require no changes, which is a relief for multi-model switching workflows.

Prompt caching targets the recurring common-prefix problem in coding agent scenarios. Tools like Claude Code and Codex stuff the system prompt, project files, and conversation history back into the context every turn, and much of that content is identical to the previous turn. Caching lets the key-value state of this prefix be reused without recomputation, so the article's statement that "the conversation content needing reprocessing in subsequent turns is greatly reduced" translates directly into up to 4x faster startup for the first response.

The --auto-route mentioned in the article is a hybrid inference routing strategy: let the local model handle high-frequency but low-difficulty operations such as reading files, searching, and executing commands, and forward requests that truly need strong reasoning to the cloud. Combined with model aliases, it allows upper-layer agents to switch the underlying model without changing code — a practical combination for balancing cost and capability.

Browser Use and Multimodal Generation

The Browser use feature now fills in observability and control granularity: the cursor is displayed when the model clicks and types, and this can be disabled in settings; prompts support Always allow, waiving confirmation for a certain type of operation throughout an entire conversation, while script-type operations still require asking every time. Each conversation has its own independent browser tab, and on desktop, browser operations continue executing even when the conversation is not in the foreground.

On image generation, the Images page can generate images through a connected image server (sd-server, Ollama, Lemonade, vLLM, llama.cpp, or a custom OpenAI-compatible address). Image and video models now skip never-used text encoder layers, saving memory on every load.

Hardware Compatibility: Fixes for Older GPUs and Hybrid Configurations

This update includes quite a few patches for hardware adaptation, especially friendly to users on non-mainstream configurations.

Windows PCs with NVIDIA drivers older than CUDA 12 no longer fail outright; instead, they install the Vulkan or CPU version of llama.cpp. AMD GPUs unsupported by ROCm (such as RDNA 1 and Polaris) can get the Vulkan version of llama.cpp even if ROCm is installed on the system or another AMD GPU supports ROCm. When PyTorch is CPU-only, GGUF image generation can run on AMD and Intel Arc GPUs via Vulkan.

Machines with multi-GPU mixed generations now select the attention backend separately for each GPU, and the crash issue with Hopper GPUs combined with conda-forge xformers has also been resolved. These fixes are hardly glamorous, but they determine whether the tool can actually run on older machines.

Deployment Recommendations

If the goal is to run large models locally, Nemotron 3.5 Bolt is currently the lowest-barrier entry point, starting at 32GB of memory; Super VL suits scenarios with 64GB or more of VRAM that need long context; Muse Spark Open leans more toward team-level multimodal agent deployment. Training support for text diffusion models is a long-term value proposition — it runs now, but the ecosystem and toolchain are still early.

One-line Summary

Unsloth's new release makes Nemotron 3.5, Meta's 816B MoE, and text diffusion models locally trainable and runnable.

Paragraph Summary

The open-source fine-tuning tool Unsloth has released v0.1.908-beta, advancing three things around "what can be run locally and what can be trained." On the model side, it introduces NVIDIA Nemotron 3.5's two variants (Super VL 120B/1M context, Bolt 74B/runnable on 32GB) and Meta's 816B MoE model Muse Spark Open, significantly lowering the local barrier for multimodal large models. On the training side, it supports text diffusion language models such as LLaDA, Dream, and SDAR for the first time, and adds DeepSpeed ZeRO-2/3, DPO, and CPU offload capabilities. On the engineering side, it optimizes prompt caching and local inference speed for coding agents, with the first response in a new Claude Code session up to 4x faster, and adds Vulkan fallback compatibility for older GPUs and mixed-generation multi-GPU setups.