Unsloth v0.1.916-beta Released: Major Performance Fixes and Dual-GPU Acceleration

Unsloth, an open-source toolchain for local LLM training and inference, has long positioned itself around one core selling point: "run large models on limited VRAM." The v0.1.916-beta release still carries a beta tag, but the changelog reads more like a significant milestone release. From chat interface lag and image/video generation loading to the default performance of model.generate() in the Python training API, nearly every entry targets "noticeable speed."

Chat Experience: No More Input Lag in Long Conversations

Typing lag in long conversations is a common problem with local chat clients, especially once you add the Glass theme or a chat background image — GPU rendering pressure translates directly into input delay. This update explicitly fixes "input lag in long chats when the Glass theme or chat background is enabled" and keeps chat search usable for extremely long histories instead of failing to load.

Cleaning up and deleting chats is also noticeably faster with large histories. Voice and audio features no longer freeze a streaming reply for several seconds on first call, and reading aloud with the Unsloth model engine no longer incurs an extra model lookup. For users running the Glass theme with background images on Linux desktop, the team says responsiveness improves by up to 2x and no longer depends on GPU rendering.

The Glass theme itself got visual fixes too: background images are now dimmed instead of blurred, and controls on other pages match the glassy look of the chat interface. The next release will continue refining the Glass theme, so this visual thread isn't finished yet.

Image and Video: Dual-Card Splitting and Faster Loading

This is the most technically substantial part of the update. On two matching, idle NVIDIA GPUs, models that fit on a single card — SDXL, Qwen-Image, Wan2.2 5B — split half of each step's computation to the second GPU, with the team citing roughly 1.5–1.8x speedups. FLUX and Lumina 2 take a different approach: splitting equal-sized image batches across the two cards in parallel.

Loading-side improvements are just as direct: SDXL loads up to 40% faster, cutting the wait for the first image. On GPUs with 16–24 GB VRAM, LTX-2.5 now encodes new prompts on the GPU rather than the CPU, significantly speeding up video clip rendering for new prompts; LTX-2.5 GGUF Q8_0 video is about 10% faster on NVIDIA GPUs with identical frames.

AMD users benefit too: image models like FLUX and Z-Image render faster on Linux with RDNA3 and RDNA4 GPUs. Commit logs show finer optimizations as well, such as compiled int8 W8A8 linear layers on RDNA3/RDNA4 ROCm making Qwen-Image-2.1 int8 21% faster per step, and the Triton AMD backend for installing flash-attn on ROCm delivering 10–15% faster diffusion steps.

The phrase "split half of each step's computation to the second GPU" refers to sequence parallelism / model parallelism in diffusion model inference. Diffusion models (like SDXL and FLUX) generate images through dozens of denoising iterations, each running a full UNet/DiT forward pass. When the whole model fits in a single card's VRAM, the most common multi-GPU acceleration isn't model sharding (which introduces heavy inter-GPU communication) but splitting along the batch dimension or inside the compute graph: either distributing multiple candidate images for the same prompt across two cards in parallel, or splitting a single sample's attention/linear layer computation along the token or channel dimension, with each card computing half and syncing via NCCL all-gather/all-reduce. The former has near-zero communication overhead but only helps with batch generation; the latter also accelerates single images, at the cost of synchronizing activation tensors (not gradients) at every step. This also explains why the team emphasizes "matching and idle" — mismatched cards mean the faster one waits for the slower one, and if other tasks occupy VRAM, the split itself may OOM.

The recurring terms GGUF, Q8_0, and int8 W8A8 refer to three different quantization formats that are easy to confuse. GGUF is the model container format of the llama.cpp ecosystem, packaging weights and metadata into a single file, with quantization granularity options like Q4_K_M and Q8_0, mainly aimed at CPU/hybrid inference and low-VRAM scenarios; Q8_0 means every 32 weights share an 8-bit scaling factor — a "near-lossless" 8-bit quantization. W8A8 means both weights and activations are quantized to 8 bits, requiring hardware or kernel support for int8 matrix multiplication, which is why the notes specifically limit it to compiled linear layers on RDNA3/RDNA4 ROCm — only when the underlying int8 GEMM kernel exists does W8A8 actually beat FP16, otherwise dequantization overhead eats the gains. Q8_0 video being "about 10% faster with identical frames" indicates the optimization happens at the compute scheduling level (better kernel selection or memory layout) without changing numerical precision.

New Features: Floating Browser and Per-Chat Browser Sessions

The most visible feature change is the floating browser: the built-in browser can become a small window hovering over the chat, movable and resizable, with each chat remembering whether it's in floating mode. Better still, multiple chats can now each run independent browser instances, so the model won't fight over the same page while working in the background.

Thinking blocks and code blocks got a redesign too, now with capped heights and scroll containers so long outputs no longer stretch the page indefinitely. On the model side, Dynamic V3.1 Qwen3.8-Flash is introduced, which the team says cuts token usage by 50% while improving accuracy by 15%.

Other additions include: support for loading Anima, Chroma, MOSS-Image, and Krea 2 image models from single-file checkpoints (all except Chroma support LoRA and LoKR files); account owners can rename managed accounts without losing chats, files, or settings; download speed limits can be set in settings; and OpenAI-compatible connections support custom request headers.

Fixes and Training Library Changes

The fixes address plenty of everyday pain points: chat-only installs with non-GGUF models downloaded no longer fail on first chat due to a missing default model; GGUF models on pure CPU machines no longer blow up memory under Auto context; HTTPS downloads on Windows can resume after pause or interruption instead of restarting; Windows updates no longer silently hang for half a minute at the end; connecting to Blender via MCP works again; and the first message in Compare mode no longer leaves an empty panel.

On the training and library side, model.generate() is now faster by default for Llama, Qwen2, and Qwen3 on Linux + NVIDIA GPUs, with speedups of several times on small models. Training startup is up to a minute faster when Hub datasets are split into hundreds of files. With "assistant completions only" enabled, responses starting with a newline are no longer excluded from training. Commit logs also show lower-level changes: CUDA graphs decoding enabled by default with memory release and eager fallback, frozen LoRA A/B matrices skipping gradient GEMMs, and optional BF16 AdamW random parameter updates.

CUDA graphs is a capture-and-replay mechanism from NVIDIA: a fixed GPU compute sequence (kernel launches, memory copies, etc.) is recorded into a graph, then replayed with a single graph launch, eliminating per-kernel CPU launch overhead. It's especially effective for model.generate()-style autoregressive decoding, where the compute graph structure is nearly identical for every generated token — a classic "many small kernels, high launch overhead" scenario. This also explains why small models see speedups of several times while large models see diluted gains, since each step's computation is already heavy. The trade-off is that CUDA graphs require fixed memory addresses and break with dynamic shapes or dynamic allocation, which is why the commits pair it with "memory release and eager fallback": unsupported operators or extremely long sequences fall back to normal per-kernel execution instead of erroring out. Enabling it by default for decoding but typically not for training is a common trade-off for this kind of optimization.

Trade-offs Worth Noting

Most of the speedups in this update come with prerequisites. Dual-card splitting requires two matching, idle GPUs, and 1.5–1.8x is a typical figure rather than a guarantee; LTX-2.5's speedup is limited to the 16–24 GB VRAM range; some optimizations (like fused AdaLN for FLUX and the int8 video decoder for MiniMax-H3) are only enabled by default on specific architectures or platforms. For typical users, the safest gains come from the chat lag fixes, faster SDXL loading, and the model.generate() improvements — all three require no extra hardware to benefit from.

Unsloth Desktop remains free and open source, with downloads for Windows, macOS, Linux (deb, AppImage, ARM64), and Windows ARM64.