Unsloth Major Update: DiffusionGemma & Gemma 4 MTP Support, 2x Inference Speed Boost

The open-source fine-tuning framework Unsloth has just landed a feature-packed release. In the latest v0.1.464-beta (corresponding to 2026.6.7), the team merged over 150 PRs in a single week, adding support for three new models — DiffusionGemma, Gemma 4 MTP, and MiniMax-M3 — alongside a brand-new model Hub, document Q&A (RAG), enhanced tool calling, and a wide range of hardware optimizations. For developers working on local LLM deployment and fine-tuning, this is a milestone update worth paying close attention to.

Background: What is Unsloth? Unsloth is an open-source framework designed specifically for fine-tuning large language models (LLMs), created by brothers Daniel Han and Michael Han. Its core value proposition: through hand-written Triton GPU kernels and mathematical optimizations, it speeds up LLM fine-tuning by 2–5x without any loss in accuracy, while reducing VRAM usage by 60–80%. This means tasks that previously required A100-class GPUs can now run smoothly on consumer-grade GPUs like the RTX 3090/4090. Unsloth is compatible with the Hugging Face ecosystem (Transformers, PEFT, TRL), supports popular parameter-efficient fine-tuning methods like LoRA and QLoRA, and has become one of the go-to choices for individual developers and small teams doing local LLM fine-tuning.

New Model Support: DiffusionGemma and Gemma 4 MTP

The most headline-grabbing additions in this update are the new models. DiffusionGemma can now be run and trained directly through Unsloth Studio. If you previously had trouble loading it, the team recommends upgrading to v0.1.462-beta or later. Studio provides a "live in-place denoising" visualization for DiffusionGemma, letting users observe the entire diffusion generation process in real time rather than just seeing the final output.

What is a diffusion language model? DiffusionGemma represents an emerging architecture that applies the diffusion model paradigm to language generation. Traditional autoregressive language models (like the GPT series) generate text token by token from left to right. Diffusion language models, by contrast, take inspiration from Denoising Diffusion Probabilistic Models (DDPM) used in image generation: starting from a noisy token sequence, they iteratively denoise it over multiple steps to gradually "recover" coherent text. This approach naturally supports parallel decoding and theoretically breaks the serialized generation bottleneck of autoregressive models. The "live in-place denoising" visualization in Unsloth Studio lets users directly observe how text evolves from chaos to clarity at each denoising step — making it highly valuable for understanding diffusion-based text generation. DiffusionGemma is Google's experimental exploration of combining the Gemma base model with diffusion language modeling.

Gemma 4 MTP: ~2x Inference Speed Boost

Gemma 4 MTP (Multi-Token Prediction) is another major highlight of this release. MTP allows the model to predict multiple successive tokens in a single step, dramatically increasing inference throughput. According to official figures, MTP delivers approximately 2x speedup for Gemma 4. The feature is enabled by default in Unsloth Studio — no extra configuration required.

How MTP works Multi-Token Prediction (MTP) is a training and inference acceleration technique proposed by Meta in 2024, and has been widely adopted in models like DeepSeek-V3. Traditional autoregressive models predict only the next token at each step. MTP adds multiple additional prediction heads (Draft Heads) to the model, each responsible for predicting tokens at farther positions — enabling "one forward pass, multiple position predictions." During inference, MTP can be used alongside Speculative Decoding: the main model generates multiple candidate tokens at once, which are then batch-validated and accepted, significantly boosting generation throughput without changing the output distribution. The ~2x speedup seen in Gemma 4 MTP comes from reducing GPU idle time and serialized decoding steps, fully leveraging the parallel compute capabilities of modern GPUs.

Gemma 4 also gains audio conversation support, compatible with wav, mp3, m4a, flac, and webm formats, as well as a "Preserve Thinking" feature that makes the model's reasoning process more complete and controllable.

New Hub and Local Model Download Manager

To improve the local model management experience, Unsloth has introduced an experimental Hub page. Users can browse, download, and manage models and datasets from Hugging Face. Unsloth automatically detects models and datasets already present on your machine and displays them alongside downloaded resources, effectively preventing duplicate downloads.

For already-downloaded GGUF models, the Hub provides direct "Run / New Chat" shortcuts, dramatically shortening the path from model management to actual conversation and further lowering the barrier to local inference. GGUF (GPT-Generated Unified Format) is the quantized model storage format defined by the llama.cpp project, which compresses model size significantly through 4-bit, 8-bit, and other quantization levels, enabling large models to run on consumer-grade hardware.

Document Q&A and RAG Knowledge Base

This update introduces an experimental Chat with Files feature, allowing users to ask questions directly against local documents and knowledge bases. Under the hood, it's powered by a fairly complete RAG implementation.

Understanding RAG Retrieval-Augmented Generation (RAG) is the dominant engineering paradigm for incorporating external knowledge into LLMs. The core workflow: documents are chunked and encoded as vectors (embeddings), then stored in a vector database. When a user asks a question, the system retrieves the most relevant document chunks as context, which are inserted into the prompt sent to the LLM to generate a grounded answer. Unsloth's "Hybrid Search" is an advanced RAG practice — it combines BM25-based sparse keyword retrieval (great for exact matching) with dense vector similarity retrieval (great for semantic understanding), merging results from both paths using algorithms like Reciprocal Rank Fusion (RRF). This outperforms either retrieval method alone in both recall and precision. The Citations feature addresses the trustworthiness problem in RAG systems, making model outputs traceable to specific document passages and significantly reducing hallucination risk.

Specific capabilities include:

• Hybrid search: combines keyword and semantic retrieval
• Citations: answers are traceable to source documents
• PDF preview: view the cited source directly
• Per-thread document management
• Built-in search_knowledge_base tool

This signals that Unsloth Studio is evolving beyond a training and inference tool into an all-in-one local knowledge workbench.

Enhanced Tool Calling, MCP, and Encrypted Tunnels

On the Agent capabilities front, Unsloth has put significant work into polish. Local tool calling reliability is substantially improved — tool card ordering is more sensible, repetitive tool loops are reduced, and tool use is now supported in GGUF vision models. According to official data, this optimization reduced tool calling "nudging" issues by 50% to 90% without sacrificing accuracy.

The new version also provides granular tool calling permission controls (Approve, Always Approve, Deny), supports bypassing permission modes, and makes MCP (Model Context Protocol) and Artifacts optional.

About MCP MCP (Model Context Protocol) is an open protocol proposed by Anthropic in late 2024, designed to standardize the interaction interface between AI models and external tools and data sources — essentially a "USB port" for the AI Agent ecosystem. Through MCP, tools and models from different vendors can interoperate under a unified specification, eliminating the engineering burden of fragmented integrations. Developers only need to implement an MCP server once to make it callable by any MCP-compatible client (including Claude, Cursor, and others). Unsloth making MCP optional signals that its Agent toolchain is aligning with broader ecosystem standards.

Tensor Parallelism and End-to-End Encrypted Tunnels

On the performance side, GGUF now supports Tensor Parallelism, delivering approximately +30% inference throughput improvement.

How Tensor Parallelism works Tensor parallelism is one of the core techniques for distributed inference of large models, systematically introduced by NVIDIA's Megatron-LM. It works by splitting a model's weight matrices along specific dimensions across multiple GPUs, with each GPU holding and computing only a portion of the weights, then merging results via All-Reduce communication. For quantized GGUF models, tensor parallelism means a large quantized model can be split across multiple consumer-grade GPUs for collaborative inference — breaking through single-GPU VRAM limits while delivering an additional ~30% throughput gain. This is a significant capability unlock for developers looking to run 70B+ parameter models on multi-GPU workstations.

Unsloth also adds a free Cloudflare HTTPS tunnel, enabling the local Studio to serve requests externally with end-to-end encryption — balancing convenience with security. Built on Cloudflare Tunnel technology, it requires no public IP or port forwarding to safely expose a local service as an HTTPS endpoint.

Broader Hardware and API Compatibility

Unsloth now uses new precompiled llama.cpp packages covering CUDA, ROCm, Windows, Linux, and macOS across all platforms, and adds an in-app "Update llama.cpp" button so users can update the local backend without reinstalling Studio. llama.cpp is a high-performance C++ LLM inference engine supporting CPU and multiple GPU backends, and is the de facto standard runtime for running quantized large models locally.

On the hardware side, this release improves AMD support on Windows/WSL, adds Strix Halo ROCm support, and refines Blackwell CUDA selection logic. Strix Halo is AMD's next-generation APU architecture targeting high-performance mobile and desktop platforms; ROCm is AMD's open-source GPU compute platform, the counterpart to NVIDIA's CUDA ecosystem. Blackwell is NVIDIA's latest GPU architecture (e.g., the RTX 50 series), and this optimization means Unsloth is now adapted for the latest generation of hardware.

API compatibility is also enhanced, with more consistent behavior for OpenAI-compatible and Anthropic-compatible APIs — including better error messages, token usage tracking, stop reasons, and compatibility with Claude Code.

How to Update

The team explicitly warns against using the unsloth studio update command, as packages built that way cannot receive the latest updates. The correct installation and update method is:

• macOS / Linux / WSL: curl -fsSL https://unsloth.ai/install.sh | sh
• Windows: irm https://unsloth.ai/install.ps1 | iex

Summary

From model support to Agent toolchains, from hardware compatibility to knowledge base Q&A, Unsloth has made meaningful advances across nearly every product line in this release. The addition of DiffusionGemma and Gemma 4 MTP expands the boundaries of available models — the former represents a new paradigm in diffusion-based language generation, while the latter pushes inference efficiency to new heights through multi-token parallel prediction. Meanwhile, features like the Hub, RAG, tensor parallelism, and encrypted tunnels reflect a clear trajectory: Unsloth is transitioning from a "fine-tuning framework" into a comprehensive local AI platform. For developers looking to efficiently run, train, and deploy large models locally, this update is well worth upgrading for.