Unsloth v0.1.45 Released: Gemma 4 MTP Acceleration, Cross-Platform Fixes, and Studio Upgrades

Open-source LLM fine-tuning and deployment tool Unsloth has released v0.1.45-beta (internal build v0.1.451-beta), bringing full support for Gemma 4 Multi-Token Prediction (MTP), extensive cross-platform compatibility improvements, and a wide range of Unsloth Studio experience enhancements. Led by core developer Daniel Han-Chen with contributions from the broader community, this release marks a significant step in Unsloth's evolution from a pure fine-tuning acceleration library toward an all-in-one local LLM workstation.

Headline Feature: Gemma 4 MTP (Multi-Token Prediction) Support

The most technically noteworthy addition in this release is full support for Gemma 4 MTP (Multi-Token Prediction). MTP represents a significant departure from the traditional autoregressive generation paradigm — where each forward pass produces only one token, causing inference latency to scale linearly with sequence length. By adding multiple "prediction heads" to the model's output layer, MTP enables the model to predict the probability distributions of the next N tokens in a single forward pass, using a verification mechanism to filter out incorrect predictions. This achieves 2–4× inference speedup without sacrificing generation quality. Meta highlighted MTP as a core training objective in the Llama 4 paper, and DeepSeek^Ad-V3 adopted a similar mechanism at scale.

In this release, Unsloth Studio adds support for a "separate-file MTP GGUF drafter" (PR #6125), specifically designed for Gemma 4's MTP architecture. Gemma 4's MTP implementation uses a "separate drafter" architecture — the draft model is stored as an independent weight file, while the main model handles verification. This closely mirrors the core idea behind Speculative Decoding, formally introduced by Google DeepMind in 2023: a lightweight draft model rapidly generates candidate token sequences, which are then verified in parallel by the larger target model. Correct predictions are accepted outright; otherwise, generation resumes from the first mismatch. This mechanism can boost throughput by 2–3× without altering the output distribution. Users can load the MTP drafter as a standalone file and use it alongside the main model for speculative-decoding-style acceleration, with higher parameter sharing and simpler deployment compared to maintaining two fully independent models.

The development team also added graceful fallback logic: when a model lacks the corresponding prediction heads or drafter, Studio will "gracefully disable MTP" (PR #6159) rather than crashing with an error. This careful handling of edge cases reflects the tool's growing production readiness. The default chat model has also been switched to Qwen3.5-4B-MTP, further underscoring MTP as the central theme of this release.

Multimodal Capability Expansion

Gemma 4's multimodal capabilities are also strengthened in this release. Studio now supports audio input for Gemma 4 GGUF models (PR #6000), with a fix for the gate logic when audio files are uploaded via the "Add photos and files" entry point (PR #6064). GGUF tool calling now supports vision input as well (PR #6009), making multimodal local inference with tool-calling capabilities possible.

Broad Cross-Platform Compatibility Improvements

This release invests heavily in cross-platform support, particularly for Windows, WSL, and AMD GPU users.

AMD ROCm and Blackwell Hardware Support

ROCm (Radeon Open Compute) is AMD's open-source software stack for GPU computing, positioned as a counterpart to NVIDIA's CUDA ecosystem. Despite AMD's significant share in the gaming GPU market, its AI/ML software ecosystem has long lagged behind NVIDIA — with delayed PyTorch support, incomplete ROCm adaptations for third-party libraries like Flash Attention and BitsandBytes, and poor ROCm stability on Windows. This update is significant for AMD GPU users. The Windows/WSL installer fixes winget msstore certificate failures and amd-smi DiskPart prompts, and formally enables support for AMD GPU (Strix Halo gfx1151) (PR #5940) — gfx1151 is AMD's latest integrated GPU solution based on the RDNA 4 architecture, targeting high-performance laptops. The team also fixed an UnboundLocalError in ROCm version detection (PR #6149) and implemented automatic reading of BNB_ROCM_VERSION from installed wheels on Windows + ROCm (PR #5986), significantly lowering the barrier for AMD GPU users deploying local LLMs on Windows.

For the latest NVIDIA Blackwell architecture, the installer now handles compatibility intelligently. Blackwell is NVIDIA's newest GPU microarchitecture released in 2024 (corresponding to the RTX 50 series and B100/B200), introducing a new sm_120 compute capability that is not fully backward compatible — CUDA kernels compiled for older architectures may cause performance degradation or installation failures on Blackwell. The installer will now "never plan non-sm_120 CUDA builds on Blackwell hardware, and never plan CPU builds on NVIDIA hosts" (PR #6156), effectively avoiding this compatibility trap and reducing frustration for users with new hardware.

Pre-built llama.cpp Integration

Pre-built llama.cpp artifacts (covering CUDA, ROCm, and macOS) are now fetched from the unslothai/llama.cpp repository (PR #5963). Studio adds an in-app one-click llama.cpp update button (PR #6097) and proactively displays an update banner when an outdated version is detected (PR #6162). When a llama.cpp version is too old to support a model's vision projector, the system automatically falls back to text-only mode (PR #6075), ensuring basic functionality remains available.

Comprehensive Unsloth Studio Experience Upgrades

If MTP is the technical centerpiece of this release, the UX polish in Unsloth Studio is the volume highlight — dozens of PRs focused on the chat interface, data management, and API compatibility.

New Hub Download Manager and RAG Knowledge Base

This release introduces a brand-new Hub + Download Manager (PR #5916) with support for launching "Run / New Chat" directly from downloaded GGUF models (PR #6152), bringing the end-to-end model acquisition and usage experience closer to mature tools like LM Studio and Ollama.

Also introduced is a "clean-room implementation" compact RAG (Retrieval-Augmented Generation) system (PR #5910). RAG is currently the most mainstream engineering approach for injecting external knowledge into LLMs: external documents are split into chunks and stored as vector embeddings; at query time, the semantically closest document chunks are retrieved and appended to the prompt as context, allowing the model to generate answers grounded in the retrieved content. Compared to fine-tuning, RAG requires no model retraining to incorporate dynamically updated knowledge, at minimal cost. Unsloth's implementation supports knowledge bases, hybrid search (combining BM25 keyword matching with vector semantic retrieval for dual-path recall), and fast indexing — significantly improving retrieval accuracy for domain-specific terminology and proper nouns. RAG is designed to be "session-scoped with pre-selection support" (PR #6140), striking a reasonable balance between ease of use and data isolation.

API Compatibility and Security Hardening

On the API front, Studio improves adherence to OpenAI and Anthropic compatibility specs (PR #6010) and supports accepting system-role messages in Claude Code requests (PR #6006). On the security side, the team fixed an arbitrary external image URL injection vulnerability in the Markdown renderer (PR #5602), prevented internal exceptions from leaking to API clients, and hardened sandbox paths (PR #6072). These changes signal that Unsloth is taking its security boundary as a server-side component seriously.

Continued Fine-Tuning and Engineering Quality Improvements

On fine-tuning — Unsloth's core competency — this release includes several meaningful fixes. LoRA (Low-Rank Adaptation) is Unsloth's primary fine-tuning method: by attaching low-rank decomposition matrices alongside pretrained weight matrices, only the newly added parameters need to be trained, achieving near-full fine-tuning performance while reducing VRAM usage by up to 10×. Applying LoRA to MoE (Mixture of Experts) models — where correctly selecting which parameters to train (all experts vs. routing layers only) is a non-trivial engineering challenge — is addressed by the MoE LoRA target parameter handling fix (PR #5345). Additional fixes include porting KTO logps truncation protection to TRL 1.x (PR #5996 / #6086), and a use_cache configuration fix when resuming inference after gradient checkpoint preparation (PR #6137).

On engineering quality, the team added a CI guard ensuring "batched left-padding generation will never silently regress again" (PR #6145), and standardized code comments and docstrings across the entire repository using the ruff 100-character line length convention. This kind of "boring but important" foundational work is often the key to an open-source project's long-term health.

Conclusion: From Acceleration Library to Full-Stack Local LLM Platform

Unsloth v0.1.45 is a quintessential "steady progress" release: it delivers frontier inference acceleration features like Gemma 4 MTP alongside extensive cross-platform fixes targeting real user pain points. From AMD/Blackwell hardware support to comprehensive improvements in Hub, RAG, and API compatibility, the product direction is clear — Unsloth is no longer just a library that "makes fine-tuning 2× faster." It's evolving into a full-stack local LLM platform covering training, downloading, inference, and RAG.

One important note: the team explicitly warns that you should not use the unsloth studio update command to update, as it will not fetch the latest version. Instead, use the official installation script (curl -fsSL https://unsloth.ai/install.sh | sh for Linux/macOS/WSL, or the PowerShell script for Windows). For developers focused on local LLM deployment and fine-tuning, this release is well worth upgrading to.