Unsloth's New Release: MTP Doubles Inference Speed for Qwen and GLM
Unsloth's MTP update doubles inference speed for Qwen and GLM with 170+ improvements.
Unsloth v0.1.805-beta brings MTP (Multi-Token Prediction) enabled by default, boosting Qwen3.8-Flash-Next and GLM-5.3-Flash inference speed by up to 2x. The release also features major MLX upgrades on Apple Silicon with up to 30x faster multi-turn conversations, expanded audio model support, improved Agent tool calling reliability, enhanced AMD/ROCm compatibility, and over 170 additional improvements across training, chat, and desktop experience.
The open-source fine-tuning framework Unsloth recently released version v0.1.805-beta, bringing a notable performance upgrade: through MTP (Multi-Token Prediction) technology, inference speed for Qwen3.8-Flash-Next and GLM-5.3-Flash models has been boosted by up to 2x. Additionally, this update includes over 170 improvements related to training, chat, hardware, and performance — making it a remarkably comprehensive iteration.
MTP Multi-Token Prediction: The Core Engine Behind Faster Inference
The biggest highlight of this release is MTP being enabled by default. MTP (Multi-Token Prediction) is a technique that allows models to predict multiple subsequent tokens in a single forward pass, significantly reducing the number of computation rounds during inference and delivering tangible speed improvements.
According to Unsloth's official announcement, Qwen3.8-Flash-Next and GLM-5.3-Flash can achieve up to 2x faster generation speed with MTP enabled. A noteworthy detail: MTP is now the default configuration, so users can enjoy the acceleration benefits without any additional setup. Of course, if there are precision or stability concerns for specific use cases, it can also be manually disabled.
Beyond this, both models received more granular optimizations: GLM can now reliably invoke tools across longer multi-turn conversations, while Qwen automatically applies officially recommended thinking and non-thinking mode configurations, lowering the barrier to parameter tuning. The team has also published usage guides for Qwen and GLM along with available GGUF quantized files.
Why MTP Matters
For users deploying large models locally, inference speed is often the core bottleneck of the experience. Traditional autoregressive decoding generates tokens one at a time, limiting efficiency. MTP predicts multiple tokens at once, essentially compressing the number of generation steps. Doubling the speed while maintaining output quality means the same hardware can handle higher throughput — or that lower-spec devices can deliver a smooth interactive experience. This is especially significant for users with consumer-grade GPUs and Mac users.
Comprehensive MLX Upgrades on Apple Silicon
This update is particularly friendly to Mac users. Built on the MLX framework, Unsloth now supports fine-tuning two large MoE (Mixture of Experts) models for text or image tasks on Apple Silicon.
Even more impressive is the leap in speed: Qwen runs noticeably faster on Mac in long conversation scenarios, with response speed in subsequent turns improving by up to 30x. Additionally, MLX models can now utilize the full context length and support longer batch generation. MLX also releases GPU memory more cleanly between multiple generations and model switches, avoiding memory accumulation issues.
Unsloth has also unlocked serving capabilities — users can serve MLX models via Unsloth's OpenAI-compatible API, and GLM-5.3 MLX fine-tuning results can be exported to GGUF format, further enhancing workflow continuity.
Major Expansion of Audio Model Support
Beyond text models, Unsloth has taken an important step in the audio domain, adding support for multiple audio models including MiniMax-Music3, Higgs, and MOSS.
In terms of user experience, the audio generation process now displays real-time progress updates, giving users clear visibility into the status of long-running tasks. Generated audio clips can also be archived and managed. On the stability front, the team fixed several issues related to custom TTS playback, Whisper pairing checks, and more, while strengthening the audio testing framework. These improvements signal Unsloth's evolution from a pure text fine-tuning tool toward a multimodal local AI platform.
Improved Reliability for Chat and Agent Tool Calling
As Agent and tool calling become increasingly important, this update includes extensive refinements to the chat and tool pipeline:
- Parallel tool calling: Multiple tool calls can be initiated simultaneously without confusing their respective parameters — critical for complex Agent workflows.
- Tools preserved in image scenarios: Tools remain available during chats that include images.
- Persistent MCP connections: Each chat maintains an independent MCP connection, speeding up tool invocations.
- Code editing capability: Local models can now use Codex's
apply_patchtool to directly edit code. - Auto Compaction: Supports continuing conversations in long chats containing images and other media.
- Deep Research approval: Users can review and approve research plans before the research begins.
Together, these details form a more mature and trustworthy local Agent environment.
Training and Hardware Compatibility Improvements
On the training and hardware front, Unsloth has delivered solid improvements as well. Multi-GPU training supports automatic device allocation, enabling larger models to be split across multiple GPUs. The team focused on fixing VRAM planning, split model training, and index_select-related crash issues across multiple PRs.
For AMD/ROCm users, this update strengthens GPU detection, installation workflows, and compatibility. AMD installations now automatically select the optimal build version on Windows and Linux, enable BF16 on more GPUs, and fix BF16 detection for gfx10 ROCm GPUs as well as ROCm detection on Debian split-stack setups.
On the export side, users can now customize GGUF shard sizes and save locations, greatly improving flexibility.
API and Desktop Experience Enhancements
As a free and open-source desktop application, Unsloth Desktop supports Windows, macOS, and Linux. Desktop update highlights include:
- A new OpenAI-compatible Videos API for generating videos through a unified interface.
- Updates download in the background and install automatically on restart, minimizing interruptions.
- Support for custom ports for LAN access.
- Clearer model download progress display, with automatic fallback to HTTP when Xet transfers stall.
- The long-missing Ctrl+F search functionality is finally available.
Conclusion
From the 2x inference speedup delivered by MTP, to MLX achieving up to 30x faster conversations on Apple Silicon, to comprehensive enhancements in audio multimodality, Agent tool chains, and AMD hardware compatibility, Unsloth v0.1.805-beta is an extraordinarily wide-ranging update. It delivers hardcore performance numbers while also investing heavily in stability and usability refinements. For developers looking to efficiently run and fine-tune open-source LLMs locally, this version is well worth upgrading to.
Related articles
The Dude System: How Dual-Detection Multi-Agent AI Catches Inconsistencies Between Papers and Code
Dude is the first dual-detection multi-agent system for paper-code discrepancy detection, using granularity-aligned negotiation and two-stage salience filtering to boost recall and precision by up to 22.8%.
Implicit Instruction Following in Full-Duplex Voice Assistants: DSB-IFEval Benchmark Analysis
In-depth analysis of the DSB-IFEval benchmark reveals full-duplex voice assistants' capability gaps in implicit instruction following, persona reasoning, and conflict resolution. Covers comparative testing of six voice systems and examines behavioral vs. content trade-offs driven by architectural differences.
Personalizing AI Teaching Assistants with Prompt Engineering: A Deep Dive into the Six-Dimension Learner Profile Framework
Explore a prompt engineering framework for AI teaching assistants using six-dimension learner profiles and Bloom's Taxonomy to deliver 96 personalized teaching styles without model retraining.