Unsloth New Release: MTP Doubles Inference Speed for Qwen3 and GLM
Unsloth v0.1.806-beta delivers up to 2x faster inference for Qwen3 and GLM via MTP, with 170+ improvements.
Unsloth has released v0.1.806-beta, with the headline feature being MTP (Multi-Token Prediction) that boosts inference speed up to 2x for Qwen3.8-Flash-Next and GLM-5.3-Flash. Apple Silicon users also benefit, with MLX backend optimizations delivering up to 30x faster responses in subsequent conversation turns. The update further expands audio multimodal support (adding MiniMax-Music3, Higgs, and MOSS), strengthens Agent tool-calling reliability (parallel tool calls, MCP persistence, Auto Compaction), and broadly improves AMD/ROCm compatibility — marking Unsloth's evolution into a comprehensive local LLM platform.
The open-source fine-tuning framework Unsloth has released v0.1.806-beta, introducing a standout performance upgrade: through MTP (Multi-Token Prediction) technology, the Qwen3.8-Flash-Next and GLM-5.3-Flash models can now run inference up to 2x faster. This update also includes over 170 improvements across training, chat, hardware, and performance — marking a significant step toward maturity for the Unsloth ecosystem.
MTP Multi-Token Prediction: Predicting Multiple Tokens in One Pass
Traditional autoregressive language models generate text one token at a time — each forward pass produces a single token, which becomes a throughput bottleneck at inference time. The core idea behind MTP (Multi-Token Prediction) is to have the model predict multiple subsequent tokens in a single forward pass, dramatically reducing the number of iterations needed during inference.
In this Unsloth update, MTP is enabled by default for Qwen3.8-Flash-Next and GLM-5.3-Flash. Official benchmarks show that both models can generate text up to 2x faster with MTP enabled. Notably, while MTP is on by default, users can still disable it manually — preserving flexibility for different hardware configurations and precision requirements.
For developers deploying large models locally, inference speed directly impacts practical usability. MTP means smoother conversations and higher throughput on the same hardware, with especially noticeable gains in multi-turn, long-context dialogue scenarios.
Optimization Details for Qwen3 and GLM
Beyond speed improvements, Unsloth has made targeted refinements for both models. Qwen automatically applies the recommended thinking and non-thinking mode settings, eliminating the need for manual tuning. GLM now supports stable tool calls across longer multi-turn conversations, and a bug that caused chat interruptions after tool execution in GLM-5.3 has been fixed.
Official configuration guides for both Qwen and GLM are available, along with downloadable GGUF quantized model files, lowering the barrier for everyday users.
MLX Inference Optimization: A Game-Changer for Apple Silicon Users
For Mac-based developers, the improvements to the MLX backend are a major highlight. Unsloth now supports fine-tuning large MoE (Mixture of Experts) models using MLX on Apple Silicon, for both text and image tasks.
More importantly, speed has been dramatically improved: long Qwen conversations on Mac now run significantly faster, with subsequent turns responding up to 30x faster. For Mac users, this is a compelling number — it means local LLM conversations are no longer bottlenecked by long waits.
Additional MLX improvements include: full context length support with longer batch generation; cleaner GPU memory release between generations and when switching models; and the ability to serve models via Unsloth's OpenAI-compatible API. GLM-5.3 MLX fine-tuning outputs can also be exported to GGUF format, completing the pipeline from fine-tuning to deployment.
Expanded Audio Multimodal Capabilities
Another major focus of this update is a significant expansion of audio support. Unsloth has added support for audio models including MiniMax-Music3, Higgs, and MOSS, covering use cases such as music generation and text-to-speech synthesis.
On the user experience side, audio generation now provides real-time progress updates so users can clearly track generation status, and generated audio clips support archiving and management. In terms of reliability, the team has fixed custom TTS playback issues, added Whisper pairing checks, and strengthened audio testing. These improvements push Unsloth beyond a pure text fine-tuning tool toward a comprehensive multimodal generation platform.
Chat, Tool Calling, and Agent Reliability Upgrades
On the Agent and tool-calling front, this update addresses a number of real-world pain points:
- Parallel tool calls: Multiple tools can now run simultaneously without mixing up their arguments — Unsloth correctly splits parallel tool call parameters at JSON object boundaries.
- Multimodal tool persistence: Tools remain available during image-based chat sessions.
- MCP connection persistence: Each chat session maintains its own MCP connection for faster tool calls.
- Code editing: Local models can now edit code using Codex's
apply_patchtool. - Auto Compaction: Long conversations involving media like images can continue via an automatic compaction mechanism.
- Deep Research approval: Users can review and approve Deep Research plans before a research session begins.
These improvements target the most failure-prone steps in Agent workflows, reflecting Unsloth's continued focus on engineering reliability.
Training and Hardware Compatibility Enhancements
On the training side, Unsloth has strengthened multi-GPU planning with automatic device placement, enabling larger models to be trained across multiple GPUs. The improved planner no longer withholds split reservation space from any single card and prices by physical core, resulting in better memory fitting.
For hardware compatibility, AMD/ROCm detection, installation, and GPU support have been comprehensively improved: AMD installation now automatically selects the optimal build on both Windows and Linux, with BF16 support on more GPUs; BF16 detection issues on gfx10-series ROCm GPUs and ROCm detection issues on split Debian stacks have been fixed. For AMD users who have long struggled with compatibility, these fixes are welcome news.
Users can also now customize GGUF shard sizes and save locations, making the export workflow more flexible and controllable.
API and Desktop Experience Improvements
Unsloth Desktop, the free and open-source desktop application, has also received several UX improvements:
- A new OpenAI-compatible Videos API (
/v1/videos) allows video generation through a standard interface. - Updates now download in the background and install automatically on restart, reducing interruptions.
- Custom port support enables LAN access.
- Model downloads show clearer progress indicators and automatically fall back to HTTP if Xet downloads stall.
Overall, the UI feels faster and less laggy, subsequent conversation turns respond noticeably quicker, and chat editing is safer — preserving tool cards, reply details, and conversation branches.
Conclusion
Unsloth's v0.1.806-beta update is marketed around "2x speed," but its actual value goes much further. From MTP inference acceleration and MLX optimization on Apple Silicon, to expanded audio multimodal support, improved Agent tool reliability, and comprehensive AMD hardware compatibility — over 170 changes paint a picture of a rapidly maturing local LLM toolchain ecosystem.
For developers looking to run and fine-tune large models locally or in private environments, Unsloth is simultaneously advancing speed, compatibility, and ease of use — three notoriously difficult goals to pursue at once. It's well worth keeping an eye on.
Related articles
DeepSeek V4 Pro Burning Through Credits Too Fast? The Hidden Logic Behind AI Model Pricing
Why does DeepSeek V4 Pro drain credits so fast while Flash barely moves? A deep dive into AI token billing, Pro vs. Flash pricing differences, and cost optimization tips.
RealPDE Competition Breakdown: The Frontier Challenge of AI-Powered Real-World Fluid Dynamics PDE Solving
A deep dive into the NeurIPS 2026 RealPDE Competition, covering the Sim2Real and LTTTA tracks, and how neural operators tackle real-world PIV and CFD fluid PDE challenges.
Building a Production-Grade 3DGS Training Library from Scratch: A Deep Dive into Full-GPU Residency and the Vulkan Stack
A veteran graphics engineer builds a production-grade 3DGS training library from scratch using C++23, CUDA, and Vulkan, achieving 60fps with 5M splats. Deep dive into its architecture and design.