Unsloth 0.1.900 Update: Laya Decision Models and Unified Media Library Now Available

The open-source LLM toolchain Unsloth has released v0.1.900-beta, bringing a series of important updates for local inference and training. The key changes in this release include support for Decision Models, a unified document and media library, a document viewer, and extensive performance optimizations for Apple Silicon. Image and video generation speeds up by approximately 4.5x, making it one of the most notable performance gains in this update.

Laya Decision Models: A Local Judgment Engine

The most distinctive new feature in this update is support for Laya Decision Models (the open-source version of Jev). Unlike traditional generative models, decision models focus on answering yes/no judgments, multiple-choice questions, and probability-scored evaluations — ideal for scenarios requiring structured decision outputs.

Users can enable the Decision API under Settings > API, select a specific model, and control whether it runs on CPU or GPU. The feature integrates with the TypeSafe SDK via a Jev-compatible /v1/systemone endpoint, providing developers with a type-safe calling interface. When GPU mode is selected, the model can run natively on Apple Silicon's MLX framework, further lowering the barrier to local deployment.

The introduction of decision models signals that Unsloth is moving beyond training and generation tools to cover structured reasoning tasks such as "judgment," providing a new foundational component for building automated pipelines and intelligent agents.

TypeSafe SDK is a strongly typed calling interface where the data structures of API requests and responses are constrained at the type level by the compiler or runtime, allowing parameter errors to be caught at coding time rather than at runtime. For decision models that require precise input formats and structured outputs, TypeSafe SDK significantly reduces the risk of integration errors. MLX is Apple's machine learning framework designed for Apple Silicon, fully leveraging its unified memory architecture (where CPU and GPU share the same physical memory), eliminating the overhead of copying data between different memory regions. This makes it generally more efficient than general-purpose frameworks when running small-to-medium models on Mac. The ability for Laya Decision Models to run natively on MLX means M-series chip users can achieve near-GPU-accelerated inference speeds without any additional configuration.

Unified Media Library and Document Viewer

The new Library tab consolidates chat history, images, videos, and other content in one place. Users can now view PDF, Word, Excel, and PowerPoint files directly within Unsloth, with links back to the original source files.

Document processing capabilities have been significantly enhanced: content controls, revision insertions, and text boxes in DOCX files are now correctly parsed, and adding Word files to a knowledge base no longer results in lost text content. Attachment cards have been improved to support reading more file formats. On the UI side, users can customize sidebar sections and reorder them via drag-and-drop, making the workspace more personal.

For users leveraging local models for document Q&A and knowledge management, the combination of library and viewer brings Unsloth closer to a fully integrated local AI workstation.

~4.5x Faster Image and Video Generation

Performance optimization is a major highlight of this release. LTX-2.3 video clip generation is approximately 4.5x faster, thanks to distilled sampling, compilation fixes, and hosted FP8 weights. VAE decoding optimizations for images and videos deliver 1.7x to 6.3x decoding speedups, while overall generation speed improvements vary depending on the specific workflow.

The first-frame rendering time for MiniMax-H3 models is reduced by up to approximately one minute, with peak VRAM usage reduced by 25–29 GiB on A100, B200, and RTX PRO 6000 GPUs. These VRAM optimizations are especially critical for memory-constrained users, meaning larger models or longer generation tasks can now run on the same hardware.

VAE (Variational Autoencoder) handles the final "decoding" step in image and video diffusion models: after the model completes denoising in the compressed latent space, the VAE converts latent vectors back into visible pixel images or video frames. Since VAE decoding requires heavy convolutional computation on every frame, it often becomes the speed bottleneck in generation pipelines for long videos or high-resolution content. The 1.7x to 6.3x VAE decoding speedup in this release means final output wait times are dramatically reduced even if the diffusion steps themselves take the same amount of time. FP8 weights (8-bit floating point) compress weight size to half that of common FP16 or BF16, delivering both lower VRAM usage and faster matrix computation on GPUs that support FP8 operations (such as H100 and B200). This makes FP8 one of the leading precision formats in large model quantization deployment today.

Deep Apple Silicon and MLX Optimizations

This update is particularly friendly to Apple chip users. Batched serving allows multiple responses to be decoded simultaneously, improving concurrent throughput. Grammar-constrained structured output has been added via response_format, enabling models to return results in a specified format.

For sliding window models like Gemma 4, TurboQuant KV cache and KV cache quantization have been added. The system also improves memory estimation logic and can automatically adjust context length based on available memory. These changes collectively reduce the memory pressure of running large models locally on Mac, making Apple Silicon an increasingly viable local inference platform.

KV Cache (Key-Value Cache) is a core acceleration mechanism in Transformer inference: during autoregressive generation, the Key and Value matrices computed for already-processed tokens are cached and reused when generating new tokens, avoiding redundant computation. As context length grows, KV cache can consume several gigabytes of memory — particularly noticeable on resource-limited Macs. TurboQuant KV Cache dramatically reduces cache size by quantizing the cached content (lowering the storage precision of each value) with minimal impact on output quality. For models like Gemma 4 that use sliding window attention, the KV cache only needs to retain key-value pairs for the most recent N tokens, which naturally caps cache size. Combined with quantization, memory pressure is further reduced, making long-context conversations on MacBooks with 16–32 GB unified memory genuinely feasible.

Practical Training and Inference Improvements

On the training side, this release brings faster block-FP8 LoRA training and supports LoRA fine-tuning on compressed-tensors W8A8 checkpoints. Users can load audio models such as Voxtral and Qwen2-Audio via FastModel, and can now export saved checkpoints from crashed or cancelled training jobs — protecting the results of long training runs.

On the compatibility front, the team has fixed multiple issues with Gemma, newer TRL trainers, and FP8 on RTX 5090 and RTX PRO 6000. Two new practical capabilities have also been added: a Skills Editor for creating, editing, and deleting skills directly on the desktop, and support for downloading models via ModelScope, providing an alternative for users unable to access Hugging Face.

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method: rather than directly modifying all weights of a pre-trained model, it attaches a pair of low-rank matrices alongside specific layers and only updates those matrices during training. This compresses trainable parameters to 1%–10% of the original model, dramatically reducing VRAM requirements and training time. block-FP8 LoRA goes further by quantizing the involved tensors to FP8 format, accelerating matrix multiplication and reducing activation memory usage on supported hardware. compressed-tensors W8A8 refers to a quantized checkpoint format where both Weights and Activations are stored as 8-bit integers. Applying LoRA fine-tuning directly on top of such models means users can perform domain adaptation without starting from full-precision weights, greatly reducing dependence on high-end GPUs.

Summary

Overall, the direction of the Unsloth 0.1.900 update is clear: on one hand, expanding the range of supported tasks through decision models and structured output; on the other, reducing the cost of local deployment through significant speedups and VRAM optimizations. For developers and researchers who prioritize data privacy and want to run AI models on personal hardware, these improvements bring local solutions one step closer to cloud services in terms of practical usability.