QLoRA
QLoRA (Quantized Low-Rank Adaptation) is a parameter-efficient fine-tuning method for large language models that combines model quantization with Low-Rank Adaptation (LoRA). Its core mechanism quantizes pretrained model weights into a low-precision format (e.g., 4-bit) to substantially reduce memory usage, while training only a small number of learnable low-rank adapter parameters. This approach achieves performance close to full fine-tuning while significantly reducing computational resource requirements, making large-scale language model fine-tuning feasible on limited hardware.
Core Facts
LoRA reduces fine-tuning costs by training only a small number of adapter layers (low-rank matrix bypass training).
QLoRA introduces 4-bit quantization technology, making it possible to fine-tune models with 65 billion parameters on a single RTX 3090/4090 GPU with 24 GB of VRAM.
Unsloth is an open-source tooling team focused on large model quantization and fine-tuning acceleration, providing GGUF quantized versions for mainstream open-source models.
Timeline (last 90 days)
QLoRA builds on LoRA by introducing 4-bit NormalFloat (NF4) quantization, double quantization, and a paged optimizer to compress VRAM usage while maintaining training stability.
The author's fine-tuning used the QLoRA method, with LoRA rank/alpha of 32/64, a learning rate of 1e-4, batch size 1 with gradient accumulation of 16, AdamW 8-bit optimizer, and training for 1 epoch.
Fine-tuning gpt-oss-20b with QLoRA requires approximately 14 GB of VRAM, and Unsloth's official recommendation is at least 16 GB.
QLoRA reduces base model memory usage through quantization, DoRA decouples magnitude and direction, LoRA+ uses different learning rates for the two matrices, and PiSSA changes initialization via singular components.
Unsloth's decision model solution uses QLoRA to fine-tune any text or vision LLM, training a model with a decision head; training, testing, exporting, and deployment can all be completed within Unsloth.
The core idea of QLoRA is to first quantize the pre-trained model to 4-bit precision to reduce VRAM usage, then train only a small number of low-rank adapter matrices without updating the original weights.
Decision models train any text or vision LLM via QLoRA and output a confidence score for each candidate option.
LayerSmith recently added a set of image configurations targeting large model training and fine-tuning, covering LoRA/QLoRA, advanced PyTorch training, and LLaMA-Factory.
There is precedent for using QLoRA to complete fine-tuning on a single GPU with 8 GB of VRAM, such as early community practices of fine-tuning LLaMA 7B on an RTX 3080.
Serverless Training is better suited to small- and medium-scale fine-tuning scenarios (such as LoRA and QLoRA) rather than pre-training billion-parameter foundation models from scratch.
15 more timeline events
All Facts (20)
LoRA reduces fine-tuning costs by training only a small number of adapter layers (low-rank matrix bypass training).
90%VerifiedQLoRA introduces 4-bit quantization technology, making it possible to fine-tune models with 65 billion parameters on a single RTX 3090/4090 GPU with 24 GB of VRAM.
90%VerifiedUnsloth is an open-source tooling team focused on large model quantization and fine-tuning acceleration, providing GGUF quantized versions for mainstream open-source models.
85%VerifiedQLoRA was proposed by a research team at the University of Washington in 2023, quantizing base model weights to 4-bit (NF4 format)
85%VerifiedQLoRA quantizes the base model weights in 4-bit NF4 format, making it possible to fine-tune a 13-billion-parameter model on a single GPU.
80%VerifiedLoRA and QLoRA are the currently mainstream parameter-efficient fine-tuning methods, allowing fine-tuning to be completed on consumer-grade GPUs.
80%VerifiedLoRA reduces the number of trainable parameters by injecting low-rank decomposition matrices into the model's weight matrices, making it possible to fine-tune models with billions of parameters on consumer-grade GPUs.
75%VerifiedQLoRA quantizes the base model to 4-bit precision for storage, while applying LoRA adapters on top of the quantized model for training.
75%VerifiedNF4 (4-bit NormalFloat) is an information-theoretically optimal data type proposed in the QLoRA paper (Dettmers et al., 2023).
75%VerifiedQLoRA introduces 4-bit quantization on top of LoRA, further compressing memory requirements to a fraction of the original.
65%VerifiedQLoRA was proposed in 2023 by Tim Dettmers and colleagues at the University of Washington.
65%VerifiedQLoRA combines 4-bit quantization, making it possible to fine-tune 70B parameter models on consumer-grade GPUs.
65%UnverifiedPEFT库支持LoRA和QLoRA等参数高效微调方法，可将需要更新的参数量从数十亿降低到数百万级别
90%UnverifiedQLoRA（Quantized Low-Rank Adaptation）是2023年由华盛顿大学提出的技术
90%UnverifiedUnsloth内置LoRA/QLoRA参数高效微调方法支持
85%UnverifiedGitHub上的开源项目unsloth-llama3-alpaca-lora提供了一套完整的4-bit QLoRA微调流水线，专门针对LLaMA 3 8B模型做了优化
85%Unverified大模型部署调优的核心技能包括使用Ollama、vLLM等框架部署开源模型，通过LoRA、QLoRA进行参数高效微调，以及利用RAG技术结合企业私有知识库
85%UnverifiedQLoRA技术可将7B参数模型的显存占用从约28GB降低到不足6GB
80%UnverifiedUnsloth在QLoRA基础上做了额外工程优化，使实际显存消耗比原始QLoRA实现还要低30-50%
75%UnverifiedUnsloth's core technical contributions include increasing LoRA/QLoRA fine-tuning speed by 2–5×and reducing VRAM usage by 50–70%.
70%