RTX 3090
NVIDIA's consumer-grade flagship graphics card featuring 24 GB of GDDR6X memory, suitable for AI training and inference workloads.
Core Facts
QLoRA introduces 4-bit quantization technology, making it possible to fine-tune models with 65 billion parameters on a single RTX 3090/4090 GPU with 24 GB of VRAM.
NVLink 4.0的双向带宽高达900GB/s，而传统PCIe总线带宽约为64GB/s
The RTX 4090 has 24 GB of VRAM, and after loading the 27B Q4 model, there is typically enough space remaining to support a context of 30,000–40,000 tokens.
Timeline (last 90 days)
The MiniDog project can complete training of a text-to-image diffusion Transformer on four RTX 3090s: 55 minutes for pre-training and 15 minutes for fine-tuning.
Running Drama 3 locally requires about 39GB of VRAM (a dual RTX 3090 configuration).
A used RTX 3090 with 24GB of VRAM costs about 800 USD, can cover 7B to 33B Q4 models, and costs about 15 USD per month in electricity.
RTX 3090 是最后一张带物理 NVLink 接口的消费卡；两张 3090 加桥接后可获得 112.5 GB/s 双向带宽。
When running Whisper ASR, a TTS voice model, and nomic-embed-text simultaneously on an RTX 3090 with 24 GB of VRAM, the three resident models occupied 17.8 GB (72%) of VRAM, peaked at 23.4 GB over the course of a week, and never spilled over into system memory.
When using LoRA, ordinary consumer-grade graphics cards (such as the RTX 3090/4090) are sufficient to fine-tune models with billions of parameters.
The 124M scale is the training capacity upper limit that a single consumer-grade GPU (such as an RTX 3090/4090) can handle.
2B-parameter-scale models can run on a single consumer-grade GPU (e.g., RTX 3090/4090 with 24 GB VRAM) or even in some CPU inference scenarios.
Long-document first-token latency is a hard limitation caused by insufficient 24GB RAM capacity, and increasing RAM often alleviates long-document first-token latency more effectively than upgrading the GPU.
Running the 125B-parameter Flash Next model on an RTX 3090 (24GB VRAM) with 24GB RAM, generation speed is stable at 9 to 10 tokens per second, with a peak of 18 for short texts.
28 more timeline events
All Facts (20)
QLoRA introduces 4-bit quantization technology, making it possible to fine-tune models with 65 billion parameters on a single RTX 3090/4090 GPU with 24 GB of VRAM.
90%VerifiedNVLink 4.0的双向带宽高达900GB/s，而传统PCIe总线带宽约为64GB/s
65%VerifiedThe RTX 4090 has 24 GB of VRAM, and after loading the 27B Q4 model, there is typically enough space remaining to support a context of 30,000–40,000 tokens.
65%Verified32B参数规模的模型在量化后通常可以压缩到约16-20GB显存占用，一张NVIDIA RTX 4090（24GB显存）就能运行
65%UnverifiedUnsloth通常需要配合NVIDIA GPU使用，推荐RTX 3090/4090或更高
80%UnverifiedMiniMind-V可以在普通消费级GPU（如RTX 3090）上完成全流程训练
80%UnverifiedGPT-OSS 20B在RTX 3090上可达约40 Token/s的生成速度，在RTX 5090上能达到200 Token/s
75%UnverifiedOfficial estimates suggest a 24GB GPU can run approximately 100–140 tokens/second.
60%UnverifiedThe MiniDog project can complete training of a text-to-image diffusion Transformer on four RTX 3090s: 55 minutes for pre-training and 15 minutes for fine-tuning.
50%UnverifiedRunning Drama 3 locally requires about 39GB of VRAM (a dual RTX 3090 configuration).
50%UnverifiedA used RTX 3090 with 24GB of VRAM costs about 800 USD, can cover 7B to 33B Q4 models, and costs about 15 USD per month in electricity.
50%UnverifiedRTX 3090 是最后一张带物理 NVLink 接口的消费卡；两张 3090 加桥接后可获得 112.5 GB/s 双向带宽。
50%UnverifiedWhen running Whisper ASR, a TTS voice model, and nomic-embed-text simultaneously on an RTX 3090 with 24 GB of VRAM, the three resident models occupied 17.8 GB (72%) of VRAM, peaked at 23.4 GB over the course of a week, and never spilled over into system memory.
50%UnverifiedWhen using LoRA, ordinary consumer-grade graphics cards (such as the RTX 3090/4090) are sufficient to fine-tune models with billions of parameters.
50%UnverifiedThe 124M scale is the training capacity upper limit that a single consumer-grade GPU (such as an RTX 3090/4090) can handle.
50%Unverified2B-parameter-scale models can run on a single consumer-grade GPU (e.g., RTX 3090/4090 with 24 GB VRAM) or even in some CPU inference scenarios.
50%UnverifiedLong-document first-token latency is a hard limitation caused by insufficient 24GB RAM capacity, and increasing RAM often alleviates long-document first-token latency more effectively than upgrading the GPU.
50%UnverifiedRunning the 125B-parameter Flash Next model on an RTX 3090 (24GB VRAM) with 24GB RAM, generation speed is stable at 9 to 10 tokens per second, with a peak of 18 for short texts.
50%UnverifiedIn long-context testing, normal mode ultimately completed up to 114,000 tokens, MTP2 completed up to 81,000 tokens, and failed to complete at 98,000 tokens.
50%UnverifiedThe UP host recommends keeping daily usage around 64,000 tokens, noting that the official spec of 260,000 tokens does not mean a single 24GB GPU can comfortably handle the full context.
50%