RTX Pro 6000
A high-end GPU from NVIDIA targeting professional workstations, equipped with 96 GB of VRAM. It is suited for tasks with large memory requirements such as local inference of large language models.
Core Facts
Timeline (last 90 days)
On a single RTX Pro 6000, using MTP speculative decoding (K=3) reduces single-stream per-token latency from 22 ms to 13 ms, increases the number of concurrent requests per card from 32 to 64, and raises aggregate output from 999.9 tokens/sec to 1954.0 tokens/sec.
The Qwen3.8-27B checkpoint occupies 28.75 GiB on disk, and after loading in vLLM the weights plus non-torch overhead are about 29.34 GiB, leaving 53.8 GiB on a 96 GB RTX Pro 6000 for the KV cache, equivalent to about 767,317 tokens.
Nick Erickson (author of AutoGluon) used TabPFN 3.5 to achieve first place in the historical Kaggle Auto Challenge with a single line of code, one RTX Pro 6000 GPU, and one minute of computation.
NVIDIA officially doubled the price of the RTX Pro 6000 to $16,000, with prices on Amazon commonly reaching $19,999.
The M5 Ultra's memory capacity is more than five times that of the RTX Pro 6000, but its memory bandwidth is only about two-thirds of the latter.
The fact that 96GB of VRAM can fit two models simultaneously does not mean two tasks can run at full speed simultaneously, because GPU compute resources are shared and competed over.
ECC error-correcting memory can automatically detect and correct single-bit memory errors; gaming cards such as the RTX 5090 typically do not include ECC, while the RTX Pro 6000, NVIDIA A-series, and H-series professional cards all come with ECC as standard.
If eighty percent of one's work involves local LLMs, very large models, Coding Agents, and one cares about quiet operation, power consumption, and maintenance costs, the M5 Ultra—especially the 256GB or above version—is recommended; if the work involves image generation, video generation, LoRA, 3D, or training, the RTX Pro 6000 is recommended.
On an RTX Pro 6000 (96 GB VRAM), the official FP8 version combined with built-in MTP speculative decoding can reach approximately 120 tokens/second, with quality nearly indistinguishable from 16-bit.
The market price of a new DGX Spark is roughly between $5,000 and $6,000, and the price continues to rise with increasing demand.
1 more timeline events
All Facts (10)
The NVIDIA RTX 5090 is based on the Blackwell architecture, is equipped with 32 GB of GDDR7 memory, and has a theoretical FP16 computing power exceeding 100 TFLOPS.
70%VerifiedNVIDIA RTX 4090拥有24GB GDDR6X显存
65%UnverifiedOn a single RTX Pro 6000, using MTP speculative decoding (K=3) reduces single-stream per-token latency from 22 ms to 13 ms, increases the number of concurrent requests per card from 32 to 64, and raises aggregate output from 999.9 tokens/sec to 1954.0 tokens/sec.
50%UnverifiedThe Qwen3.8-27B checkpoint occupies 28.75 GiB on disk, and after loading in vLLM the weights plus non-torch overhead are about 29.34 GiB, leaving 53.8 GiB on a 96 GB RTX Pro 6000 for the KV cache, equivalent to about 767,317 tokens.
50%UnverifiedThe M5 Ultra's memory capacity is more than five times that of the RTX Pro 6000, but its memory bandwidth is only about two-thirds of the latter.
50%UnverifiedECC error-correcting memory can automatically detect and correct single-bit memory errors; gaming cards such as the RTX 5090 typically do not include ECC, while the RTX Pro 6000, NVIDIA A-series, and H-series professional cards all come with ECC as standard.
50%UnverifiedIf eighty percent of one's work involves local LLMs, very large models, Coding Agents, and one cares about quiet operation, power consumption, and maintenance costs, the M5 Ultra—especially the 256GB or above version—is recommended; if the work involves image generation, video generation, LoRA, 3D, or training, the RTX Pro 6000 is recommended.
50%UnverifiedThe fact that 96GB of VRAM can fit two models simultaneously does not mean two tasks can run at full speed simultaneously, because GPU compute resources are shared and competed over.
50%UnverifiedOn an RTX Pro 6000 (96 GB VRAM), the official FP8 version combined with built-in MTP speculative decoding can reach approximately 120 tokens/second, with quality nearly indistinguishable from 16-bit.
50%UnverifiedCommunity benchmark comparison data: M2 Ultra approximately 20 tokens/second, M3 Ultra 512 GB at 39.8 tokens/second, a single B200 at 119.7 tokens/second, 8× RTX Pro 6000 at approximately 204 tokens/second, a modified RTX 4090 with 48 GB VRAM on a custom backend at 367 tokens/second, and dual H200 at approximately 375 tokens/second.
50%