Cracking the PS5 to Run Qwen 27B: GPU Inference Hits 20 tok/s

When a Game Console Becomes an AI Inference Device

A striking post recently appeared on Reddit: someone successfully ran the Qwen 27B large language model on a jailbroken PS5 using its GPU, achieving an inference speed of about 20 tokens per second. This means this consumer-grade hardware, originally designed for gaming, is being redefined as an AI inference platform.

For those familiar with the AI hardware ecosystem, this number is no small feat. Qwen 27B is a medium-to-large language model with 27 billion parameters, typically requiring high-end consumer GPUs (like RTX 3090/4090 class) or professional accelerator cards to run smoothly. The PS5's GPU is based on AMD's RDNA 2 architecture, with theoretical floating-point performance of about 10.3 TFLOPS—excellent among game consoles, but not designed for AI training or inference scenarios.

The core value of this experiment lies not in the performance itself, but in the fact it reveals a long-overlooked truth: the GPU compute power of a large number of idle or second-hand game consoles could potentially be repurposed for local AI inference.

Technical Path to Running LLMs on a Jailbroken PS5

Hardware-Level Feasibility

The PS5 is powered by a custom AMD APU containing an 8-core Zen 2 CPU and a 36-CU RDNA 2 GPU. From a hardware spec perspective, it supports FP16 and INT8 operations, with 16GB GDDR6 shared between VRAM and system memory. While 16GB is quite tight for a 27B model—even with 4-bit quantization, model weights require about 14-15GB—this is just barely on the edge of feasibility.

The Reddit post didn't detail the specific quantization scheme used, but the community speculates it likely employed GGUF format 4-bit quantization (such as Q4_K_M) and loaded the model into GPU-accessible memory through some method.

Quantization technology is the key prerequisite for understanding this experiment. LLM weights are typically stored in FP16 (16-bit floating point) format, so a 27B model would need about 54GB of VRAM, far exceeding the PS5's 16GB. Quantization compresses model size by reducing the numerical precision of each weight: 4-bit quantization represents each weight with 4 bits, shrinking the model to about 1/4 of its size, roughly 14GB; 8-bit quantization yields about 27GB. GGUF is a model file format popularized by the llama.cpp community, supporting multiple quantization levels (e.g., Q4_K_M denotes a medium-quality 4-bit quantization variant), significantly reducing memory usage while keeping precision loss manageable. The cost of quantization is a possible slight decline in inference quality, but for most dialogue and Q&A scenarios, the performance loss from 4-bit quantization is usually within acceptable range.

RDNA 2 architecture is AMD's GPU microarchitecture launched in 2020, used in the PS5, Xbox Series X, and RX 6000 series graphics cards. It supports FP16 (half-precision floating point) and INT8 (8-bit integer) operations, both crucial for AI inference—FP16 doubles computational throughput while maintaining sufficient precision, and INT8 further boosts throughput at the cost of greater precision loss. However, RDNA 2 lacks the Tensor Cores dedicated to matrix multiplication found in NVIDIA GPUs, and its general-purpose computing efficiency is relatively limited—one of the hardware roots of AMD's long-standing lag behind NVIDIA in AI inference.

Software-Level Breakthroughs

Running LLMs in a jailbroken environment requires solving several key problems: bypassing system restrictions to gain low-level GPU access, building a compatible inference framework, and managing limited memory resources. Community members speculate the implementer may have ported an inference engine similar to llama.cpp and optimized it for the RDNA 2 architecture.

A speed of 20 tok/s is acceptable for a 27B model. For reference, running Qwen 27B at the same quantization level on a high-end PC typically achieves 30-50 tok/s. That the PS5 can reach 20 tok/s shows its GPU's general-purpose computing capability was mobilized fairly effectively.

Why This Matters

Hardware Diversification for Local AI Inference

The current mainstream approach to local AI inference relies heavily on the NVIDIA ecosystem. The maturity of CUDA and the refinement of acceleration libraries like cuDNN have made NVIDIA GPUs almost the default choice. But this experiment reminds us that AMD's RDNA architecture also has considerable inference potential—it's just that the software ecosystem has long lagged behind.

If devices like the PS5 can run LLMs at reasonable speeds, then similar AMD APU devices like the Xbox Series X, or even the Steam Deck, should theoretically be similarly feasible. This would open an entirely new hardware path for local AI inference.

The CUDA ecosystem is the core of NVIDIA's moat in AI. CUDA (Compute Unified Device Architecture) is NVIDIA's general-purpose GPU computing platform launched in 2007. After more than a decade of development, it has formed a complete toolchain including cuDNN (deep neural network acceleration library), TensorRT (inference optimization engine), and native support for PyTorch/TensorFlow. When developers deploy models on NVIDIA platforms, virtually all mainstream frameworks work out of the box. By contrast, AMD's counterpart ROCm (Radeon Open Compute) started later, has limited support for consumer GPUs, and its optimization libraries for LLM inference are far less mature than CUDA. This leads to a clear gap in the convenience and efficiency of actual inference deployment, even when AMD GPUs are comparable on paper. The PS5 jailbreak environment bypasses the official software stack and directly calls low-level GPU resources, thereby actually unlocking the hardware's potential.

Implications for AI Compute Cost Structure

A used PS5 costs far less than an RTX 4090. If 20 tok/s can satisfy certain scenarios (such as personal assistants, offline translation, or local knowledge base Q&A), then replacing graphics cards with game consoles to build low-cost inference nodes is economically reasonable.

Of course, the legal risks and system stability issues brought by jailbreaking cannot be ignored. This is more a proof of concept than a promotable product solution.

When evaluating the economics of local inference solutions, cost per token and performance per watt are two often-overlooked but crucial metrics. A speed of 20 tok/s is basically acceptable for interactive scenarios like personal assistants or offline translation, but for batch text processing, it's far less efficient than cloud APIs. More importantly, power consumption: the PS5's total power draw under gaming load is about 200W, which inference likely approaches, while an RTX 4090 draws about 300-400W during inference but is more than twice as fast. Therefore, the PS5 solution's economic advantage mainly lies in hardware acquisition cost rather than operating cost. Additionally, a used PS5 costs about 2,000-3,000 yuan, while an RTX 4090 costs over 12,000 yuan. For budget-sensitive individual users who don't demand high speed, the former's value for money is indeed worth considering—provided they're willing to bear the jailbreak risk.

Limitations and Controversies

This experiment still leaves several key questions insufficiently answered. First is the memory bottleneck: 16GB of shared memory must simultaneously host both the system and the model, leaving even less actual available space, potentially facing severe performance degradation in long-context scenarios. Second is the sustainability of the jailbreak environment—Sony's firmware updates could block this path at any time.

Moreover, the 20 tok/s measured data comes from a single source and has yet to be verified by third-party reproduction. Inference stability, thermal performance, and long-term reliability all remain unknown.

The Boundaries of Local Inference Are Being Redefined

This Reddit post sparked discussion because it touches on a deeper issue: where exactly are the hardware boundaries of AI inference? As game consoles, phones, and even embedded devices are all being tried for running LLMs, our understanding of "what devices can run AI" is being continuously broken.

For developers and tech enthusiasts, the value of such experiments lies not in whether they're practical, but in expanding the boundaries of possibility. Perhaps in the near future, your living room game console really could, while on standby, help run a local AI service on the side.