Running LLMs Locally on 32GB GPUs: AMD R9700 vs Intel B70

Want to run large language models locally? VRAM is everything. An RTX 5090 costs $4,000–5,000 just to get you 32GB of VRAM, while AMD and Intel have each launched a workstation-class GPU aiming to crack that market at a lower price — the Radeon AI Pro R9700 and the Arc Pro B70. Both pack 32GB of VRAM with a 256-bit bus, memory bandwidth within 5% of each other, and are priced roughly $450 apart. On paper they should post similar speeds. In practice, the results tell a very different story.

This article is based on analysis from YouTuber RepoChad, combined with benchmark data from Puget Systems, Tom's Hardware, Storage Review, and other sources. It breaks down how these two "value cards" actually perform for local inference.

Price: The Most Volatile Variable

Both cards saw dramatic price swings this summer, rendering any older review's "value" conclusions obsolete. The R9700 launched at $1,299 and the B70 at $949 — but neither of those numbers holds anymore.

As of September price tracking, the R9700 sits in the $1,700–1,810 range, while the B70 runs $1,300–1,460. Tom's Hardware reported the B70 jumping from $999 to $1,299 in early August — a roughly 30% spike in a single move. Both cards are up about 25–27% from their three-month averages.

Breaking it down by VRAM capacity, the B70 works out to about $41/GB versus roughly $55/GB for the R9700. If you're purely buying for capacity, that gap is substantial — and it's the B70's strongest selling point.

Hardware Specs: Bandwidth Is Tied, Compute Is Not

The R9700 is built on the RDNA4 architecture with 64 compute units, 4,096 stream processors, 128 AI accelerators, 640 GB/s bandwidth, 64 MB of Infinity Cache, 47.8 TFLOPS of FP32, and 191 TFLOPS of dense FP16 matrix throughput.

The B70 runs on the XE2 Battlemage architecture (TSMC N5), with 32 XE cores, 256 XMX engines, 608 GB/s bandwidth, and roughly 23 TFLOPS of FP32.

INT8 numbers look close at a glance — AMD claims 383 TOPS (766 sparse), Intel claims 367 TOPS without specifying whether sparse is included. Puget Systems' analysis shows the R9700 delivering roughly 2× the FP32 throughput of the B70 and approximately 5× the matrix throughput. The conclusion is clear: bandwidth is roughly matched, compute is not.

A few specs worth clarifying here. FP32 (single-precision floating point) is the baseline unit for traditional graphics and general compute, reflecting peak throughput for standard floating-point operations. FP16/BF16 (half-precision) is the common precision for LLM inference — modern GPUs include dedicated matrix multiply units (AMD's AI accelerators, Intel's XMX engines) that dramatically accelerate these operations, so FP16 matrix throughput is often several times or even an order of magnitude higher than FP32. INT8/INT4 refers to quantized integer operations: lower precision, faster execution, smaller memory footprint, with TOPS (Tera Operations Per Second) as the standard unit. Sparse acceleration exploits the fact that large fractions of weight matrices are near-zero, theoretically doubling throughput — but whether a vendor's stated figure includes sparsity directly affects comparability, which is exactly why the Intel INT8 number is questionable here.

Memory Bandwidth Sets the Theoretical Ceiling

Why does bandwidth matter so much? During local inference, generating every single token requires reading the full set of model weights once. For a 27B model at roughly 4.5 bits per weight, that's about 15 GB. Divide the R9700's 640 GB/s by 15 and you get a theoretical ceiling of around 42 tokens/s; the B70's 608 GB/s gives roughly 40 tokens/s.

Those are theoretical ceilings with no tricks applied, and neither card comes close to hitting them. In practice, running Qwen3 27B in llama.cpp (Linux + ROCm 7.2.4, IQ4NL quantization), the R9700 achieves 27.1 tokens/s — about 65% of the bandwidth ceiling. The B70 (Vulkan build, Q4KXL) hits 21.4 tokens/s, just over 50%. Same memory bus width, but AMD squeezes out significantly more usable performance. The gap comes down to compute and software.

Multi-Token Prediction and Driver Updates: Two Variables That Doubled Numbers in Two Months

Two things happened this summer that completely changed these numbers.

First: Multi-Token Prediction (MTP). Qwen3 has MTP built in — the model can draft several tokens ahead, and the GPU verifies them all at once. At a draft depth of 2, the R9700 jumps from 27.1 to 46.2 tokens/s (+70%), while the B70 climbs from 21.4 to 28.4 (+33%). Verifying multiple tokens at once is closer to a mini-batch operation, and batch operations are compute-bound — so the AMD card with its much higher matrix throughput benefits far more.

A word of caution on data sources: AMD's official blog claims 51.8 tokens/s on Windows, while independent Linux testing shows 46.2. Both can be valid in their respective environments. A "227 tokens/s" figure has circulated online, but it traces back to a single social media post about an experimental DeepSeek^Ad Flash configuration that no one has reproduced.

One result that does flip the rankings: a B70 user running vLLM + GPT-Q INT4 quantization at MTP depth 4 on the same 27B model achieved 84.6 tokens/s with roughly 0.3s time-to-first-token. The catch: single-stream only — two overlapping requests could crash — and it uses a different quantization scheme, making direct comparison impossible.

Second: MESA 26.1 drivers. The impact is biggest on MoE models. On Qwen3 30B-A3B class models (which activate only ~3B parameters per token, drastically reducing compute demand), the R9700 runs around 77 tokens/s, while the B70's Vulkan path more than doubled from 38 to 76 — essentially a tie. Vulkan also scales better under load, reaching 170–176 tokens/s across 8 concurrent streams, well ahead of SYCL's ~100. The old advice to "pick SYCL over Vulkan on Intel" is now outdated.

Multi-Token Prediction (MTP) is essentially a variant of speculative decoding. Traditional autoregressive generation produces one token at a time, with each new token waiting on the last — a strict serial bottleneck that keeps the GPU mostly waiting on memory reads rather than doing real matrix computation. The core idea of MTP/speculative decoding: a lightweight draft module (built directly into Qwen3) guesses several candidate tokens at once; the main model then verifies them in parallel, accepting the correct ones and discarding the rest. The verification step is fundamentally a small-batch matrix operation that feeds the GPU more work simultaneously, which is why compute-heavy cards like the R9700 benefit more. This also explains why the AMD card — with roughly 5× the matrix throughput of the B70 — sees a 70% gain at draft depth 2, while the B70 only gains 33%.

Workload Splits: What Are You Actually Running?

Conclusions depend heavily on workload type:

• Dense mid-size models (e.g., 27B): AMD R9700 wins clearly.
• MoE models: Essentially a tie.
• Small models: B70 has a slight edge. Puget's testing of Llama 3.1 8B (FP16, vLLM) showed the B70 at 35.4 vs. 29.1 for the R9700; in MLPerf Client, the B70 is about 7% faster on token generation; 1024² image generation is nearly identical (3.6s vs 3.9s).

Translating to cost per token/s: at R9700 street price $1,750 and B70 at $1,300, using MTP results, AMD comes out to roughly $38/(token/s) versus Intel's $46/(token/s). Even at $450 more, the AMD card delivers a lower "cost per token" on dense models — the B70 would need to drop to around $1,080 to break even. On MoE workloads where speeds are near-equal, the B70 wins on price.

Software Ecosystem: The Real Dividing Line

Neither card is plug-and-play like CUDA, but the software experience differs enormously.

R9700 has a shorter path. On Windows, use LM Studio + Vulkan with MTP draft tokens set to 2; AMD recommends unchecking "try alloc." On Linux, HIP + MTP for serving is roughly twice as fast as Vulkan. Standard vLLM ROCm images support two-card tensor parallelism. Caveats exist: multi-GPU RCCL fails in VMs with PCIe passthrough, some Qwen3 builds have broken row split, and ECC is Linux-only.

B70 asks more of the user. Vulkan works out of the box in llama.cpp, but full speed requires MESA 26.1+ (i.e., Linux). Stock Ollama runs on the CPU instead of the GPU. There's no native Windows path for vLLM — you need WSL2 + Docker. Multi-GPU Vulkan on Windows is outright broken. vLLM XPU fails on bfloat16 models like Gemma, and there's an output corruption bug on Battlemage that requires an environment variable workaround.

Storage Review put it well: the hardware is ready; the software stack still needs work. In practice, you're maintaining the drivers and runtimes yourself.

Understanding the software paths requires some familiarity with the inference stack. llama.cpp is currently the most universal local inference engine, supporting CPU, CUDA, ROCm, Vulkan, SYCL, and more backends with the broadest quantization format support — but it's primarily designed for single-user scenarios. vLLM is a high-concurrency serving framework with tensor parallelism (distributing model weights across GPUs) and continuous batching, the mainstream choice for production deployments, though its ROCm/XPU support lags well behind CUDA. Vulkan is a cross-platform graphics/compute API supported by both AMD and Intel without vendor-specific drivers, offering the best compatibility at the cost of usually lower performance than native paths. ROCm is AMD's CUDA equivalent; SYCL/oneAPI is Intel's counterpart — both have better low-level optimization potential than Vulkan on Linux, but neither has complete Windows support. Ollama is essentially a wrapper around llama.cpp whose GPU detection logic remains unreliable for non-NVIDIA hardware, causing the B70 to fall back to CPU in default configurations.

Don't Overlook the Used RTX 3090

A used RTX 3090 currently runs about $1,050–1,450 on eBay — roughly B70 money. It only has 24GB instead of 32GB, but community benchmarks show it hitting 41 tokens/s on Qwen3 27B and 63 tokens/s with MTP — faster than either new card, and it brings CUDA, which eliminates virtually all the configuration headaches. If your quantization and context window fit in 24GB, it's a hard card to argue against. The trade-offs are buying used and losing 8GB of VRAM.

Final Recommendation

If you prioritize dense-model speed, a workable Windows path, a more mature vLLM stack, and your budget can absorb the higher street price — go with the R9700.

If you want the cheapest entry ticket to 32GB of workstation VRAM and mostly run MoE models, and you're willing to navigate Linux driver and runtime quirks — the B70 makes sense.

But go in with clear eyes: MTP and a single MESA update shifted these numbers by 30% to 120% over two months. Any benchmark from spring is essentially obsolete, and some of the figures in this article will be outdated by the next driver release. For local AI overall, it's genuinely good news that there are finally two real 32GB options outside NVIDIA — even if at current prices, neither has managed to dethrone the used RTX 3090.