AMD 7900XTX Running Qwen3 on Strata: IQ3 Quantization Hits 60-80 t/s

AMD Cards Finally Work: Real-World Testing with Strata

For a long time, local LLM inference has been almost exclusively an NVIDIA affair — mainstream tools like llama.cpp either lacked AMD GPU support or simply wouldn't run. The open-source project Strata has changed the game. One content creator tested AMD Radeon RX 7900XTX on Windows 11 and successfully ran Qwen3, describing the overall experience as "pretty satisfying."

This article summarizes the key benchmarks and setup notes from that test, as a reference for anyone with an AMD GPU looking to run large models locally.

Benchmark Results: Usable Output Speed, but Slow Prompt Processing

The test machine was a typical budget workstation build: Intel Xeon E5-2686 V4, 256GB DDR3-1866 RAM, paired with a 7900XTX. The model used was the IQ3_XSS quantized version of Qwen3.

Results showed generation speed (output tokens) consistently at 60-80 t/s, averaging around 68 tokens/s — more than sufficient for everyday use. The bottleneck was the prompt processing (prefill) stage: the 7900XTX on Windows only managed around 400-479 t/s, while an RTX 4080S running Q4 quantization hit over 1,000 t/s — nearly an order of magnitude higher in throughput.

In theory, the 7900XTX and 4080S should be in roughly the same performance tier, making this gap surprisingly large. The tester suspects Windows 11 optimizations are still immature, since the same project reportedly achieves much higher prefill numbers on Linux.

The most noticeable pain point in practice is slow first-token latency: after entering a prompt, users often wait 10 seconds or more before output begins, and with a 256K context window, a single generation run took 743 seconds. In short, the output stream itself is fluid, but the "time to first token" experience is poor. For conversational tasks and text generation, it's workable once output starts.

---

It's worth understanding the two core performance metrics in LLM inference. The Prefill stage is when the model reads and processes the user's input prompt and builds the KV Cache — it's compute-bound and depends heavily on the GPU's parallel matrix computation throughput. The Decode stage is where the model generates tokens one by one — it's memory-bandwidth-bound and depends more on VRAM bandwidth. The 7900XTX's theoretical memory bandwidth (960 GB/s) actually exceeds the RTX 4080S (736 GB/s), which is why decode speeds (60-80 t/s) are relatively competitive. The prefill gap is more likely a software issue — ROCm on Windows hasn't matured to the same optimization level as on Linux — rather than a hardware limitation.

---

Installation & Configuration: Nearly Fully Automated Out of the Box

Strata's installation process is refreshingly straightforward, which is one of its biggest selling points. After downloading the source package, you simply run the startup script — most steps are handled automatically:

• The program automatically downloads pre-compiled llama.cpp and ROCm kernel binaries, so no manual HIP toolchain installation is needed;
• As long as you have AMD GPU drivers installed, it just works — no need to configure a ROCm environment;
• It also downloads an additional ~6GB data directory containing MTP (Multi-Token Prediction) related files.

Tip: Pointing to a Local Model Directory

By default, Strata prompts you to select a model through its UI and downloads it automatically, but doesn't directly support specifying a local model directory. If you've already downloaded GGUF models locally and want to avoid re-downloading, you have a couple of options:

• Use startup parameters that support directory specification (the tester mentioned gguf-dl can specify both the model directory and load range);
• Manually create the corresponding JSON config file under C:\Users\<username>\AppData\Roaming\strata, specifying the model path — the program will then load it directly without downloading.

Multi-GPU Configuration

If you have two GPUs, you'll need additional allocation parameters (HIP-related settings) to enable multi-GPU parallelism. The configuration is similar to single-GPU setup; the tester has shared screenshots of the relevant parameters.

---

ROCm (Radeon Open Compute) is AMD's open-source parallel computing platform, roughly analogous to NVIDIA's CUDA — providing compiler, runtime, and math library support for GPU general-purpose computing. However, ROCm has had virtually no official Windows support for years; AMD primarily maintains the Linux version, which is the root cause of AMD GPUs' long-standing disadvantage in the local inference ecosystem. Strata's key innovation is that it pre-compiles ROCm kernel binaries for Windows, so users don't need to set up a full ROCm development environment. This "pre-compiled, bundled distribution" approach dramatically lowers the barrier to entry for AMD GPU users.

---

Practical Use: Fast-Forward, Pause, and Speed Controls All Work

Beyond pure text generation, the tester also ran an image-related generation task and described the results as "decent," with support for fast-forward, pause, day/night switching, and 2x speed — clearing the bar for daily usability.

Note that the project currently appears to support only a single algorithm path, and features are still being iterated. But for AMD GPU users, this is already a meaningful breakthrough. At a time when llama.cpp and most Llama API solutions don't run smoothly on AMD hardware, Strata offers a viable path to local inference.

Summary: AMD Local Inference Is Worth Trying

This real-world test makes a few things clear: the 7900XTX running Qwen3 on Strata delivers fully usable output speeds, and setup is painless. However, prompt processing performance on Windows still lags noticeably behind comparable NVIDIA cards, and high first-token latency is currently the biggest weakness.

For users who already own a high-end AMD GPU and don't want to buy an NVIDIA card just to run models, Strata is an open-source solution worth watching. As Windows-side optimizations continue to mature, the gap between AMD and NVIDIA for local LLM inference is expected to narrow.