Running YuE Music Generation on a 16GB AMD GPU: Full ROCm Port of 9070XT Tested

A 16GB AMD GPU Writes a Complete Chinese Song

For a long time, AI music generation models have been almost exclusively the domain of NVIDIA GPUs. Official recommended specs typically demand 24GB of VRAM on an NVIDIA card, effectively locking AMD users out. A hands-on test by a Bilibili content creator has broken that pattern — using a 16GB AMD 9070XT, they fully generated a complete 2-minute-55-second Chinese song called That Summer's Promise entirely on local hardware, with lyrics, melody, vocals, and accompaniment all produced by the model on the same machine.

The star of the show is the open-source music generation model YuE. It uses a two-stage architecture: first, it uses symbolic planning to write an editable ABC notation score, then synthesizes it into audio — with melody and instrumentation both modifiable at the intermediate stage. The model weights were released jointly by the Moonnight 2 (m-a-p) team, Hong Kong University of Science and Technology, and MAP, under a non-commercial license for research and educational use only.

More importantly, this wasn't just a "capability flag" demo — it was a real end-to-end run that produced actual output files. All four capabilities were fully validated: song generation, audio-to-score transcription, score rendering, and reference voice timbre. Peak VRAM usage was only 11–12GB, far below the official 24GB threshold.

CUDA to ROCm: There's Actually Very Little to Change

The most counterintuitive finding: across the entire song generation pipeline, the amount of code that's genuinely "NVIDIA-exclusive" is surprisingly small. The T8 workflow repository being ported (by author T8star) doesn't even contain the word "AMD" anywhere, yet migrating to ROCm turned out to be far less difficult than expected.

The only CUDA-looking code in the generation path is just two lines — torch.cuda.is_available() and is_bf16_supported(). Under ROCm, torch.cuda is itself an alias for HIP, so both lines return true values as expected. The word "NVIDIA" that appears in error messages is just display text and plays no role in the actual decision logic.

The model weights match the upstream version byte-for-byte: the hashes for YuE 2 3B and the VAE are identical, and files downloaded from the official repository match those from mirror sources exactly. So the only real changes needed are in the installation scripts — swapping the CUDA PyTorch wheel for AMD's ROCm-indexed version. At the source code level, only five changes were made, each with a clear rationale.

ROCm (Radeon Open Compute) is AMD's open-source software stack for GPU general-purpose computing, functionally analogous to NVIDIA's CUDA. ROCm's core component, HIP (Heterogeneous-computing Interface for Portability), provides an API closely mirroring CUDA, allowing most CUDA code to be ported with minimal changes. PyTorch's AMD build is constructed on top of HIP, which is why the torch.cuda namespace is preserved under ROCm and mapped to HIP's corresponding implementations. This "API alias" mechanism is the key to understanding why the migration was far easier than anticipated: most deep learning frameworks already abstract away hardware differences at a higher level. Real incompatibilities tend to be concentrated in a small number of places that depend on hardware-specific instruction sets or runtime libraries.

Five Changes and Two Deep Pitfalls

Each of the five changes addresses a specific issue: the attention backend during decoding is incorrectly identified as flash attention and needs correction; the VAE chunk frame count implicitly assumes a 24GB card; the audio toolchain triggers distributed operator evaluation at import time; the model's remaining FP32 modules cause compilation failures; and audio saving needs to switch to the soundfile backend to work around MIOpen.

Two pitfalls proved especially deep. The first: MIOpen needs to compile C++ kernels at runtime, but the environment lacks standard library headers — and even manually flattening 1,700+ header files into Clang's resource directory doesn't help. The second: dependency hell — the package manager independently resolves Torch and TorchAudio and assembles a combination with mismatched interfaces. The fix requires locking all three environments to the same version.

The author also distilled two engineering principles: patches must be idempotent and replayable — applied to a clean upstream codebase, all eight anchors must match, skip on rerun, and produce output that's byte-for-byte identical to the original; and original file line endings must be preserved, or a two-line fix will balloon into a diff spanning the entire file.

MIOpen is AMD GPU's deep learning primitives library, the equivalent of NVIDIA's cuDNN, handling high-performance implementations of convolutions, normalization, activation functions, and other operators. Unlike cuDNN which ships with pre-compiled kernels, MIOpen dynamically compiles C++ kernels at runtime when it encounters a new operator configuration (known as kernel fusion or JIT compilation). This process requires a complete C++ toolchain and system headers to be present locally. This is why simply copying 1,700+ header files into a directory doesn't solve the problem — MIOpen's compilation flow has strict expectations about header organization and dependency paths. This pitfall is especially common in containerized deployments, where standard library headers are often stripped out to minimize image size.

Chunk Frame Count: An Accidental 35.9% Speedup

The second change delivered the biggest surprise. The same request went from 613 seconds down to 393 seconds — a 35.9% improvement. VAE decoding alone dropped from 313 seconds to 107 seconds, or from 3.5 seconds per audio second down to 2.25 seconds — actually faster than the 423 seconds produced by the official CLI running in isolation.

The reason lies in the chunk frame count. On a 16GB card, 1024 frames hits a significantly slower kernel. For the same 60-second audio segment, 512 frames takes only 101 seconds, while 1024 frames takes 231 seconds — and no chunking at all takes 285 seconds. This demonstrates that matching hardware to the right kernel configuration can matter more than simply having more VRAM.

Correctness is backed by a hard signal: given the same seed and the same lyrics, the official CLI and the T8 code path — two completely different implementations — produce audio of exactly the same duration: 174.91866 seconds. This confirms that the entire port introduced zero behavioral drift.

Four Capabilities Tested and the No-Go Zones

The four capability benchmarks are clear: song generation produces 175 seconds of audio; audio transcription processes a 4-second clip in 36 seconds, outputting ABC notation and MIDI; score rendering outputs a printable PDF and a piano playback audio file; reference voice timbre completes in 47 seconds.

It's important to draw clear lines: transcription uses Sheet Sage combined with notation tools, then rendered to sheet music via abcjs; and reference voice timbre is post-processing added by the T8 author — separating vocals, swapping timbres, and remixing — with zero code references to the YuE model itself. This is entirely different from the official "cover" feature, which regenerates a new vocal performance after the score stage.

Finally, a few hard no-go zones. When installing the AMD runtime, you must use the correct package manager — otherwise a failed install will silently fall back to a CUDA version, so you should assert that no CUDA runtime is present. The directory layout has hard requirements; a standard virtual environment as a substitute will fail. Most critically: never enable FP8 quantization. The capability gate will pass, matrix multiplication will run, but relative errors reach 90–117% — this isn't precision loss, it's a computation error that silently produces garbage audio.

FP8 (8-bit floating point) quantization compresses model weights and activations to an 8-bit representation to reduce VRAM usage and accelerate inference. FP8 comes in two common formats: E4M3 (4-bit exponent, 3-bit mantissa) and E5M2 (5-bit exponent, 2-bit mantissa), suited to different numerical range scenarios. On hardware and driver combinations with solid support, FP8 quantization can deliver significant speedups with minimal precision loss. However, FP8 hardware support is very new, implementations vary across GPU architectures, and software stack support is incomplete. The 90–117% relative error described in this article is not minor precision degradation — it's a fundamental computational failure, likely due to miscalculated quantization scaling factors or defective format conversion, resulting in completely corrupted numerical values. The danger is that the model doesn't throw an error; it produces an audio file normally — it's just noise.

Closing: The Boundaries of the Open-Source Ecosystem Are Expanding

From a licensing standpoint: YuE weights are non-commercial, for research and learning only; generated works should be labeled "AI-generated"; model input accepts only style descriptors and lyrics (with an optional score), not audio references. The complete port is open-sourced on GitHub, including a full six-step workflow: cloning the upstream repository, installing three ROCm runtimes, applying the port patch, pulling the 12.2GB model with hash verification, and double-clicking the startup script to run the capability self-check.

The value of this test goes beyond "AMD can run it too." It demonstrates that many so-called hardware barriers are essentially illusions created by software default configurations. When a consumer-grade 16GB AMD GPU can generate a complete song locally, the barrier to local AI creative work is genuinely coming down.