ZYFA
An open-source FP8 and BF16 prefill attention kernel for AMD Instinct MI355X, optimized based on the FlashAttention approach, surpassing AITER on real model activation values.
Timeline (last 90 days)
Zyphra has released the open-source project ZYFA, an FP8 and BF16 prefill attention kernel optimized for AMD Instinct MI355X.
ZYFA's FP8 kernel is 1.16 to 1.76 times faster than AMD AITER on real model activation values.
In the head dimension 128 scenario, ZYFA reduces error by 62% to 78% compared with AITER.
ZYFA also provides a BF16 kernel, allowing teams to make per-layer trade-offs within the same kernel system rather than stitching together multiple libraries.
When evaluating ZYFA, the 1.16 to 1.76 times speedup range should be treated as an upper-bound reference rather than an expected value; it is recommended to conduct A/B comparisons using inputs sampled from production traffic and to monitor P99 latency.
The 62% to 78% error reduction was measured only at head dimension 128, and performance at other head dimensions needs to be verified separately.
For numerically sensitive tasks such as mathematical reasoning and long-context retrieval, it is recommended to run spot checks on ZYFA's output consistency under low traffic to confirm that precision changes do not propagate to final results.
All Facts (7)
ZYFA also provides a BF16 kernel, allowing teams to make per-layer trade-offs within the same kernel system rather than stitching together multiple libraries.
50%UnverifiedZYFA's FP8 kernel is 1.16 to 1.76 times faster than AMD AITER on real model activation values.
50%UnverifiedIn the head dimension 128 scenario, ZYFA reduces error by 62% to 78% compared with AITER.
50%UnverifiedFor numerically sensitive tasks such as mathematical reasoning and long-context retrieval, it is recommended to run spot checks on ZYFA's output consistency under low traffic to confirm that precision changes do not propagate to final results.
50%UnverifiedWhen evaluating ZYFA, the 1.16 to 1.76 times speedup range should be treated as an upper-bound reference rather than an expected value; it is recommended to conduct A/B comparisons using inputs sampled from production traffic and to monitor P99 latency.
50%UnverifiedThe 62% to 78% error reduction was measured only at head dimension 128, and performance at other head dimensions needs to be verified separately.
50%UnverifiedZyphra has released the open-source project ZYFA, an FP8 and BF16 prefill attention kernel optimized for AMD Instinct MI355X.
50%