[KongchangAI]
ProductZYFA attention kernel

ZYFA

An open-source FP8 and BF16 prefill attention kernel for AMD Instinct MI355X, optimized based on the FlashAttention approach, surpassing AITER on real model activation values.

Timeline (last 90 days)

Nov 6

Zyphra has released the open-source project ZYFA, an FP8 and BF16 prefill attention kernel optimized for AMD Instinct MI355X.

Unverified50%
Nov 6

ZYFA's FP8 kernel is 1.16 to 1.76 times faster than AMD AITER on real model activation values.

Unverified50%
Nov 6

In the head dimension 128 scenario, ZYFA reduces error by 62% to 78% compared with AITER.

Unverified50%
Nov 6

ZYFA also provides a BF16 kernel, allowing teams to make per-layer trade-offs within the same kernel system rather than stitching together multiple libraries.

Unverified50%
Nov 6

When evaluating ZYFA, the 1.16 to 1.76 times speedup range should be treated as an upper-bound reference rather than an expected value; it is recommended to conduct A/B comparisons using inputs sampled from production traffic and to monitor P99 latency.

Unverified50%
Nov 6

The 62% to 78% error reduction was measured only at head dimension 128, and performance at other head dimensions needs to be verified separately.

Unverified50%
Nov 6

For numerically sensitive tasks such as mathematical reasoning and long-context retrieval, it is recommended to run spot checks on ZYFA's output consistency under low traffic to confirm that precision changes do not propagate to final results.

Unverified50%

All Facts (7)

Source Articles