Unverified50% confidenceFactExact time
ZHA officially claims that it can match dense attention on reasoning tasks and on retrieval benchmarks at 256K length.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/3/2026
First Seen
Valid until: 2/1/2027
Sources
Related Entities
Related Claims
UnverifiedZHA (Hierarchical Attention), proposed by Zyphra, uses a recursively compressed tree-structured indexer that makes total cost grow sublinearly with sequence length, thereby achieving an overall sub-quadratic but superlinear attention complexity.73% similarUnverifiedEmsyAI uses a GQA attention mechanism configured with 16 query heads and 4 KV heads63% similarVerifiedThe computational complexity of standard self-attention is quadratic with respect to sequence length, and achieving a context of one million tokens requires introducing techniques such as sparse attention, sliding window attention, and KV Cache compression.62% similarVerifiedGQA (Grouped Query Attention) is an attention mechanism between Multi-Head Attention (MHA) and Multi-Query Attention (MQA), which reduces KV Cache memory usage by grouping multiple Query heads to share Key-Value projections.61% similarVerifiedThe Multi-Query Attention technique allows multiple attention heads to share a single set of Key and Value projection matrices, compressing KV cache memory requirements to 1/H of the original, at the cost of a slight reduction in model expressiveness.61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1129242API
curl https://kongchang.com/api/v1/knowledge/claims/1129242MCP
get_claim(id=1129242)