Unverified50% confidenceFactExact time
Hymba是NVIDIA于2024年提出的混合头注意力架构，在同一层内同时部署SSM头和注意力头，通过可学习门控机制融合输出，在1.5B参数规模下超越同规模纯Transformer和纯SSM模型
1
Sources
50%
Confidence
Long-term
Relevance
8/29/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedNemotron-H 是 NVIDIA 于 2025 年发布的混合架构系列模型，将 Mamba 状态空间模型层与标准 Transformer 注意力层交替堆叠68% similarUnverifiedMiMo V2.5 Pro supports an ultra-long context window of 1 million tokens and employs a hybrid attention mechanism.66% similarUnverifiedDeepSeek-V4 Flash采用混合注意力、mHC混合机制以及哈希路由的MoE等架构设计64% similarUnverifiedMegatron-LM was the first to systematically propose a tensor parallelism scheme for Transformers, splitting attention heads and FFN layers across different devices.64% similarUnverifiedLLaMA-2 70B采用80层、64个注意力头的架构，并使用GQA（8个KV头共享64个Q头）63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/816411API
curl https://kongchang.com/api/v1/knowledge/claims/816411MCP
get_claim(id=816411)