Mamba 3
The third-generation version of the state space model architecture. It adopts a multiple-input multiple-output (MIMO) construction, supports complex rotation and dual-endpoint writes, and raises the local retrieval rank to enhance expressiveness.
Timeline (last 90 days)
The MIMO construction of Mamba 3 promotes one value at each local interface; SSD-MIMO routes independent values for cases where the local retrieval rank is greater than one.
HybridMamba combines Mamba 1's per-coordinate decay, Mamba 2-style head structure, and Mamba 3's complex rotations and dual-endpoint writes, mixing real state blocks with rotated state blocks of different decay laws.
The SISO configurations of Mamba 1-3 can be recovered as exact special cases of HybridMamba.
Retrieval rank refers to the number of independent value channels that a model can write to and read from in parallel at a local interface, and is a key structural dimension distinguishing variants such as Mamba 1, Mamba 2, Mamba 3 and HybridMamba.
On five large-scale H100 bfloat16 workloads, the authors' fused GPU kernels improved training time geometric mean relative to the official Mamba 1/3 under matched SISO recovery configurations.
In small-model multi-token recall experiments, both HybridMamba and SSD-MIMO designs outperformed Mamba 1-3 at the hardest level.
All Facts (4)
The MIMO construction of Mamba 3 promotes one value at each local interface; SSD-MIMO routes independent values for cases where the local retrieval rank is greater than one.
50%UnverifiedHybridMamba combines Mamba 1's per-coordinate decay, Mamba 2-style head structure, and Mamba 3's complex rotations and dual-endpoint writes, mixing real state blocks with rotated state blocks of different decay laws.
50%UnverifiedThe SISO configurations of Mamba 1-3 can be recovered as exact special cases of HybridMamba.
50%UnverifiedRetrieval rank refers to the number of independent value channels that a model can write to and read from in parallel at a local interface, and is a key structural dimension distinguishing variants such as Mamba 1, Mamba 2, Mamba 3 and HybridMamba.
50%