Mamba2
An efficient sequence modeling architecture based on state space models (SSM). It ties the state dimension to input channels and supports a matrix-multiplication form of parallel computation, offering linear complexity and constant inference memory usage. It serves as an alternative or complement to Transformers in long-context scenarios.
Timeline (last 90 days)
The issue of the Mamba2 chunk scan kernel producing NaN on SM90 is very likely a numerical stability defect triggered by architectural differences.
vLLM released version v0.32.0rc3, which includes a bugfix for the Mamba2 chunk scan kernel producing NaN outputs on the NVIDIA SM90 architecture.
Mamba2 is a sequence modeling architecture based on the state space model (SSM), which binds the SSM state dimension to the input channels and rewrites the per-timestep serial recurrence into matrix multiplication form, allowing parallel computation using GPU Tensor Cores.
Mamba2's chunk scan mechanism adopts a two-stage strategy of intra-chunk parallelism and inter-chunk serialism, reducing the degree of parallelism from the sequence-length level to the chunk-count level.
Mamba2 introduces a more structured state space duality form on top of Mamba, enabling recurrent computation and parallel scan to be combined more efficiently and improving hardware utilization.
The Mamba2 architecture has linear time complexity and a constant-size state representation, and its state representation does not grow with sequence length.
Transformer's KV cache size grows linearly with sequence length, whereas Mamba2's recurrent state cache size is independent of context length and depends only on the state dimension and the number of layers.
Mamba2 maintains linear time complexity and constant state size, and therefore has a theoretical advantage in long-sequence scenarios.
Sample packing for hybrid-architecture models such as gated-delta, Mamba2, and short convolution has been improved and packing logic issues have been fixed.
All Facts (9)
The issue of the Mamba2 chunk scan kernel producing NaN on SM90 is very likely a numerical stability defect triggered by architectural differences.
50%UnverifiedMamba2's chunk scan mechanism adopts a two-stage strategy of intra-chunk parallelism and inter-chunk serialism, reducing the degree of parallelism from the sequence-length level to the chunk-count level.
50%UnverifiedMamba2 is a sequence modeling architecture based on the state space model (SSM), which binds the SSM state dimension to the input channels and rewrites the per-timestep serial recurrence into matrix multiplication form, allowing parallel computation using GPU Tensor Cores.
50%UnverifiedMamba2 maintains linear time complexity and constant state size, and therefore has a theoretical advantage in long-sequence scenarios.
50%UnverifiedMamba2 introduces a more structured state space duality form on top of Mamba, enabling recurrent computation and parallel scan to be combined more efficiently and improving hardware utilization.
50%UnverifiedThe Mamba2 architecture has linear time complexity and a constant-size state representation, and its state representation does not grow with sequence length.
50%UnverifiedTransformer's KV cache size grows linearly with sequence length, whereas Mamba2's recurrent state cache size is independent of context length and depends only on the state dimension and the number of layers.
50%UnverifiedSample packing for hybrid-architecture models such as gated-delta, Mamba2, and short convolution has been improved and packing logic issues have been fixed.
50%UnverifiedvLLM released version v0.32.0rc3, which includes a bugfix for the Mamba2 chunk scan kernel producing NaN outputs on the NVIDIA SM90 architecture.
50%