State Space Model
A class of deep learning architectures that perform sequence modeling with linear computational complexity. Representative models include Mamba. Compared with Transformer-based architectures, state space models are better suited for real-time inference scenarios due to their lower computational overhead.
Core Facts
Timeline (last 90 days)
To maintain usable efficiency at the tens-of-millions context scale, it is usually necessary to introduce optimizations such as sparse attention, chunked retrieval, state space models or hybrid architectures.
Typical representatives of state space models include S4, Mamba and others, which write the recurrence as a continuous differential equation or discrete linear recurrence and achieve efficient parallel training with the help of convolution or selective scan.
Priming面临的第一个质疑是：跨架构的权重映射不是总能找到合理对应关系，如果目标架构引入了全新计算模块（如状态空间模型层），预训练权重就没有现成组件可以填进去。
The study finds that the output geometries of Transformers, state space models, and recurrent neural networks are aligned to a far greater degree than their activation geometries.
State space models (SSMs) such as Mamba completely abandon the attention matrix and use recurrent hidden states to achieve O(N) complexity, but still fall short on tasks requiring precise key-value retrieval.
The paper conducts theoretical matching experiments covering three architectural classes—contractive state-space models, gated recurrent models, and attention models—and the experimental results reproduce the state and stability laws predicted by theory.
State Space Models (SSMs) pull the direction back toward fixed-size recurrent memory, but use a different state structure and update rules.
The paper claims that its method substantially outperforms Mamba and other state space models in the dynamical systems reconstruction setting.
The state transitions of SSMs are still essentially linear recurrent structures, creating a natural bottleneck in expressive power for reconstructing highly nonlinear chaotic attractors.
All Facts (10)
Mamba 等基于状态空间模型（SSM）的架构以线性复杂度实现序列建模，更适合实时推理
65%UnverifiedTo maintain usable efficiency at the tens-of-millions context scale, it is usually necessary to introduce optimizations such as sparse attention, chunked retrieval, state space models or hybrid architectures.
50%UnverifiedTypical representatives of state space models include S4, Mamba and others, which write the recurrence as a continuous differential equation or discrete linear recurrence and achieve efficient parallel training with the help of convolution or selective scan.
50%UnverifiedPriming面临的第一个质疑是：跨架构的权重映射不是总能找到合理对应关系，如果目标架构引入了全新计算模块（如状态空间模型层），预训练权重就没有现成组件可以填进去。
50%UnverifiedThe study finds that the output geometries of Transformers, state space models, and recurrent neural networks are aligned to a far greater degree than their activation geometries.
50%UnverifiedState space models (SSMs) such as Mamba completely abandon the attention matrix and use recurrent hidden states to achieve O(N) complexity, but still fall short on tasks requiring precise key-value retrieval.
50%UnverifiedThe paper conducts theoretical matching experiments covering three architectural classes—contractive state-space models, gated recurrent models, and attention models—and the experimental results reproduce the state and stability laws predicted by theory.
50%UnverifiedState Space Models (SSMs) pull the direction back toward fixed-size recurrent memory, but use a different state structure and update rules.
50%UnverifiedThe paper claims that its method substantially outperforms Mamba and other state space models in the dynamical systems reconstruction setting.
50%UnverifiedThe state transitions of SSMs are still essentially linear recurrent structures, creating a natural bottleneck in expressive power for reconstructing highly nonlinear chaotic attractors.
50%