State Space Model (SSM)
A class of neural network architectures that process sequences through linear recurrence, compressing historical information into a hidden state that evolves over time. Per-step decoding computation is independent of sequence length and memory usage is constant. Representative models include Mamba and Mamba-2.
Core Facts
State space models (SSMs) describe the evolution of a sequence via recurrence over a hidden state, and their continuous-time form can be discretized into a linear recurrence, handling long sequences with linear complexity.
State space models (SSMs) such as the Mamba series follow the same recurrent paradigm: a fixed state cannot losslessly retain all historical details, and earlier information gets overwritten by subsequent updates.
State Space Models (SSMs) pull the direction back toward fixed-size recurrent memory, but use a different state structure and update rules.
Timeline (last 90 days)
The core recurrence of an SSM is h_t = A_t h_{t-1} + B_t x_t, with output y_t = C_t h_t.
SSMs consistently lag behind Transformers on tasks requiring in-context learning and precise retrieval.
State space models (SSMs) describe the evolution of a sequence via recurrence over a hidden state, and their continuous-time form can be discretized into a linear recurrence, handling long sequences with linear complexity.
SSM's ability to perform precise retrieval over ultra-long contexts remains disputed.
Compared with the quadratic complexity of Transformer, SSM has the advantages of linear time complexity and constant inference memory in long-sequence modeling.
State space models (SSMs) such as the Mamba series follow the same recurrent paradigm: a fixed state cannot losslessly retain all historical details, and earlier information gets overwritten by subsequent updates.
Most SSM and linear attention models use a linear attention mechanism at the bottom layer, with Mamba-2 implemented through the SSD structure and xLSTM implemented through covariance updates.
State Space Models (SSMs) pull the direction back toward fixed-size recurrent memory, but use a different state structure and update rules.
All Facts (8)
State space models (SSMs) describe the evolution of a sequence via recurrence over a hidden state, and their continuous-time form can be discretized into a linear recurrence, handling long sequences with linear complexity.
80%VerifiedState space models (SSMs) such as the Mamba series follow the same recurrent paradigm: a fixed state cannot losslessly retain all historical details, and earlier information gets overwritten by subsequent updates.
75%VerifiedState Space Models (SSMs) pull the direction back toward fixed-size recurrent memory, but use a different state structure and update rules.
75%UnverifiedMost SSM and linear attention models use a linear attention mechanism at the bottom layer, with Mamba-2 implemented through the SSD structure and xLSTM implemented through covariance updates.
70%UnverifiedThe core recurrence of an SSM is h_t = A_t h_{t-1} + B_t x_t, with output y_t = C_t h_t.
50%UnverifiedSSMs consistently lag behind Transformers on tasks requiring in-context learning and precise retrieval.
50%UnverifiedCompared with the quadratic complexity of Transformer, SSM has the advantages of linear time complexity and constant inference memory in long-sequence modeling.
50%UnverifiedSSM's ability to perform precise retrieval over ultra-long contexts remains disputed.
50%