[KongchangAI]
ConceptSSM

State Space Model (SSM)

A class of neural network architectures that process sequences through linear recurrence, compressing historical information into a hidden state that evolves over time. Per-step decoding computation is independent of sequence length and memory usage is constant. Representative models include Mamba and Mamba-2.

Core Facts

Timeline (last 90 days)

Nov 11

The core recurrence of an SSM is h_t = A_t h_{t-1} + B_t x_t, with output y_t = C_t h_t.

Unverified50%
Nov 11

SSMs consistently lag behind Transformers on tasks requiring in-context learning and precise retrieval.

Unverified50%
Nov 8

State space models (SSMs) describe the evolution of a sequence via recurrence over a hidden state, and their continuous-time form can be discretized into a linear recurrence, handling long sequences with linear complexity.

Verified80%
Oct 18

SSM's ability to perform precise retrieval over ultra-long contexts remains disputed.

Unverified50%
Oct 18

Compared with the quadratic complexity of Transformer, SSM has the advantages of linear time complexity and constant inference memory in long-sequence modeling.

Unverified50%
Oct 15

State space models (SSMs) such as the Mamba series follow the same recurrent paradigm: a fixed state cannot losslessly retain all historical details, and earlier information gets overwritten by subsequent updates.

Verified75%
Oct 14

Most SSM and linear attention models use a linear attention mechanism at the bottom layer, with Mamba-2 implemented through the SSD structure and xLSTM implemented through covariance updates.

Unverified70%
Oct 6

State Space Models (SSMs) pull the direction back toward fixed-size recurrent memory, but use a different state structure and update rules.

Verified75%

All Facts (8)

Source Articles