Transformer, RNN, SSM: Where Does an AI Model's Memory Actually Live?

In discussions about AI architectures, Transformer, RNN, and SSM (State Space Models) are often compared as if they're competing in a performance race. But shift the lens — instead of asking which architecture is more powerful, ask "where does memory actually live in the model" — and many seemingly complex design differences suddenly become clear. This is the central question behind a recent in-depth discussion in the Reddit machine learning community.

Reexamining Architectures Through the Lens of "Working Memory"

When we observe these architectures through the lens of "working memory," the question shifts from "who is faster and stronger" to "what kind of carrier is historical information compressed into." Memory might take the form of a compact recurrent state, a continuously growing KV cache, or something closer to the structure of the network itself.

This question matters because it touches on a fundamental limitation of current large models: during inference, model weights are frozen. No matter how large the context window, the model is managing context — not genuinely converting present experience into persistent model knowledge. Where memory lives determines how long a model can remember, how much it can retain, and how it forgets.

RNN: Elegant but Constrained Recurrent State

RNNs store memory in a recurrent hidden state that propagates forward through time steps. The design is elegant — information is passed along like a baton, step by step.

But there's a clear bottleneck. The original post highlights a key ratio: a model can have roughly O(N²) parameters, yet carries only about O(N) cross-time state. In other words, there's a serious mismatch between the model's "capacity" and the "memory" it can retain across time.

This raises an interesting reflection: were RNNs phased out because the idea of recurrence itself was flawed? Or was the real problem the imbalance between memory and computation? If it's the latter, then recurrence as a mechanism may not have received a death sentence after all.

The "vanishing gradient" problem is also a key reason RNNs are limited. When error signals need to back-propagate through time steps, gradients decay (or explode) exponentially, making it difficult for the model to learn long-range dependencies. LSTM and GRU alleviated this through gating mechanisms — forget gates, input gates, and output gates that learn "when to clear, when to write" — essentially letting the model decide the rhythm of state updates. But gating doesn't fundamentally resolve the O(N²) parameters vs. O(N) state imbalance: the hidden state dimension is ultimately fixed, and large amounts of information are inevitably lost through repeated compression. This is why recurrent architectures have consistently underperformed Transformers on very long sequences. The bottleneck may not be the concept of recurrence itself, but the hard ceiling on state capacity.

Transformer: Pinning the Past as "Sticky Notes"

Transformers make almost the opposite trade-off. During cached inference, rather than compressing the past into a single hidden state, they store past representations as key-value (KV) entries, retrieved later through the attention mechanism.

The original post offers a fitting analogy: these KV entries are like sticky notes — each token leaves behind a "key" for retrieval and a "value" recording its content. This mechanism is extremely powerful, but it carries an intriguing characteristic: since weights are frozen during inference, the model is effectively managing context rather than distilling experience into persistent model knowledge. A disconnect emerges — fixed weights on one side, rapidly changing KV cache memory on the other.

This is why the KV cache grows continuously with sequence length. It trades that growth for powerful long-range retrieval, at the cost of memory that can never be "digested" into the model itself.

SSM and Mamba: Selectively Remembering and Forgetting

State Space Models (SSMs) redirect back toward fixed-size recurrent memory, but with a fundamentally different state structure and update rules. Selective SSMs, typified by Mamba, make "what to retain" input-dependent — which information is remembered and which is forgotten depends on the current incoming token.

Their states don't need to be as small as those in classical RNNs, but they're still fundamentally compressing history into a bounded memory. This raises a deeper question: must state exist in a compressed "working dimension," or can it live somewhere closer to the model's internal neuron or connection structure?

The mathematical roots of SSMs trace back to linear time-invariant systems in control theory: a set of matrices (A, B, C) describe how inputs update a hidden state and how that state maps to outputs. Early SSMs like S4 imposed special structure on matrix A (HiPPO initialization) to efficiently capture long-range dependencies while maintaining linear-recurrent inference efficiency. Mamba builds on this by introducing a "selectivity" mechanism: matrices B and C, along with the discretization step Δ, become input-dependent functions rather than fixed parameters. This allows the model to dynamically decide "how long this piece of information is worth remembering" based on the current token's content, rather than decaying all information uniformly. This design significantly narrows the gap with Transformers on language modeling tasks, while keeping inference memory usage constant — it doesn't grow linearly with sequence length, which has real practical implications for deploying long-context applications.

BDH: Letting Memory Live in the "Synapses"

The post specifically highlights an example that sparked the author's thinking — BDH (Dragon Hatchling). It combines linear attention in a high-dimensional neuron space with a low-rank GPU implementation.

Its recurrent attention state is an N × D matrix (where N is much larger than D), rather than an actually unrolled N × N connection matrix. Under a graph interpretation, related neural activity produces Hebbian-like updates to connections, giving working memory a synaptic-level explanation.

In this structure, working memory and learned connection patterns are formally much closer together. The author is nonetheless clear-eyed: this doesn't mean experience is truly solidified into trained weights, and fixed-size states still carry finite information capacity.

Hebbian learning comes from a hypothesis proposed by neuroscientist Donald Hebb in 1949: "neurons that fire together, wire together." Its core idea is that synaptic strength changes are positively correlated with the co-activation of neurons on both sides, considered the biological basis of associative memory and long-term potentiation (LTP) in neural networks. In BDH's architecture, the linear attention state update is formally very similar to Hebb's rule: the outer product v⊗k accumulated over the state matrix is equivalent to strengthening connection weights when "value neurons" and "key neurons" are simultaneously activated. This analogy gives working memory an intuitive neuroscientific interpretation, but there's also a fundamental distinction: BDH's state does not persist into model parameters after sequence inference ends, whereas long-term changes in biological synapses constitute genuine learning and memory storage.

The Real Question May Not Be "Who Replaces Who"

The discussion lands in a measured place: the author doesn't claim this will "kill" Transformers, nor that it solves the challenge of continual learning. What interests them more is a methodological shift — is "where does memory live" a better question than the traditional "architecture horse race"?

This leaves an open debate: do SSMs and these more network-centric architectures actually give us a better way to handle memory, or are we still hitting the same wall — forced to compress history into a finite state?

Regardless of the answer, this perspective offers a valuable analytical thread. Rather than fixating on which architecture wins a benchmark, it's more illuminating to see clearly what each design pays for memory: an imbalance between capacity and computation, an ever-expanding cache, or the information ceiling of a bounded state. Understanding where memory lives may be the true key to grasping the design of the next generation of architectures.