Causal Neural Memory
A class of external memory modules updated incrementally in a causal (autoregressive) manner. It maintains a set of readable and writable memory slots and performs write operations based on the current input at each time step, so that memory capacity grows with the number of parameters rather than sequence length. Compute and memory overhead grow linearly with sequence length, and the memory parameters can be fine-tuned via gradient updates at test time.
Timeline (last 90 days)
METTA decomposes the fast adaptation phase into two time scales: causal neural memory handles fine-grained within-episode adaptation and runs with linear sequence complexity, while compact episode-level summaries support cross-episode long-horizon task inference.
Causal neural memory is a class of external memory modules updated incrementally in a causal (autoregressive) manner, whose typical implementations include linear attention, state space models, or fast weight mechanisms.
All Facts (2)
METTA decomposes the fast adaptation phase into two time scales: causal neural memory handles fine-grained within-episode adaptation and runs with linear sequence complexity, while compact episode-level summaries support cross-episode long-horizon task inference.
50%UnverifiedCausal neural memory is a class of external memory modules updated incrementally in a causal (autoregressive) manner, whose typical implementations include linear attention, state space models, or fast weight mechanisms.
50%