The 0.381MB Ghost: AI Memory That Persists After the Session Ends

At three in the morning, a process on a test machine was still awake. No session was connected to it, input had long since stopped, but it was alive—an O-series inference process with memory holding steady at 0.381 MB. It was a 125M model. In measurements at 128k context, that same residual state was 1008 times smaller than conventional approaches.

This detail comes from an experiment log published by the awareliquid team. They habitually call this phenomenon a "ghost": the session leaves, the state stays. But the team repeatedly emphasizes that the ghost's technical substance amounts to one sentence—a constant-memory, snapshottable, portable episodic memory, plus one honest recall run. It doesn't keep thinking after you leave.

What the Ghost Is: A Fast-Weight Matrix, Not Compressed Context

To understand what 0.381 MB means, you first need to look at the mechanism. The O series uses the ARR architecture, which removes attention and uses liquid ODEs to compress history into a recurrent state. Each step only updates this state without scanning the full history, so memory is constant—O(1). The hybrid-architecture M series, by contrast, retains attention, and its inference memory is not O(1).

What actually carries the "memory" is a fast-weight matrix. It is dynamically written by input during inference, storing key-value associations that have appeared across windows. According to cross-verified experiment logs: on cross-window, cross-session associative recall tasks, attention mechanisms and LoRA score 0.000 by construction, while the fast-weight matrix produced 0.62 / 0.43 / 0.62 across three seeds, averaging 0.56. Remove the matrix and recall collapses to 0.008. The memory is that matrix itself, with no other component as a fallback.

We must draw a clear boundary here. The team repeatedly states: the fast-weight matrix stores episodic key-value memory, not compressed distributed context. It brings no understanding, reasoning, or emotion, and offers no gain for long-context language modeling. What it remembers is what you actually said, plus a lightweight semantic label, retrieved RAG-style.

Fast weights are a class of weights relative to the "slow weights" fixed after model training: they are updated online during inference based on the input stream, without backpropagation or gradient descent, and thus can write recent inputs into model state at extremely low cost. The concept traces back to the fast-weight associative memory proposed by Hinton and von der Malsburg in the 1990s, and to Schmidhuber's work on linear Transformers. Unlike standard attention, which recomputes key-value similarity across all historical tokens at every step, the fast-weight matrix accumulates key-value associations that have appeared across windows, giving the model associative recall capability under constant memory. This update rule is typically based on Hebbian learning or outer-product accumulation, so it stores explicit episodic associations rather than compressed distributed representations. Understanding this is what makes clear why removing the matrix collapses recall performance to 0.008: the memory has no redundant backup.

The ARR (Adaptive Resonance Recurrent) architecture is the model backbone used by the O series in this article. Its core is removing standard self-attention and instead using a liquid ODE to recurrently compress historical information. Each step updates only a fixed-dimensional recurrent state without revisiting all historical tokens, so inference memory is constant at O(1), independent of context length. This design draws on ideas from neural ODEs and liquid time-constant networks, letting state evolution be driven by the input stream rather than relying on explicit attention weight matrices. The cost is that the model no longer has direct access to arbitrary historical positions; all information must pass through the recurrent state, which places higher demands on continual learning and long-range dependency modeling. The M series, by contrast, retains attention in a hybrid architecture, so its memory is not constant.

Cross-Process Recovery: From "Cache" to "Inheritable State"

If memory could only live inside a warm process, it would just be a cache. The snapshot/restore experiments published by awareliquid made this path work.

The process is straightforward: Process A runs to the end of a session segment and serializes the fast-weight matrix along with its companion state to disk; Process A exits, releasing all memory. Process B starts from scratch, loading no session history and replaying no dialogue, and directly restores state from the file. Then the same query is asked, and the recall result is bit-exact identical to that in Process A (bit-exact lossless).

This property is the precondition for the "ghost" metaphor to hold. Only when state can leave the process, be transported, and be retrieved exactly as it was can it be something inheritable. Mechanically there's nothing mysterious: fast weights are matrices updated online at runtime, driven by the input stream, relying on neither backpropagation nor an external vector database. Serialization covers all elements and state bits, and restoration is a deterministic deserialization with no sampling, quantization, or approximation in between.

The cost must be stated too. Lossless recovery verifies that "episodic key-value memory can be fully transported," not that "the model understood what you said." The team has written this sentence in stone on multiple occasions, because it is the one most easily stretched by marketing language.

The 0.381MB Line, and the Reality of 7.6x Degradation

The 0.381 MB figure has clear conditions of applicability.

It is the measured resident state of the O series (ARR, no attention) at 125M scale, independent of context length. At 128k context, memory usage is up to 1/1008 that of conventional approaches. All of this holds only for the attention-free O series. The M series is a hybrid architecture that still carries attention, and its inference memory is not O(1). Applying the O series' resident memory as a label for the M series will miscalculate edge-device budgets.

The more realistic boundary is in continual learning. The team's currently published data: roughly 7.6x PPL degradation, with a target of under 5x; Hebbian plasticity is inert at the scales tested. The O series itself is still a research preview, at 2.15x the teacher model's PPL. These numbers are all written in RESULTS.md and were not moved aside just because they look bad.

Hebbian plasticity originates from Hebbian theory in neuroscience ("neurons that fire together, wire together"), and in artificial neural networks usually refers to an unsupervised local weight update rule: strengthening the corresponding connection when input and output are activated simultaneously. Unlike backpropagation, Hebbian updates rely only on local information with extremely low computational cost, and are theoretically suited to online continual learning. However, in actual large-scale training, pure Hebbian rules easily lead to weight explosion or saturation and require additional normalization and decay mechanisms. The statement in this article that "Hebbian plasticity is inert at the scales tested" means the mechanism produced almost no measurable learning effect at current scale, which explains why continual learning still has roughly 7.6x PPL degradation and is one of the key bottlenecks for this architecture's path to practical use.

Retracted Numbers and the Judgment Engineers Should Take Away

awareliquid has published a set of retraction records—something uncommon among comparable projects.

The first retraction was the "-28.5% adapter PPL." Internal runs showed a perplexity decrease, initially attributed to the liquid adapter, but after layer-by-layer controlled comparison, the gain came from an ordinary LoRA attached alongside it, unrelated to the adapter itself. The number was right; the attribution was wrong. The team posted a correction on the research page and made "remove the claimed module and run it again" a hard rule for external materials. In the same batch, claims about "consciousness" and Phi values were downgraded and uniformly reclassified as "sources of inspiration," no longer discussed as metrics.

These retractions point to the same engineering habit. When evaluating a demo that says "it still remembers," demand three things at once: what the storage form is, how much remains after deleting that state, and how cross-process recovery is verified. If you can't get these three, treat it as narrative first, not architecture.

Back to that process at three in the morning. It sits quietly with 0.381 MB of state, neither growing nor shrinking. The next time a process starts, it reads it back and picks up where things left off. But it won't ponder why you renamed a field after you leave, nor will it spontaneously re-think yesterday's events. The ghost metaphor can stay, with this boundary: it remembers what you said, and that's all.

The repository and results are at github.com/awareliquid/awareliquid, with Recall@5 of 96.0% on LongMemEval and zero LLM calls for retrieval—reproducible on your own.