Knowledge Injection Writability Study: Controlled GPT-2 Experiments vs. Production Memory

Title

Knowledge Injection Writability Study: Controlled GPT-2 Experiments vs. Production Memory

Meta Description

A controlled study on knowledge injection into large models, using a GPT-2 2×2 experiment to compare against Qwen and DeepSeek^Ad production memory systems, finding that successful writing does not equal question-time recall, with high surface hit rates but zero strict answer scores, and revealing anchoring's suppressive effect on drift.

Article Content

Whether knowledge can be "written into" a large model's memory and stably retrieved is another important technical route beyond retrieval augmentation. A study titled "Writability and the Pathway: A Controlled Study of Knowledge Injection into Addressed Memory," through controlled GPT-2 experiments and comparative tests against Qwen and DeepSeek production-grade memory systems, delivers a rather restrained conclusion: writing can succeed, but "successful writing" is still a considerable distance from "being correctly recalled by a question."

Controlled Experiment Design: A 2×2 Matrix Decomposing the Writing Pathway

The study mounted newly initialized modules onto a frozen GPT-2, constructing a 2×2 controlled design. The first dimension is update scope: row-only updates versus simultaneous row-plus-projection updates. The second dimension is whether fixed module-residual anchoring is enabled.

All experimental groups reproduced 50 trained-complete results across three sampling seeds, indicating that basic memory writing itself is achievable. But when the question shifts to a natural language query, the gap in target string hit rates immediately emerges: the two row-only experimental groups both averaged a 5.3% hit rate, while projection updates without anchoring reached 28.7%, and with anchoring added, 22.0%.

It must be particularly noted that despite projection updates showing clearly higher surface hits, all experimental groups still scored zero on strict answers. This means the model may have "uttered" the target word without forming a determinable complete answer.

On frozen backbones and newly added modules: The "frozen GPT-2" here refers to not updating the original billions of parameters, but instead attaching a trainable small module beside a certain layer (usually the residual stream), forming a "backbone fixed, bypass adjustable" writing approach. The purpose is to decouple "what the model itself already knows" from "newly written content," avoiding global parameter drift during fine-tuning that would contaminate the control results.

The so-called "row vector" typically refers to a bias vector in the module that is directly added to the hidden state, while "projection" refers to a small linear transformation matrix (of the form W·x), which has stronger expressive power and can perform directed rotation or scaling on activations. "Module-residual anchoring" adds an additional fixed residual term to this bypass module, making its output closer to zero when inactive, thereby limiting its perturbation of the backbone behavior after training. Understanding these three points is necessary to read the different "writing degrees of freedom" represented by each cell in the 2×2 matrix.

Anchoring Reduces Drift, but Surface Hits Do Not Equal Entity Retrieval

The role of module-residual anchoring is relatively clear in this dataset: it pulls the hit rate under natural questioning down from 28.7% to 22.0%, while also reducing the measured behavioral drift. This is a typical trade-off—exchanging a certain recall rate for better preservation of non-target behaviors.

The researchers' judgment is quite cautious: surface hits do not prove that the model has achieved entity-level retrieval. In other words, the model may have merely written certain word sequences into the output distribution, rather than truly storing the "entity-answer" correspondence into memory. This is also why strict answer scores all drop to zero.

This conclusion has direct implications for practical applications: if knowledge injection effectiveness is evaluated only by keyword hit rate, the system's true capability is likely to be overestimated.

Behavioral drift refers to changes in the model's output distribution on tasks it originally already mastered or on unrelated prompts after injecting new knowledge. It is typically measured using a set of held-out general prompts, comparing outputs against the pre-injection baseline. Large drift means the model "messed up old things in order to remember new things," which in actual deployment may manifest as degraded general conversation quality, formatting errors, or unstable persona settings.

Surface hit only checks whether the target string appears in the generated result, without verifying whether the model has truly established a "question entity-target answer" mapping. For example, asking "What is the capital of France," if the model answers "The capital of France is Paris, but Paris is actually an Italian city," a surface hit would judge it as positive, while an entity-level retrieval judgment would deny it due to the contradictory answer. This is precisely the key point emphasized in the original text that "strict answer scores are all zero": a high surface hit rate may only reflect that the model has learned to emit certain tokens in relevant contexts, rather than truly remembering factual associations.

Production Memory Systems: Qwen and DeepSeek Writing Performance

In the archived Qwen experiments, the study conducted gradient writing tests on Qwen3.8-Flash-Next-FP8. Results showed that the designated first answer token was ranked highest among 132 independently reset targets, of which 93 were newly promoted (originally not in first place) and 39 were already in first place.

A matched 24-target comparison provided finer differences: under the same number of iterations but non-equivalent computation procedures, the Adam optimizer produced 13 new promotions, while SPSA produced only 2. The study explicitly noted the premise of "non-equivalent computation procedures," so this comparison can only serve as a reference and cannot directly infer optimizer superiority.

Additionally, an 80-line case preserved continuation content and replay records, while its rewritten form limited the applicable scope of that case.

Gradient writing and zero-order optimization: The "gradient writing" mentioned in the Qwen experiment refers to directly computing the loss gradient of the target token through backpropagation and updating trainable parameters. This requires the model or writing module to be differentiable with respect to the target token's logits, so it is typically used in white-box systems or systems with accessible gradients.

SPSA (Simultaneous Perturbation Stochastic Approximation) is a zero-order optimization method that does not rely on backpropagation, but instead estimates the gradient direction by randomly perturbing parameters and observing loss changes. It is more friendly to scenarios where the computation graph is non-differentiable or black-box APIs are used, but each update requires multiple forward passes, and convergence is typically slower with higher variance. The original text specifically noted "non-equivalent computation procedures" precisely because Adam follows an exact gradient path while SPSA follows an approximate gradient path; the same number of iterations does not mean the same computational budget or information content, so that 13-versus-2 comparison can only serve as a directional observation, not as decisive evidence of optimizer superiority.

DeepSeek Zero-Order Experiments and Conclusion Boundaries

The results of six zero-order cases on DeepSeek-V4.1-Flash were more conservative: only one new target token promotion appeared, along with one regression.

Combining all experiments, the authors' core judgments can be summarized in three points: First, bounded writing is feasible; second, it is necessary to evaluate separately "trained expression," "behavior under questioning form," and "preservation of non-target behaviors"; third, top first-token ranking does not prove complete answer success, nor can it derive universal conditions for writable memory.

For teams exploring knowledge injection, memory editing, or model post-training, the greatest value of this study is perhaps not any specific number, but that it establishes a more discriminative evaluation framework—treating "written in" and "asked out" as two things that need separate verification.

One-line Summary

Controlled experiments reveal: knowledge can be written into large model memory, but "written in" and "asked out" are two different things.

Paragraph Summary

A study on knowledge injection, through a frozen GPT-2 2×2 controlled experiment and comparative tests of Qwen3.8-Flash-Next-FP8 and DeepSeek-V4.1-Flash, systematically distinguishes "successful writing" from "being correctly recalled by a question." The GPT-2 experiment shows that with row-only updates, natural question hit rate is only 5.3%, rising to 28.7% with projection updates added, but all experimental groups score zero on strict answers; module-residual anchoring reduces hit rate to 22.0% while lowering behavioral drift. Qwen gradient writing elevates the first answer token to the highest position for 132 targets, of which 93 are new promotions, but the Adam vs. SPSA comparison is for reference only due to non-equivalent computation procedures. DeepSeek zero-order cases show only one new promotion and one regression. The authors conclude: bounded writing is feasible, but "trained expression," "behavior under questioning form," and "preservation of non-target behaviors" must be evaluated separately.