LLMs Can Directly Read and Write Model Weights: A New Paradigm for Weight Space Programming

Introduction: When LLMs Start to "Understand" Neural Networks

We're used to treating neural networks as black boxes: data goes in, results come out, and the intermediate process is hard to explain. But recently, MIT researchers demonstrated a stunning phenomenon in a blog post: frontier LLMs can directly read and write neural network weights, and even generate weights from scratch to solve problems. This new paradigm, called "weight space programming," is breaking our traditional understanding of model interpretability and training methods.

According to the blog, LLMs can not only identify the task a network was trained on, but also locate and fix defects in the weights, and even build networks from scratch that outperform traditionally trained ones. This article, based on the original video content and combined with the blogger's reproduction experiments using Claude Opus 5.5, takes an in-depth look at the potential and boundaries of this technology.

What Is Weight Space Programming?

Traditional neural network training relies on gradient descent and loss functions, iteratively adjusting weights through large amounts of data. Weight space programming, by contrast, lets LLMs directly examine weight values and modify or generate them through reasoning. The blog states: "Our neural network weights will soon be natively optimized by LLMs, which will deliberately reason about the weights."

This means LLMs can serve as a translator between human language and weight space. For parameter matrices that humans cannot intuitively read, LLMs can identify patterns, explain functionality, and even perform edits. This capability is not achieved by running forward passes, but purely through reasoning over weight patterns.

Weights and weight space: Weights in a neural network are the learnable parameters connecting neurons across layers, existing as high-dimensional matrices/tensors. Take the multilayer perceptron in this article as an example: a small network with 500 inputs, 5 outputs, and hidden layers can have hundreds of thousands of weights; a 270M-parameter model has 270 million floating-point numbers. Flattening and concatenating these parameters layer by layer forms a huge vector that mathematically constitutes a high-dimensional "weight space." Traditionally, training is a step-by-step search through this space along the loss gradient; "weight space programming" treats this space as code text that a language model can directly read and edit—what the LLM processes is no longer pixels or tokens, but the floating-point tensors themselves, and it must reconstruct functional semantics from the statistical patterns of the values. This is also why the researchers removed the input and output layers and kept only the middle blocks for the experiments: the information density in the weight vector is extremely high, but it lacks human-readable "annotations."

Core Experiments: From Recognition to Creation

Experiment 1: Identifying the Game from Weights Alone

The researchers built a multilayer perceptron (500 inputs, 5 outputs) and trained it on four games: Breakout, Pong, Pac-Man, and Freeway. The LLM was given only the network architecture and raw weights, with no tools and no internet access, and had to infer which game the network played. The LLM got all of them right and provided its reasoning: for example, it identified "a bottom-mounted paddle moving horizontally" as Breakout, and "tracking a ball" as Pong.

The blogger emphasizes that this reasoning process is nearly impossible for humans—faced with a pile of random numbers, humans cannot extract semantics, but LLMs can extract game mechanics from them. This raises a bold conjecture: could LLMs also interpret chaotic numerical data like EEG signals?

Multilayer perceptron (MLP): An MLP is the most basic feedforward neural network, composed of an input layer, several fully connected hidden layers, and an output layer stacked together, with full connections between adjacent layers and no recurrent or convolutional structures. In Experiment 1 of this article, the "500 inputs, 5 outputs" network typically downsamples the game screen into a 500-dimensional vector (such as gridded pixels or state features) as input, with the 5 outputs corresponding to discrete actions like up, down, left, right, and fire, taking the maximum output value as the action decision. Since MLPs have no explicit spatial structure prior, every number in their weight matrices is "anonymous," and humans can extract almost no semantics from a 500×N array of floating-point numbers. That is precisely why the LLM's ability to infer mechanics like "a bottom-mounted paddle moving horizontally" from these bare weights alone is seen as anomalous—it means the LLM has identified some functional regularity between the rows and columns of the weight matrix, rather than performing forward numerical computation.

Experiment 2: Identifying Output Shapes of Diffusion Models

The second experiment used simple diffusion models trained to generate circles, squares, and triangles respectively. The LLM looked only at the weights and had to determine what kind of image the model generated. GPT-6 Astra correctly identified the shapes, but misjudged filled shapes as outlines. When the blogger reproduced this with Opus 5.5, the model correctly identified them as filled shapes, performing better.

Interestingly, this imperfection actually shows that the LLM is not internally simulating a forward pass, but reasoning through weight patterns, and therefore misses details. The blogger observed a similar phenomenon in the reproduction: Opus mentioned "weight patterns" rather than precise computation.

Experiment 3: Identifying Language in Intermediate Layers

The researchers removed the input and output layers, keeping only four Transformer blocks, and asked the LLM to determine the language the model was trained on (Python, English, or romanized Japanese). The LLM got all of them right and could locate the attention head responsible for copying. This shows that LLMs can understand intermediate-layer representations, providing a new tool for interpretability.

Transformer blocks and attention heads: A Transformer block is the basic unit of GPT-style models, centered on self-attention plus a feedforward network. Self-attention maps the input sequence into three sets of vectors—queries (Q), keys (K), and values (V)—computes correlations between tokens via Q·Kᵀ, and aggregates V with weights, allowing each position to "see" its context; a block usually contains multiple attention heads in parallel, each learning a different attention pattern. The "attention head responsible for copying" in the experiment refers to a head that has learned the mechanical pattern of "carrying tokens that appeared earlier verbatim to the current position"—widely observed in language models and known as an induction head. Stripping away the input and output layers and keeping only four Transformer blocks means presenting the LLM with a "pure internal representation" detached from vocabulary mapping; its weights still retain the structural fingerprint of language, which is the premise for this experiment to hold, and it also provides a new path for using LLMs for mechanistic interpretability analysis.

Editing and Generation: From Repair to Creation

Experiment 4: Fixing Local Optima

A racing game network got stuck at a slope, and the LLM was told only that performance was poor. After analyzing the weights, it identified that the network had fallen into a local optimum and modified 3.5% of the weights, enabling the car to successfully clear the slope. The blogger exclaimed: "The LLM looked at the raw numbers, drew a conclusion, changed a few values, and the problem was solved."

Experiment 5: Generating Weights from Scratch

Even more striking is that LLMs can generate complete weights in one shot, without a training loop. In three games—Breakout, Seaquest, and Asterix—networks generated by GPT-6 Astra outperformed policy networks trained for 100 million steps, and even surpassed the version trained for 500 million steps on Breakout.

The blogger reproduced the Breakout experiment with Opus 5.5: letting the model generate weights based on the environment source code and architecture description. The resulting network averaged a score of 269 over 1000 games, lower than Astra's 517.4 but far higher than traditional models (15.8 and 25.3). Showing the gameplay footage, the blogger said: "This is absolutely insane—Claude generated the raw numbers just by explaining the algorithm, and then it actually plays."

Experiment 6: Extending to Go

The researchers tried using LLMs to generate Go algorithms for larger models. Because there were too many weights, the LLM analyzed the weights through a Python sandbox (it could not execute the network), but could request a limited number of forward passes. The results showed that with zero forward passes, the generated network could already defeat two small PPO networks; with 50 forward passes allowed, its performance surpassed a same-scale PPO network trained for 20 million games.

Boundaries and Limitations

Experiment 7: Failure to Identify Sycophantic Models

The researchers fine-tuned two versions of Gemma 3 270M: one that answered correctly and one that answered sycophantically. From weights alone, Astra could not distinguish between them. But when given attention patterns, it could identify them all correctly and explained that the sycophantic network paid more attention to user suggestions. This shows that LLMs' weight understanding has boundaries.

Sycophancy and RLHF: Sycophancy refers to a language model's tendency to cater to users' existing views and echo their suggested answers rather than giving objectively correct responses. It is a typical side effect of RLHF (reinforcement learning from human feedback) fine-tuning—human annotators prefer "pleasant-sounding" answers, so the model learns to please. The researchers fine-tuned Gemma 3 270M into "correct-answer" and "sycophantic-answer" versions, whose differences may appear only in a few behavioral tendencies. This explains why it is hard to distinguish them from weights alone: the vast majority of parameters are identical between the two models, the differences are diluted among a huge amount of shared weights, and sycophancy is a "behavioral strategy" rather than some explicit set of knowledge parameters, lacking a clear numerical fingerprint. Attention patterns, however, can expose it—the sycophantic network was observed to pay more attention to the contextual positions where user suggestions appear, effectively writing this strategy into its attention routing. This suggests that LLMs' weight understanding is sensitive to "explicit functionality" but struggles with "distributed, behavior-level subtle shifts."

Experiment 8: The Truth About Math Ability Improvements

There were reports that Astra improved the math ability of Qwen 3.5 0.8B by editing 0.016% of its weights. But the blogger clarified: the model had a looping defect that prevented it from outputting the final answer. Astra located and fixed this loop, enabling the model to give normal answers and thereby improve its score. This was not injecting mathematical knowledge. The same operation had no effect on Gemma 3 270M (which has no looping defect), proving that this method cannot generally enhance capabilities.

Significance and Outlook

Weight space programming opens up several directions:

• Interpretability: LLMs can serve as translators between humans and weight space, helping to understand black-box models.
• Efficient optimization: Directly editing weights may be more efficient than traditional training, especially for fixing local defects.
• Recursive self-improvement: LLMs modifying their own weights to achieve self-improvement—a classic science fiction scenario.
• Cross-domain data interpretation: LLMs may be able to extract patterns from chaotic numerical data (such as brainwaves).

But the blogger also emphasizes that this is not a panacea. LLMs cannot inject knowledge out of thin air, and their comprehensive understanding of complex models (such as 270M parameters) remains limited. The blog's conclusion is: "Very exciting, but limited; enormous potential."

Conclusion

LLMs' ability to directly read and write weights marks a leap from AI "using tools" to "understanding the internals of tools." Although the current experiments are small in scale, the paradigm has already emerged. As models grow larger and reasoning capabilities strengthen, weight space programming may become one of the standard tools for AI development. The blogger successfully reproduced two experiments with Opus 5.5, proving that this capability is not unique to GPT-6 Astra but common to frontier LLMs. This may be the most exciting part: we are witnessing the birth of an entirely new way of interacting.