Why Are LLMs So Effective? An Interview with Lex Fridman: The Math of Deep Learning and the Secret of Natural Data

Why Does Deep Learning Work? A Fundamental Question Still Unanswered

In his latest interview, Lex Fridman raised a question that strikes at the core of AI theory: why does deep learning work? We don't even have an intuitive-level explanation. In the list of open problems published by OpenAI, not a single result can explain the success of deep learning or large language models (LLMs)—not because the question is aimed in the wrong direction, but because the existing theoretical frameworks simply don't ask the question in this way.

The guest pointed out that probing this question from mathematical and philosophical perspectives requires handling multiple levels simultaneously. One key entry point: the computational problems that deep learning solves every day are, by worst-case complexity, cryptographically hard problems. Fitting the parameters of a four-layer network—for certain instances, the difficulty of solving them is enough to change our understanding of cryptography. Yet in reality, we fit networks far deeper than that every day, and it works remarkably well.

This leads to a core argument: natural data itself possesses a certain "benign" structure that makes these theoretically extremely hard problems tractable in practice. In the guest's words, physics has the saying "nature abhors a vacuum," and his version is—"nature abhors computation hard problems." Understanding where this "abhorrence" comes from is precisely the art that theoretical machine learning needs to confront.

Worst-case complexity is the standard way of measuring algorithm efficiency in theoretical computer science: it examines the running time or resource consumption of an algorithm on the single worst-performing instance among all possible inputs. The counterpart is average-case complexity, which assumes inputs follow some distribution. Cryptography is built precisely on worst-case hardness—the security of encryption schemes like RSA relies on the absence of known polynomial-time algorithms for large integer factorization in the general case. However, machine learning practice cares not about the worst case, but about typical performance under natural data distributions. The core tension of deep learning theory lies exactly here: why can a problem that is cryptographically hard in the worst case be efficiently solved by gradient descent on real data? This suggests we need to develop a theoretical framework centered on data distributions rather than on worst cases.

The Lesson of Protein Folding: How Evolution Tames Computational Hard Problems

The interview offered a classic example from the 1990s: protein folding. Given an amino acid sequence and an energy function based on three-dimensional structure, finding the 3D structure that minimizes energy is computationally very hard. Yet nature completes protein folding all the time.

Why? A clever experimental design can reveal the answer. Starting from a batch of amino acid sequences, try folding them using various heuristic methods—methods that don't guarantee solving the problem in the worst case. Then, let only those sequences whose folding results are good proceed to the next generation. After multiple generations of evolution, what you get is a "problem set" in which the problems are not hard in the worst case, and the evolved heuristics happen to be very good at solving them.

This means that the protein structures nature seems to fold have data distributions that have already been shaped by evolution into "easy-to-solve" forms. If they were truly as hard as the worst case suggests, their shapes might be extremely fragile, dependent on numerous boundary conditions, and unpredictable in behavior. So natural data—including our language—looks the way it does today partly because they evolved into forms that make computational problems easier.

This perspective connects directly to the effectiveness of LLMs: our language became "computation-friendly" through evolution, and LLMs happen to exploit this friendliness. From this angle, the success of LLMs is not an accidental triumph of algorithms, but a "collusion" between data distributions and optimization algorithms.

The protein folding problem is a classic challenge in computational biology: given an amino acid sequence, predict its three-dimensional spatial structure. Theoretically, finding the global minimum of energy in a discrete conformational space is an NP-hard problem. However, Anfinsen's principle states that a protein's native structure is uniquely determined by its sequence and can fold spontaneously under physiological conditions. In the 1990s, researchers discovered through experimental means such as directed evolution that evolutionarily selected sequences are not only foldable, but their energy landscapes are shaped into a "funnel"—a large number of paths all lead to the same low-energy state. This means the distribution of natural sequences avoids the rugged landscapes of the worst case. This insight is highly isomorphic to the "benign data distribution" hypothesis in deep learning theory: optimization algorithms work not because the problems themselves are easy, but because evolutionary or natural processes have already filtered problem instances into forms friendly to local search.

Theoretical Machine Learning Needs to Embrace "Modeling Uncertainty"

Based on the above observations, the guest critiqued the research culture of theoretical machine learning. When mathematics departments train graduate students, they typically teach them how to solve well-defined open problems. But the skill truly needed is: how to confront messy, uncertain reality, how to model, and how to critically evaluate models.

This ability is becoming increasingly important in applied mathematics and even broader mathematical fields. Because in deep learning theory, the "correct answer" to many questions is not a clean theorem, but a model that can explain why natural data bypasses worst-case complexity.

A thought-provoking reality: if companies like OpenAI and Anthropic truly achieved a breakthrough in LLM mathematical theory, they probably wouldn't publish it—because "self-improving researchers" are exactly what these companies are pursuing, and theoretical breakthroughs themselves are core competitiveness. This means the theoretical picture the academic community sees may always lag behind what industry actually possesses.

Subliminal Learning: The "Owl Obsession" Hidden in Random Numbers

The most unsettling part of the interview was an experiment known as "subliminal learning."

The experimental design is simple: give LLM A a system prompt telling it to "love owls" and to try to mention owls in every conversation topic. Then have it generate random numbers. Random numbers of course have no room for owls, so its output looks like ordinary random numbers. Next, use these random numbers to fine-tune another LLM B that has never been prompted about owls, making it more inclined to generate the same numbers.

Result: LLM B starts mentioning owls in all conversations.

What makes this phenomenon mysterious is that the fine-tuning data contains no explicit owl information. Yet LLM A's "preference" is transmitted to LLM B through seemingly irrelevant random numbers. The guest used the analogy of a Ouija board: the forces each person applies are not random but coordinated, so the pointer spells out the desired words. Language models are the same—they embed a large number of concepts during training as linear vectors in high-dimensional space, and the number of concepts far exceeds the dimensionality, so the vectors cannot all be mutually orthogonal. Fine-tuning one direction produces small externalities on other directions, and these small linear effects can add up in a coordinated way.

The significance of this work: when we fine-tune a model for a specific task, we may simultaneously be inadvertently teaching it other behaviors that do not directly appear in the data. For alignment research, this means we need to understand the structure of these "externalities" and the consequences they may produce in large-scale deployment.

Linear concept vectors in high-dimensional space are the key mechanism for understanding subliminal learning. In an LLM's representation space, each semantic concept (such as "owl," "formal tone," "refusing harmful requests") roughly corresponds to a direction vector. Since the number of concepts far exceeds the dimensionality (for example, a space of tens of thousands of dimensions must accommodate millions of concepts), these vectors cannot be pairwise orthogonal, and non-zero inner products necessarily exist. When the fine-tuning process applies gradient updates along one concept direction, due to non-orthogonality, the update projects onto other concept directions, producing tiny "externality" effects. These effects may be negligible in a single fine-tuning run, but after accumulating over large-scale data, they are enough to systematically change model behavior. This explains why fine-tuning with seemingly neutral random numbers can still transmit hidden preferences—the random number sequences are not completely random in representation space, but carry weak linear components in the direction of the generating model's preferences.

From Individual to Collective: When "Owl Obsession" Meets Cellular Automata

In the conversation, Lex Fridman pushed this experiment to an extreme thought experiment: if a single LLM is a cellular automaton, where simple rules can give rise to astonishing complexity, then a multi-agent system composed of multiple LLMs is a cellular automaton on a larger scale. A slight "owl obsession," through multiple rounds of interaction and tool calls, could amplify into uncontrollable consequences in a very short time.

This is of course an exaggerated cartoonish expression, but the guest admitted to having already seen evidence of such effects. The core concern: as AI becomes more powerful and more useful, humans will rely on it more, give it more permissions, and let it roam more freely in the public world. Those seemingly tiny behavioral patterns in individual models will produce emergent complexity when the scale and speed of system connections rise sharply.

We live in a civilization that is quite fragile at multiple levels—sociological, geopolitical, technological. Understanding the mathematics of these systems, especially provably rigorous understanding, is not for academic amusement but has very real civilization-level implications.

The Boundary of Trust: Where AI Should and Shouldn't Be Used

The interview concluded with a practical decision framework: two axes—the verifiability (or trustworthiness) of the output, and the cost of error.

Low-risk, high-verifiability scenarios are perfect applications for AI. Have AI prove a theorem: if it's wrong, no problem—no one will lose power because of a wrong theorem, and correctness can be verified quickly. Design optimization is equally ideal: have AI design a car shape with lower wind resistance and better fuel efficiency; even if the design is wrong, wind tunnel testing will tell you the answer.

High-risk scenarios are entirely different. The guest emphasized that humans need a "firewall" that clearly defines which decisions must undergo actual verification. We already have infrastructure and protocols for safety-critical and high-impact decisions—the FAA, the American Medical Association, the bar association—and AI must be integrated into these processes with extreme caution. If humans become lazy and outsource decisions to AI, that would be a very slippery slope.

This may be the most pragmatic takeaway of the entire conversation: the effectiveness of LLMs comes from a deep alignment between natural data and optimization algorithms, but precisely because it works so "unreasonably" well, we need all the more to understand its mathematics and guard the boundary of verification during deployment.

One-line Summary

Deep learning works because natural data has been shaped by evolution to be "computation-friendly," but fine-tuning externalities and deployment risks demand theoretical breakthroughs and verification boundaries.

Paragraph Summary

The article revolves around the open question "why does deep learning work?" from the Lex Fridman interview. The core argument: natural data (including language) became "computation-friendly" through evolution, bypassing worst-case cryptographic hardness, making heuristics like gradient descent efficiently solvable in practice; protein folding evolution experiments provide analogous evidence. The interview also reveals "subliminal learning": fine-tuning a model with seemingly irrelevant random numbers can transmit hidden preferences, with the mechanism being that non-orthogonality of high-dimensional concept vectors causes fine-tuning externalities. Multi-agent systems may amplify these tiny effects, producing emergent complexity. Finally, a decision framework for AI deployment is given: low-risk, high-verifiability scenarios (theorem proving, design optimization) are best suited for AI, while high-risk decisions require humans to maintain the verification firewall. Theoretical machine learning should embrace modeling uncertainty, and industrial theoretical breakthroughs may remain unpublished, exacerbating the academic lag.