Can Next-Token Prediction Plus Reinforcement Learning Lead to General Intelligence?

"We are on the path to AGI; the only question is what the post-AGI world looks like." — A great deal of AI discussion on Reddit assumes this premise by default. [D] One user on r/ArtificialIntelligence posted, pulling the discussion back to an earlier step: do next-token prediction (NTP) plus reinforcement learning (RL) have any mechanistic reason to produce open-ended general intelligence?

The poster explicitly rules out two types of arguments: trend extrapolation and the belief that "there will be new breakthroughs in the future." What they want is a hard-nosed argument grounded in causal mechanisms, computational learning theory, information theory, and first principles. Below is a breakdown of this question, along with the strongest answer currently available—and its boundaries.

Why This Question Is Worth Taking Seriously

Most AGI discussions skip the mechanistic argument and go straight to consequence projection: alignment problems, economic disruption, concentration of power. This mode of discussion carries an implicit, unverified assumption—that the current training paradigm is sufficient in principle to lead to general intelligence.

If that does not hold, then all post-AGI projections are built on sand. The poster's question is really: what properties of current systems do we understand that give us reason to expect general intelligence to emerge from them? This is not pessimism about technological progress, but a demand to replace "faith" with "mechanism."

The Mechanistic Argument for NTP: Compression Is Understanding

The strongest argument in support of NTP's ability to produce general capabilities comes from the equivalence of compression and prediction. Solomonoff inductive inference and minimum description length theory point out that the optimal predictor is the optimal compressor—to accurately predict the next token, a model must internalize the regular structure underlying the data generation process.

Ilya Sutskever has expressed this intuition many times: predicting the next token is compressing the world, and compression requires understanding. If the training data covers a sufficiently broad range of natural and human knowledge, the process of minimizing prediction loss will force the model to learn cross-domain abstract representations.

The strength of this argument depends on a key premise: the breadth of the training distribution. Natural language text does encode a great deal of human cognitive achievement, but text is only a projection of the world, not the world itself. That NTP can compress the regularities in text does not mean it can compress all the regularities of the physical world, social dynamics, and embodied interaction.

Another mechanistic argument is the emergence of in-context learning. A sufficiently large Transformer acquires the ability during pretraining to infer new tasks from a small number of examples, which some researchers regard as evidence of meta-learning—what the model learns is not a specific task, but an algorithm for "how to learn tasks."

Solomonoff inductive inference was proposed by the mathematician Ray Solomonoff in the 1960s and is a formal framework for inductive inference in algorithmic information theory. Its core idea is that, given an observed sequence, the optimal prediction should weight all computer programs capable of generating that sequence by "program length"—the shorter the program (that is, the simpler the hypothesis), the higher the weight. This framework mathematically proves that the optimal predictor is equivalent to finding the shortest program that generates the data, that is, the optimal compressor. The minimum description length (MDL) principle was further developed by Rissanen, turning this idea into an operational model-selection criterion: taking the optimal trade-off between fitting the data and model complexity. Together they form the mathematical foundation of "compression is understanding" and are the core theoretical tools for understanding why NTP can learn the regularities of the world.

The Role of RL: From Imitation to Goal Orientation

If NTP provides a compressed representation of the world, RL provides the ability to perform goal-directed search in representation space. RLHF and subsequent reasoning training (such as chain-of-thought reinforcement) allow models not only to imitate training data, but to explore the strategy space under a reward signal.

The mechanistic argument for this path is: RL is essentially a general search process. As long as the state space and action space are expressive enough for the target task, RL can in principle find a solution. DeepMind's AlphaZero proved that in domains with clear rules and definable rewards, RL can learn strategies from scratch that surpass humans.

But here there is a key gap that the poster identified: open-ended. AlphaZero's game space is closed, and its reward function is fixed. The hallmark of truly general intelligence is the ability to autonomously generate new goals and define new problems, rather than optimizing under a given reward. The mechanism of RL itself does not provide a source for goal generation.

Open-ended is a core concept in evolutionary biology and artificial intelligence, referring to a system's ability to continuously produce novel and increasingly complex behaviors or goals rather than converging to a fixed solution. In the AI context, open-ended learning requires an agent not only to solve given problems, but also to autonomously generate new problems, new goals, and new skills. The work of Kenneth Stanley and others on NEAT and novelty search shows that RL with a fixed reward function easily falls into local optima or goal saturation, whereas a truly open-ended system requires intrinsic motivation mechanisms (such as curiosity and empowerment) to drive exploration. This concept is directly related to the "missing goal generation" discussed in the article: standard RL's rewards are externally given, whereas open-ended intelligence requires goals to emerge from within the system, which is precisely the mechanistic gap that the current paradigm has not yet solved.

The Response from Computational Learning Theory

From the perspective of computational learning theory, the combination of NTP+RL can be understood as learning at two levels: the pretraining stage learns the generative distribution of the data, and the fine-tuning stage learns the reward distribution of the task. Under the PAC learning framework, generalization ability depends on the capacity of the hypothesis space and the coverage of the training samples.

Supporters would say: scale itself is the mechanism. When the number of parameters and the amount of training data cross a certain threshold, the hypothesis space is sufficient to contain the functions needed to solve a wide range of tasks. The performance of the GPT series on multiple benchmarks is cited as evidence—the same set of weights can handle heterogeneous tasks such as translation, programming, mathematics, and legal reasoning without fine-tuning.

Opponents would point out: benchmark performance is not the same as generality. These tasks are still in-distribution, or have a computable similarity to the training distribution. Truly open-ended general intelligence requires rapid adaptation in completely novel domains, and the current evidence is not sufficient.

The PAC (Probably Approximately Correct) learning framework was proposed by Leslie Valiant in 1984 and is a foundational model of computational learning theory. It formally defines "learnable": a concept class is PAC learnable if there exists an algorithm that can output an approximately correct hypothesis with high probability in polynomial time. Under the PAC framework, generalization error depends on the complexity of the hypothesis space (such as VC dimension) and the amount of training samples. The argument cited in the article that "scale itself is the mechanism" is essentially saying that when the number of parameters is large enough, the hypothesis space is sufficient to cover a wide range of tasks, and massive data satisfies PAC's sample coverage requirements. But the PAC framework assumes that training and test data come from the same distribution, which is precisely the technical root of "benchmark performance is not the same as generality"—out-of-distribution generalization has no guarantee under the PAC framework.

Gaps and Open Questions

Synthesizing the Reddit discussion and the existing literature, the mechanistic argument for NTP+RL leading to AGI has three clear gaps:

The absence of goal generation. RL optimizes a given reward, but does not explain where new goals come from. Part of the openness of human intelligence comes from intrinsic motivation, curiosity, and autonomous goal setting, and these have no corresponding mechanism in the standard RL framework.

The absence of embodiment and interaction. Pure text training cannot acquire physical intuition, social feedback, or causal knowledge from real-time interaction. If a world model is learned only from text, it may only be a world model of text.

The absence of continual learning. The current paradigm has two stages, training and deployment, and after deployment the weights are frozen. General intelligence needs to be continuously updated through interaction, not just temporarily adapted in context.

The poster asked for "the strongest argument." The honest strongest version is probably: NTP provides the compression foundation for general representations, RL provides the ability to perform goal-directed search over those representations, and in principle their combination can approach general intelligence, but the three mechanistic gaps of goal generation, embodied interaction, and continual learning have not yet been covered by the current paradigm. This is not a denial of progress, but a marking of the missing inferential links between "can" and "must be able to."

Frequently Asked Questions

Are NTP and autoregressive generation the same thing? Basically yes. NTP is the training objective, and autoregressive generation is the inference method; both are based on the same mechanism of "predicting the next token from the preceding context."

Why is RL important for AGI? Pure NTP learns the distribution of the data, while RL allows the model to actively explore the strategy space under a reward signal, moving from imitation toward goal-directed behavior.

What is the focal point of debate on this question? The core disagreement is whether the capabilities required for general intelligence are an extension of the NTP+RL training distribution or require an entirely new learning mechanism. The former sees it as a matter of scale, while the latter sees it as a matter of paradigm.

One-line Summary

Can NTP+RL lead to AGI? The strongest mechanistic argument and its three unresolved gaps.

Paragraph Summary

The Reddit discussion demands an answer based on mechanistic argument rather than trend extrapolation: can next-token prediction (NTP) plus reinforcement learning (RL) produce open-ended general intelligence? NTP's strongest argument is "compression is understanding"—the optimal predictor is equivalent to the optimal compressor, forcing the model to internalize the regularities of the data. RL provides goal-directed search capability, but the standard RL framework does not explain how new goals are autonomously generated. From the perspective of computational learning theory, scale may allow the hypothesis space to cover a wide range of tasks, but benchmark performance is not the same as generality. The article points to three clear gaps: the absence of goal generation, the absence of embodiment and interaction, and the absence of continual learning. The most honest conclusion is that NTP+RL can in principle approach AGI, but the three mechanistic gaps have not yet been covered by the current paradigm.