LLMs Cross a Critical Threshold: Surpassing Top Experts on Some Research Problems

LLMs Have Crossed an Invisible Threshold in Scientific Research

Large language models (LLMs) are approaching — and in some research domains have already surpassed — the world's top human experts. Many in the field view this as a defining inflection point. A widely discussed observation on Twitter noted that LLMs have already exceeded the best human experts on certain research problems, and that this reveals an easily overlooked truth: in the realm of scientific discovery, there is a fundamental difference between a model that is slightly below top human performance and one that is slightly above it.

This distinction explains why recent progress in mathematics has felt like a sudden explosion. In reality, the capability curve has been climbing steadily for years — but for scientific research, a model only begins to generate genuinely new knowledge once it clears the highest bar of human expertise. Until that point, no matter how close it gets, the model remains a follower rather than a pioneer.

Why the Threshold Effect Matters So Much

Scientific discovery differs fundamentally from typical benchmark tasks: it doesn't reward being "close to correct" — only breakthroughs count. A model that falls slightly short of a human expert cannot contribute conclusions that humans haven't already reached. But a model that exceeds human experts by even a small margin can potentially solve problems that no one has solved before.

This is the essence of the threshold effect. During gradual capability improvement, most gains are invisible — users don't feel a meaningful qualitative difference when a model goes from 80% to 90% accuracy. But once a model clears the ceiling set by human experts, results seem to appear in a sudden rush. The original poster argues that the recent burst of achievements in mathematics is a direct manifestation of this mechanism: capabilities didn't suddenly spike — they simply crossed the line that separates "valuable" from "not yet valuable."

Capabilities Remain "Jagged"

It's worth emphasizing that this breakthrough does not mean LLMs have comprehensively surpassed humans across the board. The original poster acknowledges that LLM capabilities remain jagged — models still fall clearly short of humans in many respects.

The term "jagged capability" describes how models can perform remarkably well on highly structured, verifiable tasks, yet fail on commonsense reasoning or inference that humans find trivially easy. Mathematics was among the first domains to see a breakthrough partly because its results can be strictly verified, its reasoning chains are explicit, and it naturally lends itself to high-quality training and feedback signals. In research domains that depend more heavily on experimentation, intuition, and cross-disciplinary synthesis, the model's weaknesses remain pronounced.

In other words, "surpassing experts" is currently a localized, problem-specific phenomenon — not a wholesale dominance at the level of general intelligence. Understanding this unevenness is essential for accurately assessing AI's real role in scientific research.

---

The concept of "jagged capability" was originally coined by researchers to describe the puzzling distribution of abilities in large language models: the same model can solve graduate-level mathematics problems yet frequently fail at elementary tasks like calculating what date it will be in one week. This unevenness is not accidental — it is closely tied to the distribution of training data, the quality of reinforcement learning feedback signals, and the intrinsic structure of different tasks. Highly formalized, symbolic tasks (such as mathematical proofs and code generation) are naturally amenable to automated verification, generating abundant high-quality supervision signals that drive rapid capability gains. Tasks requiring tacit knowledge, cross-modal perception, or "common-sense physical intuition," by contrast, progress slowly due to the absence of clear feedback mechanisms. Recognizing the existence of jagged capability helps avoid two common cognitive traps: over-extrapolating a model's overall intelligence from its impressive performance on a hard problem, and dismissing genuine domain-specific breakthroughs because the model stumbled on something that seemed "simple."

---

The Prelude to a New Era of Scientific Discovery?

Building on progress in mathematics, the original poster offers a cautiously optimistic prediction: as models continue to improve, breakthroughs will emerge in other fields as well, potentially ushering in a new era of AI-driven scientific discovery.

The logic is that the threshold effect is not unique to mathematics. Whenever a discipline's problems are sufficiently verifiable and capable of forming effective feedback loops, the cumulative growth of model capabilities will eventually clear the human expert ceiling, triggering a similar wave of results. Physics, chemistry, biology, and materials science are all plausible candidates for the next wave.

Of course, this vision comes with real constraints. Many scientific fields lack the clean verification mechanisms that mathematics offers — experimental costs are high, feedback cycles are long, and the jagged gaps in model capabilities may be amplified in complex, real-world settings. Rather than interpreting this as "AI is about to replace scientists," it is more apt to see a new collaborative paradigm taking shape: models handle high-speed exploration and breakthroughs within verifiable solution spaces, while humans focus on posing questions, designing validation, and integrating cross-disciplinary insights.

---

Verifiability is the core variable that determines which disciplines will benefit first from AI breakthroughs. Mathematics leads not only because its propositions have clear truth values, but because modern proof-assistant tools — such as Lean, Coq, and Isabelle — can automatically verify the correctness of formal proofs, providing reinforcement learning with precise reward signals. This feedback loop allows models to improve at scale without requiring human experts to review each step. By contrast, hypotheses in biology or materials science often require weeks or even years of wet-lab experiments to validate; the extended feedback cycle drastically reduces the available signal for reinforcement learning. The question of "which field will be the next mathematics" is, at its core, a question about which fields can build low-cost, high-speed automated verification infrastructure — whether through computational simulation, high-throughput experimentation, or formal specification frameworks.

---

Conclusion: The Significance of an Inflection Point

This brief but widely circulated observation resonated because it identified a truth hidden beneath the capability curve: in scientific research, "just barely exceeding humans" is far more disruptive than "perpetually approaching humans." Mathematics is simply the first field where this effect has become visible. If the threshold effect replicates across more disciplines, we may be standing at the starting point of an AI-driven wave of scientific discovery. Optimism is warranted — but so is a clear-eyed recognition of the current limitations and unevenness of model capabilities.