From Understanding Papers to Independent Innovation: A Guide to Advancing Mathematical Skills for ML Researchers

A Common Frustration: Why Can You Understand but Not Create?

In the machine learning research community, there's a bottleneck that virtually every junior researcher encounters. A Reddit user posed a highly representative question in the community:

"As part of a lab, I read ML papers frequently and can mostly understand the math in them. But I want to transition from 'consuming' this type of content to 'producing' it. How do the smart people at top labs have such a high level of mathematical ability to produce results? How can I improve to reach that standard?"

This question touches on a core cognitive gap in research training — understanding someone else's derivations and independently creating new mathematical tools are two entirely different abilities. The former is passive pattern recognition; the latter requires active abstraction, modeling, and proof capabilities.

This article explores how to cross the divide from "understanding" to "creating," drawing on general principles of research training.

The Fundamental Difference Between Understanding and Creating

Reading Comprehension Is "Fill in the Blank"; Research Is "Formulating the Question"

When you read a paper, the author has already laid out the logical chain: problem definition, assumptions, derivation steps, and conclusions. Your brain performs verification work — checking whether each step holds. This is essentially a "fill-in-the-blank" cognitive activity. The human brain is naturally adept at recognizing correctness within a given framework.

This difference has deep theoretical foundations in cognitive science. According to Bloom's Taxonomy, cognitive activities can be ranked from low to high: remembering, understanding, applying, analyzing, evaluating, and creating. Reading a paper and following derivations roughly corresponds to the "understanding" and "analyzing" levels, while independently producing research results occupies the highest "creating" level. Cognitive science research shows that verification cognition and generation cognition engage fundamentally different brain regions and thinking modes: the former relies mainly on pattern matching and logical verification circuits, while the latter requires deep coordination of working memory, divergent thinking, and abstract reasoning. This explains why someone can easily follow a complex proof yet feel completely lost when facing a blank page.

But when producing research results, no one gives you the framework. You must judge for yourself: Is this problem worth studying? What mathematical language should describe it? Which assumptions are reasonable? How do you construct a proposition that can be proved or verified? This is "formulating" cognition, and the difficulty increases by orders of magnitude.

The "Tacit Mathematical Ability" of Top ML Researchers

The mathematical ability of researchers who produce results in top labs often manifests in several hard-to-observe dimensions:

• Mathematical intuition: They can "feel" why a method works, even predicting the form of results before rigorous proof.
• Fluent tool deployment: Probability theory, linear algebra, optimization theory, and convex analysis are not knowledge they need to look up, but rather a "native language" they can combine and use instantly.
• Abstraction and reduction ability: They excel at stripping away surface appearances from complex real-world problems and reducing them to known or solvable mathematical structures.

The "tacit ability" mentioned here has a precise term in knowledge management — tacit knowledge, first systematically articulated by philosopher Michael Polanyi in 1958. Polanyi noted that "we know more than we can tell" — this knowledge cannot be fully transmitted through language or text and can only be acquired through practice. In mathematical research, this tacit knowledge is particularly evident. Historically, Ramanujan was renowned for his astonishing mathematical intuition, often "seeing" the correct form of formulas directly; Terence Tao has discussed the cultivation of mathematical intuition multiple times on his blog, pointing out that such intuition is actually a compressed representation of vast experience in the subconscious. In modern ML theory research, the ability exhibited by researchers like Sham Kakade and Percy Liang to rapidly establish mappings between problems and tools is precisely this long-accumulated tacit knowledge at work.

These abilities are not the exclusive province of natural talent — they are the result of long-term deliberate training.

Systematic Methods for Improving Mathematical Production Ability

Step One: Shift from Passive Reading to Active Reconstruction

Merely "reading and understanding" papers is far from sufficient. Truly effective training is closing the paper and re-deriving the core results yourself. When you close the PDF and try to independently walk from the problem definition to the conclusion, you'll immediately discover gaps in your understanding — those steps you thought you "understood" were actually just cases where you accepted the author's conclusions without internalizing the derivation logic.

This method is known as "retrieval practice" in learning science, and extensive empirical research shows that actively retrieving information from memory is over 50% more effective for learning than repeatedly reading the same material. For mathematical derivations, this means rebuilding proof chains without any reference materials, which exposes your dependence on intermediate steps and insufficient understanding of key turning points.

An advanced approach: after reading the methods section of a paper, stop and ask yourself "If it were me, how would I solve this problem?" Then compare with the author's approach. This "predict-then-compare" training rapidly develops research intuition.

Step Two: Build the "Load-Bearing Pillars" of Mathematical Foundations

For ML research, several mathematical branches form an indispensable foundation:

• Linear algebra: Not just matrix operations, but understanding the geometric meaning of eigenvalues, SVD, subspaces, and other concepts. In ML research, SVD decomposition directly gave rise to Principal Component Analysis (PCA) and matrix completion theory, and spectral methods are the theoretical pillars of graph neural networks and community detection. Understanding random projection results like the Johnson-Lindenstrauss lemma requires deep linear algebra intuition about high-dimensional geometry. Start with Gilbert Strang's Linear Algebra and Its Applications, then advance to the linear algebra applications in Roman Vershynin's High-Dimensional Probability.

• Probability theory and statistics: Bayesian inference, concentration inequalities, and probability from a measure-theoretic perspective are keys to understanding modern ML theory. Specifically, concentration inequalities (such as Hoeffding's inequality, Bernstein's inequality, and Matrix Bernstein inequality) are core tools for deriving generalization bounds in statistical learning theory; the measure-theoretic perspective is a necessary prerequisite for understanding generative models (such as the Fokker-Planck equation in diffusion models and optimal transport theory). Rigorous understanding of PAC learning theory, VC dimension theory, and Rademacher complexity all depend on a solid probability theory foundation.

• Optimization theory: Convex optimization, Lagrangian duality, and convergence analysis of gradient methods. Research on why SGD can find good local minima in non-convex landscapes in deep learning (such as Rong Ge et al.'s work on saddle points), convergence proofs of the Adam optimizer (and later-discovered non-convergence counterexamples), and recent analysis of optimization landscapes in over-parameterized networks under Neural Tangent Kernel (NTK) theory are all representative deep applications of optimization theory in ML. Boyd's Convex Optimization is the standard starting point for this direction.

• Real analysis and functional analysis: Increasingly important in theoretical ML research, especially indispensable when dealing with function spaces and infinite-dimensional problems. Reproducing Kernel Hilbert Space (RKHS) theory is the mathematical foundation of kernel methods; the universal approximation theorem for neural networks requires the language of functional analysis to state and prove; the recently emerging field of operator learning (such as DeepONet, Fourier Neural Operator) models directly on function spaces. Understanding these frontier directions requires solid command of Banach spaces, bounded linear operators, weak convergence, and other concepts.

The key is not whether you've "studied" these subjects, but whether you can apply them flexibly. Test your mastery by working through proof exercises, not just reading textbooks.

Step Three: Deliberate Practice of Proofs and Modeling

The original poster asked "Is it just practice? What should I practice?" The answer is emphatically yes, and the direction of practice is crucial:

The concept of "deliberate practice" was systematically proposed by psychologist Anders Ericsson. Its core characteristics include: clear goals, immediate feedback, targeted training of weak areas, and continuously pushing beyond the comfort zone. Ericsson's research shows that expert-level performance depends not on practice duration itself, but on the quality and specificity of practice. Applying this theory to mathematical research training means you shouldn't vaguely "read more papers" or "do more exercises." Instead, identify the weakest link in your ability chain — Is your formal modeling ability insufficient? Is your mastery of the tool library not fluent enough? Or do you lack the inductive ability to go from concrete to abstract? — and then design targeted training accordingly.

1. Redo derivations from foundational papers: Select seminal papers in your field and reproduce their mathematical derivations line by line. For example, those studying deep learning theory should try to independently derive the core theorem of the Neural Tangent Kernel; those studying reinforcement learning should derive the Policy Gradient Theorem and its variance analysis from scratch. Every point where you get stuck is a precise marker of your ability boundary.

2. Try to improve or generalize existing results: Can you relax an assumption? Can you generalize to a more general case? This is direct practice for producing new results. Many important academic contributions essentially stem from this kind of questioning: "If we replace condition X with condition Y, does the conclusion still hold?" For example, classical matrix completion theory initially relied on an "incoherence" assumption, and later researchers gradually relaxed this condition, producing a large body of high-quality work.

3. Create your own problems: Observe an experimental phenomenon, attempt to describe it formally in mathematical language, build a model, and verify it.

Mindset and Environment: Critically Underestimated Factors

Acknowledge That Mathematical Ability Improvement Is a Long-Term Process

Many people mistakenly believe top researchers "were always strong from the beginning." In reality, they also went through a prolonged journey from imitation to creation. The growth curve of mathematical production ability is often nonlinear — you may feel stagnant for a long time, then suddenly have a breakthrough at some point. This plateau period is precisely when abilities are being restructured at a fundamental level, and you should not give up because of it.

Cognitive science calls this phenomenon a "learning plateau," which is well-documented across all domains of skill acquisition. The neuroscience explanation is: during the plateau period, the brain is reorganizing and optimizing synaptic connections (so-called "synaptic pruning" and "myelination"), integrating scattered knowledge fragments into efficient cognitive schemas. This process manifests externally as stagnation or even brief regression, but internally it represents the construction of higher-level ability structures. Research has shown that "eureka" experiences in mathematics learning often occur after a person has accumulated many failed attempts on the same type of problem — those seemingly fruitless struggles actually build deep representations of problem structure at an unconscious level. Terence Tao has also mentioned on his blog that many of his research breakthroughs came from "suddenly seeing the structure of a problem after a long period of confusion."

Leverage the Collaborative Lab Environment

The original poster mentioned being part of a lab, which is actually an enormous resource. Discussing unsolved problems with advisors and peers is one of the most efficient ways to train research ability. Observing how senior researchers attack a new problem, how they adjust their thinking when stuck — this kind of "tacit knowledge" is nearly impossible to obtain from books. Actively participating in group meetings, even challenging others' derivations, provides invaluable real-world training.

Vygotsky's "Zone of Proximal Development" (ZPD) theory provides theoretical support for this: the level an individual can reach working independently is limited, but under the guidance of more experienced people, they can access higher ability levels. Lab group meetings, one-on-one discussions, and whiteboard derivations essentially create this kind of "scaffolded" learning environment. MIT's mathematics department has a famous tradition — graduate students discuss problems on blackboards in the common room, and many important ideas are born through this kind of informal collaboration.

Start with Small Original Contributions

You don't need to aim for breakthrough results from the start. Even supplementing an existing method with a more rigorous proof, finding a counterexample, or proposing a small improvement is a real step from "consuming" to "producing." Research ability accumulates through repeated small original attempts.

Conclusion

The transition from understanding papers to producing papers is essentially growing from a verifier into a creator. There are no shortcuts on this path, but there is a clear methodology: active reconstruction rather than passive reading, solidifying mathematical foundations, deliberate practice of proofs and modeling, leveraging environmental resources, and maintaining a long-term mindset. Researchers at top labs don't possess mysterious innate talent — they simply walked the path from imitation to innovation earlier and more systematically.

Key Takeaways