Machine Learning Math Reading List: A Progressive Path from Probability & Statistics to Deep Learning

An Engineering Student's Dilemma: Is This Reading List Right?

I recently came across a highly relatable question on Reddit. An engineering student with a solid foundation in calculus and linear algebra (from continuum mechanics courses) had already grasped the fundamentals of machine learning through Transformers and modern architectures. Now, he wanted to go deeper into the mathematics itself — not hand-wavy intuition videos, but real derivations and classic textbooks that build systematically from intuition to advanced content.

He articulated a key requirement:

"Most importantly, I want to see how the math works inside the models, not abstract theory floating in isolation. Most modern models (including language models) are fundamentally probabilistic, so that perspective matters a lot to me."

This hits on a pain point shared by many self-learners: the disconnect between math and models. Let's take a systematic look at his reading list and give it a thorough review.

reddit source

Breaking Down the Original List: A Three-Tier Learning Framework

The reading list spans three distinct tiers, with a surprisingly coherent logic:

Statistics & Probability

《All of Statistics》 (Wasserman)

Math-Oriented Machine Learning

《Foundations of Machine Learning》 (Mohri et al.)
《Mathematics for Machine Learning》 (Deisenroth et al.)
《The Elements of Statistical Learning》 (Hastie et al., with plans to first read the gentler《ISL》)

Deep Learning

《Deep Learning》 (Goodfellow et al.)

He also emphasized a filtering principle: content quality over exercise quantity. He'd rather have a book with excellent explanations and derivations but fewer exercises than one packed with problems but poor exposition — because exercises can be sourced elsewhere, while weak content can't be fixed. That's a remarkably mature judgment.

Book-by-Book Review: What to Keep, What to Adjust

《Mathematics for Machine Learning》: A Bridge to Get You Started

For someone who already has a solid calculus and linear algebra background, this book functions more as a "review + bridge." Its value lies in reorganizing linear algebra, matrix calculus, and probability theory within a machine learning context — using PCA, linear regression, and SVM as concrete applications of these tools. For readers who want to "see how math works inside models," this is an excellent transitional resource. That said, it may feel a bit elementary for those with strong foundations, so a brisk read-through to connect the dots is probably the right approach.

《All of Statistics》: The Core Text for a Probabilistic Perspective

This book is arguably one of the gold standards for self-studying statistics. It covers probability theory, statistical inference, and some statistical learning in a remarkably compact format. For the stated goal of understanding models as "fundamentally probabilistic," Wasserman delivers strong coverage of both frequentist and Bayesian perspectives. One caveat: the pace is fast and the density is high — it's best suited for mathematically mature readers, which happens to match the questioner's profile perfectly.

《Foundations of Machine Learning》and《ESL》: Two Theory-Heavy Classics

Both books sit in the "rigorous" category, but with different emphases:

《Foundations of Machine Learning》 focuses on learning theory (PAC learning, VC dimension, generalization bounds, theoretical guarantees for kernel methods). It answers the question: why does machine learning actually work?
《ESL》 takes a statistical modeling perspective, covering everything from linear methods to tree models, boosting, and unsupervised learning — broader in scope and more practically oriented.

The strategy of reading《ISL》before《ESL》is wise — the former builds intuition in an accessible way, while the latter adds rigorous derivations.

《Deep Learning》(the "Goodfellow Bible"): Requires Realistic Expectations

This is a classic reference for deep learning, and the early chapters (linear algebra, probability, numerical computation, ML fundamentals) are excellent as mathematical groundwork. But it's worth noting: this book was published in 2016, before Transformers. For a reader already familiar with modern architectures, the latter half of the book (especially the sequence modeling sections) is noticeably dated. It's best treated as a reference for "mathematical foundations + early deep learning principles" rather than a guide to current architectures.

Key Books Worth Adding

Based on the three stated priorities — content quality first, probabilistic perspective throughout, and seeing math operate within models — the following texts frequently come up in community discussions:

Probabilistic & Bayesian Perspectives

《Pattern Recognition and Machine Learning》 (Bishop): If one book perfectly embodies "a probabilistic framework woven through every model," this is it. From Bayesian linear regression to graphical models and variational inference, nearly every method is presented within a probabilistic framework, with thorough derivations.
《Probabilistic Machine Learning》 (Kevin Murphy): Think of this as a modern update to Bishop, split into an introductory and an advanced volume. It covers deep learning, Transformers, diffusion models, and other cutting-edge topics — all through a consistently probabilistic lens. For bridging the gap left by the Goodfellow book's age, this series is the ideal choice.

Modern Supplement for Deep Learning

《Understanding Deep Learning》 (Simon Prince): Published in 2023 and freely available online, this book features clear explanations and beautiful illustrations covering modern architectures like Transformers and diffusion models. It's an excellent replacement or companion for the Goodfellow book.

Closing Thought: It's Not About Length — It's About Depth

The original reading list is already quite solid, with a clear structure and logical progression. The key adjustment needed is in the deep learning section — the mathematical foundations in Goodfellow are worth keeping, but the coverage of modern architectures should be supplemented by Murphy's《Probabilistic ML》or Prince's《Understanding Deep Learning》. And if the goal is specifically to have "a probabilistic perspective woven throughout," then Bishop and Murphy's books should form the backbone of the list, not mere supplements.

For self-learners, the most important thing has never been how long the reading list is — it's whether you can commit to truly working through one or two core texts and doing the derivations by hand. The more reference books you accumulate, the easier it is to fall into the trap of collecting without reading.