Advanced Machine Learning Reading List: How to Choose the Right Book Among These 4 Classics?

From Beginner to Advanced: The Dilemma of Choosing a Machine Learning Learning Path

On the road to self-studying machine learning, choosing the right learning materials is often more crucial than grinding through problems. Recently, a learner started a discussion on Reddit: he already had two introductory books as a foundation—Grokking Machine Learning and Oxford University Press's Fundamentals of Machine Learning—and wanted to pick up another to complete his knowledge system. His goal was crystal clear: model finetuning and small language models (SLM), while hoping the chosen content would retain long-term technical relevance.

Background: What are Finetuning and SLM? Model finetuning refers to the technique of further training a model's parameters on top of a large-scale pre-trained model using datasets from a specific domain or task. With the rise of large pre-trained models like GPT and BERT, finetuning has become a core paradigm in NLP and AI application development—it allows general-purpose models to be adapted to specialized scenarios at a cost far lower than training from scratch.

The rise of this paradigm is essentially the mature implementation of Transfer Learning in the NLP field, and transfer learning itself has a longer developmental history. As early as in computer vision, transferring CNN models pre-trained on ImageNet to downstream visual tasks had already validated the effectiveness of this paradigm. Transfer learning in NLP started relatively late—early methods like Word2Vec (2013) and GloVe (2014) only achieved transfer at the word-vector level and could not capture contextual dynamic semantics. ELMo (2018) first introduced context-dependent word vectors, after which BERT and GPT pushed transfer learning to the level of entire model architectures, fundamentally changing how NLP engineering works. Behind this evolution lies a profound data-economics logic: labeled data is expensive and scarce, while unlabeled text on the internet is nearly inexhaustible—the pre-training paradigm is precisely the core mechanism that converts this data asymmetry into a technical advantage. Models trained on massive amounts of unlabeled text encode rich linguistic and world knowledge in their parameters, and this knowledge can be "transferred" to downstream tasks. Google's release of BERT and OpenAI's release of GPT-1 in 2018 were important milestones for this paradigm, and "pre-training + finetuning" gradually replaced the old paradigm of training a dedicated model from scratch for each task, becoming the standard workflow of NLP engineering.

Small Language Models (SLM) typically refer to models with parameter counts in the 1B to 13B range (such as Phi-3, Mistral 7B, Llama 3 8B, etc.), which can be deployed locally on consumer-grade GPUs or even CPUs, with extremely low inference costs, making customized finetuning accessible to individual researchers or small-to-medium teams. The rise of SLMs represents a technical route distinct from simply stacking parameter counts: through knowledge distillation, data quality optimization (such as the "textbook-quality data" strategy of the Phi series models), and more refined training schemes, they achieve practical performance close to GPT-4 at a parameter scale far smaller. The popularity of SLMs has made "ordinary developers being able to own their own private large model" a reality, so mastering the ability to finetune SLMs is rapidly becoming an important competitive edge for AI engineers.

He listed four candidates:

• Hands-On Machine Learning with Scikit-Learn & PyTorch (Aurélien Géron)
• Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow (Aurélien Géron)
• Build a Large Language Model From Scratch (Sebastian Raschka)
• Deep Learning with Python 3rd Edition (François Chollet)

All four books are widely recognized classics in the industry, but their focuses are entirely different. This article will analyze the positioning and applicable scenarios of each book one by one, based on this learner's specific goals.

The Underlying Logic of a Machine Learning Learning Path The machine learning knowledge system has a clear hierarchical dependency: mathematical foundations (linear algebra, calculus, probability theory) → traditional machine learning algorithms → deep learning fundamentals → Transformer architecture → pre-trained large models → finetuning and alignment techniques. Skipping ahead often leads to the predicament of "being able to tune parameters but not knowing why it works." Understanding this dependency chain is key to judging "which layer the current gap is in" when choosing a book. For a learner who already has two introductory books as a foundation, the core question is: Is the deep learning foundation already solid enough to move directly into learning Transformers and finetuning? The answer directly determines the priority of book selection.

Breaking Down the Four Classic Books One by One

Géron's Two Hands-On Machine Learning Books

Aurélien Géron's Hands-On Machine Learning series can be considered the "bible" of the machine learning practice field. Known for its strong engineering-practice orientation, it covers everything from traditional machine learning (classification, regression, ensemble learning) all the way to deep learning, encompassing nearly the complete skill tree a practitioner needs.

You may not have noticed, but the two versions the poster listed differ significantly. The classic version is Scikit-Learn, Keras, and TensorFlow, which is currently the most widely circulated, with the deep learning portion centered on TensorFlow/Keras. The Scikit-Learn & PyTorch version is relatively uncommon—here it's important to understand a key industry background:

PyTorch vs TensorFlow/Keras: The Tech Stack Divide in the LLM Era PyTorch (open-sourced by Meta in 2016), with its "dynamic computation graph" and design closer to native Python style, quickly won favor in academic research circles. Understanding the engineering-philosophy differences behind this technical choice helps make wiser tech-stack decisions: the "dynamic computation graph" (Define-by-Run) adopted by PyTorch and the early "static computation graph" (Define-and-Run) of TensorFlow represent two entirely different engineering philosophies. Static graphs fully define the computation flow before execution, facilitating compiler optimization and cross-platform deployment, but debugging is difficult; dynamic graphs build the computation graph on the fly during each forward pass, naturally supporting Python's conditional branches and loop structures, with a debugging experience almost indistinguishable from ordinary Python programs—this is extremely friendly for researchers who need to rapidly iterate and experiment with different model architectures. This "research friendliness" is the fundamental reason PyTorch quickly captured market share in academia, and academic code subsequently flowed into industry, further strengthening PyTorch's dominance in the LLM development ecosystem. Although TensorFlow 2.x introduced Eager Execution to close this gap, the first-mover advantage was already established.

Entering the LLM era, this landscape has become even clearer: the code implementations of the vast majority of top NLP conference papers are based on PyTorch; Hugging Face's Transformers library is built around PyTorch, and the official implementations of nearly all mainstream open-source large models (LLaMA, Mistral, Qwen, Gemma, etc.) prioritize PyTorch support; the toolchains for mainstream finetuning techniques like LoRA and QLoRA (such as the PEFT library) are also based on the PyTorch ecosystem. It's worth mentioning that Hugging Face has grown from an initial chatbot company into the most important open-source infrastructure provider in the AI field—its Transformers library hosts over 500,000 pre-trained models, and its accompanying PEFT library, TRL library (reinforcement learning finetuning), and Accelerate library (distributed training acceleration) together form a complete LLM development toolchain. Familiarity with this ecosystem is almost a prerequisite for entering industry LLM development positions. Although TensorFlow still has advantages in industrial deployment (TFLite, TF Serving), PyTorch has become the de facto industry standard in LLM research and development.

PyTorch currently dominates the research community and the LLM ecosystem. If the content of this version is mature, it would be a better fit for the actual tech stack of those who want to do finetuning and SLM.

For the learner in this case, Géron's books are more suitable as reference tools for building a solid foundation. However, it should be noted that this set of books has relatively limited coverage of LLMs and finetuning; its strengths lie in traditional ML and deep learning fundamentals rather than the large-model specialization direction.

Chollet's Deep Learning with Python 3rd Edition

François Chollet is the creator of the Keras framework, and his Deep Learning with Python is renowned for its clear concepts and elegant code. The third edition has been substantially updated, adding content on Transformers and generative models.

The biggest feature of this book is that it makes the profound accessible—rather than scaring readers off with heaps of mathematical formulas, it uses intuition and code to help understand the essential logic of deep learning. For learners who have some foundation and want to truly understand how neural networks work, it's an excellent choice. However, it still centers on the Keras ecosystem, which is somewhat distant from the PyTorch + Hugging Face tech stack commonly used in current LLM finetuning. For learners whose goals point clearly toward finetuning and SLM, Chollet's book is more suitable as a "conceptual supplement" than a main focus.

Raschka's Build a Large Language Model From Scratch

If we start purely from the goal of "finetuning and SLM", Sebastian Raschka's Build a Large Language Model From Scratch is undoubtedly the most on-target choice among the four books.

Raschka is an assistant professor of statistics at the University of Wisconsin-Madison and also holds a research position at Lightning AI. He has over a decade of open-source contribution experience in the machine learning education field: his GitHub repository "rasbt/python-machine-learning-book" was once one of the most-starred machine learning tutorials on GitHub, and the three editions of his accompanying book Python Machine Learning have sold over 100,000 copies cumulatively. His role as a researcher at Lightning AI (the parent company of PyTorch Lightning) gives him a practical perspective on cutting-edge LLM training infrastructure—PyTorch Lightning is a high-level wrapper around the native PyTorch training loop, widely used to simplify the engineering complexity of large-scale model training. This book continues his consistent writing style of combining academic rigor with engineering practicality, distinguished by rigorous code comments and step-by-step derivations. It is specifically designed for the growth path of engineers who "want to truly understand the underlying mechanisms and implement them by hand"—it is neither an application guide that stops at the API-calling level nor a daunting theoretical monograph, forming a sharp contrast with the current abundance of API-focused LLM tutorials.

This book guides readers to implement a GPT-like model from scratch using PyTorch, covering the complete workflow from tokenization, attention mechanisms, and Transformer architecture to pre-training and finetuning. Before diving into this book, understanding the core design ideas of the Transformer is highly beneficial:

The Transformer Architecture: The Cornerstone of Modern Large Language Models The Transformer was proposed by Vaswani et al. in the 2017 paper Attention Is All You Need, fundamentally changing the technical paradigm of the natural language processing field. Its core innovation lies in the "self-attention mechanism": unlike RNN/LSTM, which need to process sequences step by step, the Transformer allows the model to simultaneously attend to information at all positions in the sequence when processing each word, thereby effectively capturing long-range dependencies and supporting large-scale parallel computation.

However, this powerful capability comes with a significant computational cost: the complexity of the self-attention mechanism grows quadratically with sequence length (O(n²)), and expanding the context window from 2048 to 8192 causes a 16-fold increase in attention computation. This bottleneck has spawned numerous improved variants such as sparse attention, linear attention, and sliding-window attention (as adopted by Mistral). Meanwhile, the "neural network Scaling Laws" proposed in 2020 revealed the power-law relationship between model performance and parameter count, data volume, and computation, providing a theoretical basis for the development of ultra-large-scale models.

Key components of the model include: multi-head attention (Multi-Head Attention, allowing the model to simultaneously capture different types of dependencies from multiple "perspectives"), positional encoding (Positional Encoding, injecting positional information into an attention mechanism that itself contains no sequence-order information), and the feed-forward network in each layer (Feed-Forward Network, responsible for applying a non-linear transformation to the attention output). The GPT series (including the underlying model of ChatGPT) adopts the "decoder" (Decoder-only) variant of the Transformer, specifically optimized for autoregressive text generation tasks—that is, when predicting the next word each time, it can only see the previous context. BERT, on the other hand, adopts the "encoder" (Encoder-only) variant, using bidirectional attention suitable for text-understanding tasks. Understanding how components like multi-head attention, positional encoding, and feed-forward network layers work is a prerequisite for deeply mastering LLM finetuning techniques—and this is precisely the core teaching content of Raschka's book.

This book does not stop at the API-calling level but lets you truly see what happens "under the hood" of large language models. For those who want to do finetuning and small language models, this bottom-up understanding is crucial.

Recommendations for Specific Goals

Clear Goals Determine the Order of Book Selection

Returning to the poster's core needs—finetuning + SLM + future-proof technical relevance. Based on this, the priority recommendations are as follows:

First choice: Raschka's Build a LLM From Scratch. This is the only work among the four books that directly hits the "large model + finetuning" goal, and it uses the PyTorch tech stack, aligning seamlessly with mainstream toolchains such as Hugging Face and LoRA finetuning. For learners hoping to stay competitive in the LLM era, it is the choice with the strongest "future relevance."

Second choice: Géron's Hands-On ML (PyTorch version preferred). If the foundation is not yet solid, using it first to fill in the engineering-practice capabilities of deep learning and PyTorch will make subsequent study of Raschka's content twice as effective with half the effort. This order also aligns with the knowledge-hierarchy dependency mentioned earlier—Géron is at the "deep learning fundamentals" layer, while Raschka is at the "pre-trained large models → finetuning" layer. Solidifying the foundation layer first before climbing upward is the most efficient path.

Thoughts Beyond the Books

It's worth pointing out that the poster already has two introductory books and may be falling into the trap of "book-hoarding anxiety." In the machine learning field, thoroughly reading one book + extensive hands-on practice far outweighs collecting many books and skimming them superficially.

For highly practical directions like finetuning and SLM, it's recommended to directly get hands-on with the Hugging Face Transformers library while reading Raschka's book, trying to finetune open-source small models (such as Llama 3 8B, Qwen, etc.) using LoRA/QLoRA.

LoRA and QLoRA: Industry-Standard Tools for Efficient Finetuning The core idea of LoRA (Low-Rank Adaptation, proposed by Microsoft Research in 2021) is that the parameter-update matrix of a pre-trained model has a "low intrinsic rank" property—the dimensions of information that actually need to be adjusted during finetuning are far lower than the surface dimensions of the parameter matrix. Based on this insight, LoRA injects low-rank matrix-decomposition "adapters" into the key linear layers of the model without modifying the original pre-trained weights: it decomposes the original weight update ΔW into the product of two low-rank matrices A and B (ΔW = BA), and trains only these newly added small number of parameters (usually only 0.1%~1% of the total parameter count) to achieve results close to full finetuning on specific tasks. This makes it possible to finetune 7B or even 13B parameter models on a single consumer-grade GPU (such as the RTX 3090/4090).

QLoRA takes this a step further. To understand its principle, one must first grasp the basic logic of model quantization: standard PyTorch models default to storing parameters in float16, and a 7B parameter model requires about 14GB of VRAM. Quantization techniques compress storage requirements by mapping parameters from high-precision floating-point numbers to low-bit integers (such as int4), theoretically compressing VRAM usage to 25% of float16. The "4-bit NormalFloat" (NF4) format used by QLoRA is a quantization scheme specifically optimized for the distribution characteristics of neural network weights—neural network weights typically follow a normal distribution, and NF4 uses non-uniform quantization intervals so that each quantization level corresponds to a similar number of weight values, thereby preserving more precision information at the same bit width. The key point is that the quantized base-model weights remain frozen during training, while the low-rank adapters introduced by LoRA are still stored and trained in bfloat16 precision—this mixed-precision strategy of "quantized base model + full-precision adapters" is the core of how QLoRA achieves an exquisite balance between VRAM efficiency and training precision, reducing the finetuning VRAM requirement of a 7B model to about 5GB, bringing it within reach of mainstream consumer-grade GPUs. Hugging Face's PEFT library provides out-of-the-box implementations of LoRA and QLoRA, and combined with the TRL library (supporting SFT supervised finetuning and RLHF reinforcement-learning finetuning), it forms the current standard toolchain for SLM finetuning engineering practice.

Only by advancing theory and practice alternately can you truly internalize what you have learned.

Conclusion: There Is No Best Book, Only the Book Best Suited to Your Goals

Each of the four books has its own strengths; the key lies in how well it matches your own goals. For learners aiming at finetuning and small language models, you can refer to the following concise guidance:

• Want to deeply understand LLM underlying principles and implement finetuning by hand → Raschka
• Want to fill in deep learning engineering-practice fundamentals → Géron (PyTorch version)
• Want to systematically master core deep learning concepts → Chollet

After choosing a good book, what matters more is continuous hands-on practice. In the rapidly evolving AI field, the true core competitiveness that stands the test of time is the ability to build, understand, and finetune models from scratch. Books provide the map, but true insight only happens through walking—use Raschka's book to understand the internal mechanisms of GPT, then use Hugging Face's toolchain to apply LoRA finetuning to real projects. This "theory → tools → practice" path is the most efficient gateway into the LLM development field today.

Key Takeaways