Bilibili Hot Pick: A Hands-On Guide to Mastering Diffusion Model Principles and Source Code from Scratch

Diffusion models have drawn explosive attention since around 2022, with top-conference papers emerging endlessly. Although the hype has cooled somewhat, it remains a core hot direction for paper innovation and model improvement. Yet many learners fall into two extremes: either they get stuck grinding through obscure mathematical derivations, or they can only run open-source code but can't explain the underlying logic—let alone modify models to innovate. Recently, a learning path circulating on Bilibili attempts to help beginners systematically master diffusion models through a four-step approach: "principle derivation → lightweight toolkit → source-code closed loop → innovation implementation." This article, drawing on the core points of that video, outlines this practical, paper-publishable learning path.

1. Build the Theory: Clarify the Framework Before Discussing Derivations

The theoretical threshold for diffusion models is indeed high, but beginners don't need to dive into advanced mathematics right away. The video stresses that the key is to clarify the core idea of diffusion models—how the forward diffusion process gradually adds noise and how the reverse generation process gradually denoises—and to fully grasp the complete logic of these two stages. Many papers are packed with dense formulas, yet people get stumped when questioned during their defense; the root cause is that the basic framework isn't solid.

After mastering the basic framework, you can selectively read high-quality papers closely, focusing on mainstream improvement approaches such as model acceleration, lightweighting, and structural optimization. This content maps directly to subsequent innovation directions. By getting familiar with commonly used improvement ideas early, you won't "see only code and no path" when doing research.

The forward diffusion process and the reverse generation process are the two pillars of diffusion models. The forward process is a fixed Markov chain that gradually adds Gaussian noise to the data over T steps until the data becomes approximately pure noise; the noise intensity at each step is controlled by a preset variance schedule (such as linear or cosine scheduling), and this process requires no learning. The reverse process does the opposite: starting from pure noise, it uses a parameterized neural network (usually a U-Net or Transformer) to progressively predict and remove noise, ultimately recovering the original data distribution. The training objective is to have the network predict the noise added at each step, with the common loss being the mean squared error of noise prediction. Understanding the mathematical form and physical intuition of these two processes is the foundation for reading all subsequent improvement papers.

2. Get Hands-On with Tools: Start Practicing Source Code with a Lightweight Open-Source Package

The most common pitfall for beginners is to tackle a massive, overly complex open-source project right away—and then get discouraged. The video strongly recommends starting with the DiT toolkit based on the Transformer architecture. It's widely used in the industry, comes with many ready-to-use open-source models, is highly adaptable, and—more importantly—has clean, concise code that's suitable for line-by-line debugging and understanding while running.

For your own projects, model modifications, and innovations later, this toolkit is equally reusable, so its practicality spans the entire learning cycle. The right approach is to compare the principles, acceleration optimizations, and lightweighting improvements you learned earlier against the DiT source code line by line—mathematical formulas you don't understand, algorithm flows you can't sort out, and experimental logic you can't connect—all get resolved into a closed loop through hands-on source-code practice. First thoroughly master two or three core codebases to fully understand how diffusion models actually run, then move on to advanced algorithm design and implementing your own innovations; the difficulty will be halved directly.

DiT (Diffusion Transformer) is an open-source toolkit that introduces the Transformer architecture into the backbone network of diffusion models. Traditional diffusion models mostly use U-Net as the denoising network, whereas DiT replaces U-Net with a Transformer, leveraging its strong global modeling capability and scalability to achieve performance superior to U-Net on image generation tasks. The DiT toolkit typically includes pretrained DiT models, training scripts, and sampling code, with a clear code structure and decoupled modules—very suitable for line-by-line debugging. Its core idea is to patch the image and treat it as a sequence input to the Transformer, and to inject timestep and class conditions through adaptive layer normalization (adaLN). Mastering the DiT source code lets you understand how Transformers and diffusion models are combined, laying the foundation for subsequent structural innovation.

Model acceleration and lightweighting are key challenges for deploying diffusion models. Diffusion models typically require tens to thousands of sampling steps, making inference slow and difficult to meet real-time demands. Common acceleration approaches include: distillation (e.g., progressive distillation compressing multi-step sampling into few steps), sampler improvements (e.g., DDIM and DPM-Solver approximating the reverse process with fewer steps), and model structure lightweighting (e.g., reducing parameter count, using low-rank decomposition, or pruning). Lightweighting focuses on reducing computation and storage overhead while maintaining generation quality—for example, by adopting more efficient attention mechanisms or hybrid architectures. Closely reading papers in these directions helps you quickly grasp current mainstream efficiency-optimization ideas, which are also high-frequency entry points for paper innovation.

3. Core Principle: Understand the Theory and Master the Source Code Before Discussing Innovation

The video repeatedly stresses one principle: model improvements that skip hands-on source-code practice are castles in the air. Modifying structures without understanding the principles produces unconvincing results; discussing innovation without having run the source code means the experimental logic in your paper won't withstand scrutiny. Only after completing the chain of "principle understanding—source-code practice—experimental validation" can your personal innovations hold up and your research projects land with confidence.

4. Supporting Materials and Suggested Learning Pace

The video's author has compiled a curated collection of diffusion model papers, accompanied by original paper texts, accessible principle-explanation videos, and runnable source code, covering the entire process from theoretical learning to source-code practice to innovation ideas. For beginners, it's recommended to proceed at the pace of "theoretical framework → selected papers → line-by-line DiT source-code debugging → reproducing two or three core codebases → attempting small modifications," with each step measured by whether you can explain it clearly and run it successfully—not by whether you've finished watching the videos.

The diffusion model learning curve is steep at first and flattens later. Once you get through the principles and source code, model improvement and paper innovation become much smoother. Rather than spinning in abstract theory, it's better to get the code running first and then map the formulas back one by one.

One-line Summary

A four-step learning path: from principle derivation to hands-on DiT source code, then to innovation implementation.

Paragraph Summary

The article introduces a four-step path for systematically learning diffusion models from scratch: first clarify the core framework of forward noising and reverse denoising, then selectively read papers on acceleration, lightweighting, and structural optimization; next start with the Transformer-based lightweight DiT toolkit, debugging the source code line by line and connecting principles with code; finally, emphasize understanding the theory and mastering the source code before discussing innovation—model improvements that skip source-code practice lack说服力. The author recommends progressing at the pace of theoretical framework → selected papers → DiT source-code debugging → reproducing core code → small modifications, with accompanying paper collections and explanation videos to help learners get past the initially steep learning curve.