The Full LLM Building Pipeline: Why Most Explanations Stop at the Transformer

Most "how LLMs work" explainers stop dead at the Transformer and attention mechanisms. But the parts that actually determine whether a model is good to use—data pipelines, post-training recipes, evaluation systems, systems engineering—usually get a single sentence.

A recent note circulating in technical communities turns Yann Dubois's guest lecture at Stanford CS229 (Building Large Language Models) into a timestamped index you can check line by line, packaged as an installable agent skill. It doesn't say anything new, but it makes clear what the industry is actually competing on.

Five Components: Who's Actually Optimizing What

The speaker breaks LLM building into five parts and delivers an uncomfortable conclusion: the architecture that academia clusters around is precisely not where the actual leverage lies.

Component	Determines	Driven by
Architecture	Capacity and shape	Academia
Training objective	What counts as "good" during training	Academia
Data	What the model has seen	Industry
Evaluation	Whether you can tell it got better	Industry
Systems	Whether training can finish	Industry

This division isn't accidental. Architectural innovation publishes papers fast and is reproducible; data cleaning, evaluation design, and distributed training—none of these get you into top conferences, yet they directly determine model success or failure. The Llama team's data group was about 15 people, more than the modeling group, and used more CPUs than GPUs—that detail alone says it all.

Pretraining: What "Training on the Internet" Actually Means

The phrase "just train on internet data" conceals all the work. The real pipeline is roughly eight steps:

1. Crawl—Common Crawl is about 250 billion pages, roughly 1 PB.
2. Extract—HTML to text. Random pages are full of menus, footers, and truncated sentences.
3. Filter—NSFW, harmful content, PII, via blocklists plus small classifiers.
4. Deduplicate—Template text, multiple URLs for the same page, paragraphs repeated thousands of times.
5. Heuristics—Anomalous token distributions, odd word lengths, too short or too long.
6. Model filtering—Train a classifier on "sites linked from Wikipedia vs. random web pages," favoring the former.
7. Mixing—Upweight code and books, downweight entertainment content. More code measurably improves reasoning ability.
8. Final stage—High-quality data with a decaying learning rate—essentially overfitting on Wikipedia-grade text.

On scale: loss falls as a power law with parameters, data, and compute, so small-scale experiments can predict large-scale results. Chinchilla's conclusion was about 20 tokens per parameter—early models were generally "data-starved."

Post-Training: From Autocomplete to Assistant

SFT is next-token training on ideal answers written by humans. The key insight: it teaches format and persona, not knowledge. LIMA scaled examples from 2,000 to 32,000 with almost no change in quality. It also teaches models to give plausible-sounding answers instead of admitting they don't know—one source of hallucinations.

RLHF replaces demonstrations with preferences: show two answers and have a human pick which is better, train a reward model (Bradley–Terry), then optimize the policy with PPO, adding a penalty term to prevent over-optimizing the reward. Binary rewards are too sparse—you learn which answer won, not by how much.

DPO skips the RL loop: increase the likelihood of preferred answers and decrease that of non-preferred ones. Under standard assumptions the optimum matches PPO's, but it's far simpler to implement, so it's become the open-source community's default.

The economics matter more than most people think: humans' self-consistency on binary preference tasks is only about 66%, human annotation costs about $300 per 1,000 items, while LLM annotation is about 50x cheaper and agrees with the human majority more often. The bias to watch for: annotators prefer longer answers, so models learned to be verbose—that's where ChatGPT's chattiness comes from.

The Bradley–Terry model is a statistical model for pairwise comparisons, assuming each option has a latent "strength" parameter, with the probability that one option is preferred related to the difference in their strengths. In RLHF, it's used to turn human preference annotations into a learnable reward function, so the model can generalize to unseen answer pairs.

PPO (Proximal Policy Optimization) is a reinforcement learning algorithm that ensures training stability by limiting the magnitude of policy updates. In RLHF, it's used to fine-tune the language model based on reward model feedback, while a KL penalty term keeps the new policy from drifting too far from the original model, avoiding reward hacking.

DPO (Direct Preference Optimization) is a preference optimization method that requires no reinforcement learning loop. Through a mathematical transformation, it converts the RLHF objective into a simple classification loss: directly increasing the likelihood of preferred answers and decreasing that of non-preferred ones. Because it's simple to implement and stable to train, DPO has become the mainstream choice for fine-tuning models in the open-source community.

Evaluation: Where Claims Quietly Expire

• Validation loss isn't comparable—PPO and DPO have different objectives, so losses can't be compared across them.
• Perplexity loses meaning after PPO—the model is a policy, no longer a distribution over text.
• Chatbot Arena is the most trusted signal, but it's slow, expensive, and biased toward technical users.
• LLM-as-judge correlates about 98% with Arena, costs under $10, and returns results in under 3 minutes—but has known length bias and preference for its own style.

So before believing any benchmark number, ask three questions: which benchmark, for whom, and who's judging.

Systems Engineering: The Part Nobody Tweets About

Mixed precision (store weights in 32-bit, compute in 16-bit, update back to 32-bit) and operator fusion (torch.compile makes training about 2x faster by passing data through GPU memory once instead of once per operator). This work isn't sexy, but it often determines whether a training run can finish at all.

Consensus and Disagreement Under Multi-Source Cross-Verification

According to several tech creators' interpretations of the lecture on Bilibili and YouTube, as well as the original note itself, several facts reach consensus: data and evaluation are the real battlegrounds for industry; SFT mainly shapes format rather than injecting knowledge; DPO is replacing PPO as the open-source default due to its simplicity.

Disagreement arises on evaluation: one camp argues that LLM-as-judge's high correlation is enough to replace human Arena in most scenarios, while the other (including the original note's cautious wording) emphasizes that its length and style biases will be amplified in production environments, so it can only serve as a fast-iteration signal rather than a final arbiter.

A supplementary point from a single source: the note's author explicitly labels these as unofficial notes on a 2024 lecture, preserving the speaker's hedged wording rather than upgrading it to fact, and explicitly excluding newer methods like long context, reasoning models, and synthetic data pipelines from coverage.

The full index is open-sourced on GitHub (MIT license), with a timestamp on every claim, so you can verify it yourself instead of trusting a secondhand summary.

Background

Chinchilla scaling laws come from a 2022 DeepMind paper, whose core conclusion is: under a fixed compute budget, model parameter count and training token count should scale roughly in a 1:20 ratio to achieve optimal performance. Earlier models (like GPT-3) had far more parameters than training data, and were called "data-starved." This finding directly drove subsequent models to generally increase data scale, and made the quality and quantity of data pipelines a focal point of industry competition.