What Does LLM Training Actually Do? The Underlying Logic from Random Noise to Intelligence

The model you chat with was never truly "written." Not a single line of its behavior was authored by a human. Llama 3.1 8B contains 8,006,464,512 numbers (according to Meta's official model card), and no human chose any one of them. Everything it does—grammar, reasoning, confident mistakes—is just what these numbers happen to produce. So where do these numbers come from? A loop, run billions of times.

The Training Loop: From Random Noise to Prediction Machine

Training starts with a pile of text and a model whose numbers are pure random noise. The engine reads a passage of text and repeatedly predicts the next token. It then compares each prediction with the token that actually appeared and scores the deviation. This score is called the loss; for language models, it's typically cross-entropy loss, which penalizes confident wrong guesses more heavily than hesitant ones.

Note that the "teacher" here is the text itself. No one labels anything. The next token that actually follows is the label. This is why it's called self-supervised learning, and why it scales: the internet is the answer key.

Each step of this loop works as follows: predict, score, backpropagate, update. You do this on a batch of text, then the next batch, then the next. The whole process is that simple.

Self-supervised learning is the key to scaling LLM training: it doesn't require human annotation but instead constructs supervision signals from the text itself—given the preceding context, predict the next token. This contrasts with traditional supervised learning, which relies on human-labeled tags (e.g., "this email is spam"). Self-supervised learning leverages the vast amounts of text readily available on the internet, where every sentence implicitly contains thousands of prediction tasks, providing nearly unlimited high-quality training data. This property—"the answer key is in the data"—enables models to learn from trillions of tokens without any human labeling cost.

The Core Challenge: Eight Billion Knobs, One Error

When the model makes a mistake, the error is just a single number at the output, but inside the model there are billions of numbers. Which one caused this deviation? Every number is a weight, a knob that shapes what the next layer receives. Eight billion knobs, one bad prediction. This is the credit assignment problem: eight billion suspects.

The solution is older than deep learning itself. For each weight you want to adjust, you need a signed number: the gradient, which answers "if I turn this knob up a bit, does the error go up or down, and how fast?" Weights are arranged in layers, with each layer's math feeding into the next, and calculus happens to have a rule for exactly this shape of problem: the chain rule, which says the sensitivity of the whole pipeline is the product of sensitivities along the path. By computing these sensitivities layer by layer starting from the output, while reusing each layer's computed arithmetic, you can get the gradient for every weight at roughly the cost of one forward pass. The error propagates backward through the network—this is where the name backpropagation comes from. The 1986 Nature paper that made this method famous was titled "Learning representations by back-propagating errors" (Rumelhart, Hinton, and Williams).

A brief history: the 1986 paper didn't invent this from nothing. Reverse-mode automatic differentiation dates back to Linnainmaa (1970), and Paul Werbos applied it to multilayer networks in 1982. Rumelhart and colleagues independently rediscovered it, unaware of the earlier work, and their Nature paper is the one the field remembers. This same line of thought later earned Geoffrey Hinton a share of the 2024 Nobel Prize in Physics.

The chain rule is a fundamental rule in calculus for computing the derivative of composite functions: if y = f(g(x)), then dy/dx = f'(g(x)) * g'(x). In neural networks, each layer's output is the next layer's input, and the entire network is one giant composite function. The chain rule tells us that the sensitivity of the loss to a given weight equals the sensitivity of the loss to that layer's output multiplied by the sensitivity of that layer's output to the weight. Starting from the output layer and computing these local sensitivities backward layer by layer and multiplying them together efficiently yields the gradients for all parameters. This is the core of backpropagation, and it avoids the enormous overhead of computing gradients by individually perturbing each parameter.

The Update Step: Descending in a Billion-Dimensional Space

Once every weight has a gradient, you move them all a little: new weight = old weight - learning rate × gradient. The learning rate is the step size. Too large, and the model jumps over good regions and starts getting worse; too small, and it crawls forever. This descent idea is ancient; gradient descent is usually credited to Cauchy, 1847. Training is walking downhill in a billion-dimensional space, where "down" means "the next word matches the text better."

Gradient descent is an iterative optimization algorithm used to find the minimum of a function. The basic idea is: at the current point, compute the function's gradient (i.e., the direction of steepest ascent), then move a small step in the opposite direction, repeating until convergence. In LLM training, the objective function is the loss function (measuring the difference between predictions and the true next token), and the parameter space has billions or even trillions of dimensions. Although high-dimensional spaces are hard to visualize, the mathematical properties of gradient descent guarantee that, with an appropriate learning rate, the loss will gradually decrease. The choice of learning rate is critical: too large may cause oscillation or even divergence, too small leads to slow convergence. Modern optimizers like Adam further improve efficiency by adaptively adjusting the learning rate for each parameter.

A Concrete Example at Scale: GPT-3's Public Record

The clearest public record of this loop at full scale is the GPT-3 paper (Brown et al., 2020). From its tables and appendices, we can quote directly:

• 175 billion weights, arranged in 96 layers.
• Each update sees a batch of 3.2 million tokens.
• Training ran on a total of 300 billion tokens. 300 billion divided by 3.2 million is roughly 94,000 updates. The model's entire education is 94,000 fine-tunings, each adjusting 175 billion knobs.

From the 1986 paper to 2020, this loop gained many tools, all documented in the same appendix:

• Adam optimizer (beta1 = 0.9, beta2 = 0.95): keeps a running memory of each weight's history and scales each step individually. Momentum plus a per-knob step size, rather than a fixed step size for everyone.
• Gradient clipping to 1.0: if a batch produces an unusually large gradient, it gets truncated. One ugly batch can't knock the whole model off course.
• Learning rate schedule: warmup for the first 375 million tokens, then cosine decay to 10% of peak over 260 billion tokens. Confident, cautious steps early; small, slow steps late.
• Weight decay 0.1, a gentle pressure preventing weights from growing without bound.

Modern practice is the same loop with better tools. Llama 2 (2023) used AdamW, beta1 = 0.9, beta2 = 0.95, weight decay 0.1, gradient clipping 1.0, and was pre-trained on 2 trillion tokens. The Llama 3 flagship (2024) ran this loop on 15.6 trillion tokens, roughly 50 times the compute of the largest Llama 2. The 1986 recipe, industrialized.

The Adam optimizer (Adaptive Moment Estimation) combines ideas from momentum and RMSProp, maintaining two exponential moving averages for each parameter: the first moment (mean of gradients, similar to momentum) and the second moment (mean of squared gradients, used to scale the learning rate). During updates, the parameter change depends not only on the current gradient but also on the magnitude and direction of historical gradients. This makes Adam perform excellently under sparse gradients and non-stationary objectives, making it the default choice for training LLMs. Here beta1 and beta2 control the decay rates of the two moving averages. AdamW is an improvement over Adam that decouples weight decay from the gradient update, more effectively regularizing the model.

What Does Training Actually Produce?

After all these updates, no single weight has meaning. No knob is labeled "knows the capital of France." A fact, a rule, a style—all are smeared across billions of weights and only exist when the entire network runs. This aligns with the earlier point about hallucinations: knowledge in LLMs is compressed into weights, not filed away somewhere. Training doesn't install facts. It adjusts the model until its next-token guesses are statistically consistent with the text it has read.

This is also why the model can't check its own answers. There's no lookup table to consult, no correctness certificate, only patterns. And this is why confident errors happen: tuning optimizes for text that sounds right, and sounding right isn't the same as being right.

So the whole story is: LLMs aren't written or taught. They're tuned. Random noise, turned 94,000 times (at GPT-3 scale) or fed 15.6 trillion tokens of text (at Llama 3 scale), until predictions align with reality. The results are remarkable, and the mechanism is a loop so simple that a 1986 paper title still describes it precisely.

Key Number Verification

To ensure accuracy, the following numbers have all been traced to original sources:

• Llama 3.1 8B parameter count 8,006,464,512 (Meta model card)
• Rumelhart, Hinton & Williams, "Learning representations by back-propagating errors", Nature 323, 533-536, 1986
• Backpropagation history: Linnainmaa 1970 (reverse-mode automatic differentiation), Werbos 1982 (multilayer networks), Rumelhart independently rediscovered
• Hinton shares the 2024 Nobel Prize in Physics
• Cauchy 1847 first proposed gradient descent
• GPT-3 (Brown et al., arXiv:2005.14165): 175B parameters, 96 layers, batch of 3.2M tokens, total tokens 300B, Adam beta1 0.9 beta2 0.95, gradient clipping 1.0, cosine decay to 10% over 260B tokens, linear warmup 375M tokens, weight decay 0.1
• Llama 2 (Touvron et al., arXiv:2307.09288): AdamW, beta1 0.9 beta2 0.95, weight decay 0.1, gradient clipping 1.0, pre-trained on 2T tokens
• Llama 3 (Grattafiori et al., arXiv:2407.21783): flagship 405B model pre-trained on 15.6T text tokens, approximately 3.8e25 FLOPs, nearly 50 times the largest Llama 2

This article is based on a technical analysis by independent AI creator Kai, with all data traced to original papers and model cards.