The Essence of LLMs: How Next-Token Prediction Writes Code

You use AI to write code every day, but have you ever wondered: what exactly is that large language model (LLM) on the other side?

The short answer is: it's a program that keeps adding the next word. It reads the existing text, figures out the most likely next word, adds it, and repeats. That's the whole job—and yet it's enough to write code, answer questions, and fix bugs. This article uses a snippet of Python code to walk you through this loop.

What's actually happening at each step?

Look at a half-written piece of Python code, with the cursor sitting after return:

def add(a, b):
    return

What comes next? If you write code, you barely have to think: a + b. The function is called add, the parameters are a and b, and a + b is the natural choice.

The LLM does the same thing. It reads all the text before the cursor and scores every possible continuation: a + b scores high, a - b a bit lower, None lower still. Then it picks one and adds it. Anthropic's official glossary says the same: the model behind Claude is trained to predict the next word based on the preceding text.

How does one word become an entire function?

A single word doesn't get you far on its own. The trick is the loop: the model appends the word it just predicted to the end of the text, then uses the now-longer text to predict the next word.

After adding a, the text ends with return a, and the most likely next fragment is +. Add +, and it predicts b. Then it judges that the function is complete and stops. The entire function, the entire answer, is built up word by word this way. The "typing effect" you see in the chat window is this loop in motion. Strictly speaking, the unit of each prediction isn't necessarily a complete word—it could be part of a word or a symbol. This unit is called a token. For simplicity, the original text just says "word" throughout.

A token is the basic unit of text that a language model processes, and it's not exactly the same as a "word." A token can be a complete English word, part of a word (like "un-" or "-ing"), a punctuation mark, or a Chinese character. Different models use different tokenizers; for example, the GPT series commonly uses the BPE (byte pair encoding) algorithm, keeping common words whole and splitting rare words into subwords. What the model actually outputs is a probability distribution over the next token, not a directly output "next word." This design lets the model handle arbitrary text—including symbols and indentation in code—while keeping the vocabulary size in the tens of thousands. Understanding that tokens exist helps explain why models sometimes "misspell" words or make mistakes in code indentation—what they see is a sequence of tokens, not a stream of characters the way humans see it.

If it only predicts the next word, how can it write code?

You might be thinking: isn't this just the autocomplete on my phone's keyboard?

The difference is how accurate the guess needs to be. To predict a + b after return, the model has to know that add means addition and how Python functions are written. In longer code, it also has to remember which variables were defined earlier and what each function does.

During training, the LLM read enormous amounts of text and code. The only way to predict the next word well is to pick up the patterns along the way: syntax, common sense, how code is written. Georgetown University's research center CSET explains it this way: a simple task like predicting the next word drives the model to develop a broad understanding of language, and that understanding transfers to many other tasks.

There's something even more surprising: the model doesn't work purely one step at a time. In 2025, Anthropic published a study looking deep inside Claude to see how it "thinks," and two findings are worth noting.

The first is about writing poetry. Researchers asked it to write a rhyming couplet starting with "He saw a carrot and had to grab it." If it really predicted one word at a time, you'd expect it to look for the rhyme only at the end of the second line. But in fact, before writing the second line, it had already settled on ending with "rabbit," then wrote toward that goal: "His hunger was like a starving rabbit."

The second is about addition. Asked what 36 plus 59 equals, it said 95. Asked how it calculated that, it described the school method: add the ones digits first, then carry. But when researchers observed what was actually happening internally, they found it ran two paths simultaneously—one estimating the rough magnitude of the answer, the other precisely computing the final digit. The model itself didn't know it worked this way.

Put these two points together and the picture becomes clear: word-by-word writing is how the model works on the outside; to predict accurately, it has developed its own methods on the inside. These methods weren't written in line by line by anyone—the model learned them during training, and even the company that built it needs dedicated research to see them.

Pre-training is the first stage of LLM training: the model repeatedly practices "predicting the next token" on massive amounts of unlabeled text, adjusting billions or even hundreds of billions of parameters through gradient descent. This stage is extremely compute-intensive, typically requiring thousands of GPUs training for weeks. After pre-training, the model has broad language ability and world knowledge, but doesn't yet know how to "be an assistant." The subsequent assistant training usually includes supervised fine-tuning (SFT, using human-annotated question-answer pairs to teach the model to imitate helpful responses) and reinforcement learning from human feedback (RLHF, using a human-preference reward model to guide the model toward more desirable outputs). Anthropic's training of Claude also includes a similar "helpful, honest, harmless" alignment process. It's this stage that shifts the model from "continuing text" to "following instructions," and it's also an important source of behavioral differences between models from different vendors.

Why does it follow instructions like an assistant?

The training described so far—reading massive amounts of text, practicing next-word prediction—is called pre-training. A model that has only been pre-trained is good at continuing text but not at answering questions or following instructions. Anthropic's glossary says this directly.

So there's another round of training afterward that teaches the model to be an assistant: understanding your question, answering appropriately, and refusing to do things it shouldn't. The Claude we use day to day has gone through this round. In the glossary's words, Claude isn't a bare language model—it has been trained to be a helpful assistant. How this round of training works is another topic, and it's the next article in this series.

A context window is the maximum number of tokens a model can read in a single inference, including your question, the code you paste, the conversation history, and system instructions. Modern models' context windows range from tens of thousands to hundreds of thousands of tokens, equivalent to dozens to hundreds of pages of text. When generating each token, the model can only "see" what's inside the window; text outside the window isn't considered. Therefore, putting relevant code, error messages, and clear instructions inside the window is key to improving output quality. Also, when generating each token, the model either samples independently or takes the highest-probability token (depending on the temperature parameter). It has no mechanism to "go back and revise" what it has generated, which is why it can produce code that looks plausible but is actually wrong—it's just continuing the most likely token sequence, not running or verifying the program.

What does this mean when you use AI to write code?

Once you know the model is continuing text, two things immediately become useful.

First, everything you give it is part of the text it's continuing. Your question, the code you paste, the instruction files in your project: it reads it all and starts writing from there. The clearer that text is, the easier it is for the model to continue correctly. Compare "write me a function" with "write a Python function called add that takes two integers and returns their sum"—the latter leaves far less room for guessing.

Second, what it writes is "the most likely thing to come next," not something it has verified. Most of the time, the most likely answer is also correct. But occasionally it will generate a function name or parameter that looks completely plausible but doesn't actually exist. So before relying on it, run the code and read it.

Does the same apply to Codex or DeepSeek?

Yes. Codex runs on OpenAI's models, and DeepSeek^Ad runs on models DeepSeek trained itself. They're all LLMs, and they all work by continuing text word by word. What differs is what they read during training, how they were later trained to be assistants, and what the tools wrapping them do on their behalf. Whichever tool you use, the two habits above apply.

Summary

• An LLM is a program that keeps adding the next word: read the existing text, pick the most likely next word, repeat.
• To predict accurately, it picks up syntax, common sense, and how code is written during training—and can even plan ahead.
• Everything you give it becomes the text it continues, so the clearer it is, the better the result; check the code it writes before using it.

One-line Summary

An LLM is essentially a next-word prediction program that writes code and performs other tasks by continuously continuing text in a loop.

Paragraph Summary

The article uses the example of completing a Python function to explain the core mechanism of LLMs: it reads existing text, scores every possible continuation, picks one to append, and repeats. Although it only "predicts the next word," to predict accurately the model masters syntax, common sense, and how code is written during pre-training. Anthropic's research also found that the model internally plans rhyme ahead of time and estimates addition in parallel, showing that word-by-word output is backed by more complex internal strategies. After pre-training comes assistant training, which turns the model from a text continuer into an instruction-following assistant. For users, this means providing clear context is essential, and because the model outputs what's most likely rather than what's verified, code should be run before you rely on it.